Download this free, 114-page End-of-Line Equipment Playbook jam-packed with strategies for success, best practices, and pitfalls to avoid.  Learn more >>
Article |

Robot gains 'eyesight'

Guided by a simple, compact vision system, a robotic depalletizer picks three 65-lb bags of seed at a time, handling even the most irregular loads without failing.
Print Reprint
FILED IN:  Controls  > Strategy
These images, taken directly from the camera, show a neat layer and an irregular layer. The pallet on the right shows how skewedThese images, taken directly from the camera, show a neat layer and an irregular layer. The pallet on the right shows how skewedThe vision system1s electronics are compact enough to be contained within the housing of the camera itselfA compact camera (above) helps direct the robot to the precise location of the bags. Each pickup head is positioned on a spring-A compact camera (above) helps direct the robot to the precise location of the bags. Each pickup head is positioned on a spring-

What separates a robotic packaging machine from a non-robotic machine is that the payload's trajectory through three-dimensional space can be programmed and if necessary reprogrammed.

However in the town of Algona in northwest Iowa a new kind of robotic packaging application is at work depalletizing heavy bags of seed. Instead of working off of solely preprogrammed trajectories this robot receives the precise coordinates for the pick-up location of each bag in real-time from an inexpensive machine vision system.

The plant is a production facility of Des Moines IA-based Pioneer Hi-Bred Intl. a worldwide supplier of seed to farmers. Pioneer has 73 seed conditioning and packaging facilities in 23 countries around the globe.

The robotic depalletizer at Algona is actually on a repackaging line. At the end of every growing season in compliance with federal regulations Pioneer recalls all unsold seed tests for spoilage and re-bags good seed for the next selling season. The depalletizer is used at the start of the line placing the returned bags on a conveyor that leads to an automatic bag slitter. The seed is then reclaimed for repackaging.

Related Sponsored Content

Bags consist of sewn open-mouth gusseted multi-wall paper/plastic structures whose specific constructions vary depending on bag size. Algona and seven other Pioneer production facilities like it across the U.S. processes 500 bags this way every year.

The depalletizer was built and installed by robotic systems integrator Automated Concepts (Council Bluffs IA) and uses a model M410-IWX heavy-duty four-axis robot with RJ-2 controller from Fanuc Robotics (Auburn Hills MI). The vision system is DVT's (Norcross GA) SmartImage Sensor. The depalletizer in production since August has freed up two to three workers who were required to depalletize the bags manually.

Several things make this installation unique:

* The robot picks up three bags at a time. "No one in the industry was moving more than one bag of this size at a time" says Glen Burgess Pioneer's project manager who initiated the project. "And no one was doing it close to the total throughout that I needed which was twenty bags per minute."

* While vision-guided robots are unusual in packaging they're not altogether new. However this is one of the first robotic motion control applications for the SmartImage Sensor which DVT markets as an inexpensive type of machine vision. Both Pioneer and Automated Concepts confirm that the DVT unit was much less costly than other systems they evaluated: about $6 versus $20 to $60 for others. Plus it was comparatively easy to set up.

* With vision the robot is able to handle even the most irregular pallet loads no matter how skewed the bags are with an uptime of better than 95%. In fact the robot's ability to handle irregular loads "far exceeded" Pioneer's expectations says Burgess accommodating even the poorest-quality pallet loads.


To achieve the desired 20 bag/min speed Burgess determined that three bags needed to be picked each cycle. Fanuc "was the only one that would meet the payload weight and the cycle time requirements" says Brian Turner general manager at Automated Concepts which works with several robot manufacturers. Automated Concepts also favors Fanuc "because of reliability and ease of programming" he adds.

Tough to pick up

Although the robot could physically handle three bags by weight the challenge was how to actually pick them up. "If these were corrugated boxes it would have been a no-brainer" says Pioneer's Burgess.

Boxes because they have defined uniform angles can be picked up with end tooling that grips. But bags aren't uniform so gripping was out of the question. Pioneer felt the only way to pick up bags was with a vacuum cup. Because Pioneer wanted to pick cup three at once most vendors shied away from it. That's because the top surface of each bag isn't necessarily at the same exact height depending on how the bags have been stacked and how the product has shifted inside the bags.

Pioneer and Automated Concepts "did three months' worth of experimentation with the vacuum cup end tooling" says Burgess. "We went through probably four different sets of end tooling and cups before we got one to do what we needed with a paper bag."

The solution: Each cup is on its own spring-loaded floating housing that gives each cup a few inches of vertical play. Also the cups have "a special kind of a gasketing material that helps them to conform with the top of the bag" says Burgess. The three cups each shaped like an ellipse are at a fixed distance from one another.

"When we finally got the ability to allow the cups to move vertically as they contacted the bag we were able to really start picking these things up" says Burgess. The vacuum goes through the outer kraft paper layer and locks onto the 0.5-mil layer of high-density polyethylene in each bag.

However Pioneer soon found that although the robot could pick up three bags it could only do so if the vacuum cups were positioned in the exact center of each bag. There was little room for play.

The problem wasn't the machine it was the pallet loads themselves. "Inconsistent" is an overly polite word to describe them. Pioneer first manually sorts returned product by hand so many pallet loads are built by hand. That often means bags are skewed and loosely positioned.

Also the fill volume inside each bag varies significantly. Pioneer's seed kernels can vary in size each year so bags are sold by count-typically 80-not by weight. Bag sizes too can vary on a pallet depending on the bulk of the seed inside.

Because of extreme variations in pallet loads it was all but impossible for a traditional automated robot operating off of preprogrammed trajectories to handle. The project had hit a wall. Traditional control techniques wouldn't suffice for this application. It became clear that the robot needed to be able to see.

A new type of vision

After evaluating a number of vision systems Pioneer and Automated Concepts chose DVT's SmartImage Sensor primarily for its low cost and ease of integration. Until now typical packaging applications for DVT's unit have consisted of fixed-focal-length on-line inspection.

The DVT unit's electronics are so compact they're housed inside the camera itself which is mounted directly overhead on the 20' H ceiling. No other computer or processing component is needed only an RS-232 port with a serial cable that connects the camera right to the robot's controller.

The vision system doesn't just snap a picture of the pallet from overhead and magically spit out the coordinates. It needs a little help in the form of visual cues. The DVT system looks for two things: the Pioneer logo on each individual bag and the letter "N" in the Pioneer name which is below the logo.

The vision system then determines whether the line that's described by those two points is straight or angled. Finally by comparing the location of those points with the four points described by the four corners of the pallet it has all the information it needs to determine skew and positioning for each bag. What the vision system lacks however is depth perception meaning it can't tell by itself the height of the current layer that's being depalletized.

The solution was a photoeye on the end of the palletizing arm. For each picking cycle the arm is programmed to descend until it touches the topmost layer of bags triggering the reflective sensor. "Where it meets the bag we've used mathematics to say 'Okay that's the top of the load'" says Burgess.

Based on that height the vision system and robot controller know that each "pixel" or point in the digital image translates into a given number of millimeters of length and width at that height. From there the actual coordinates can be calculated for each bag. All this computation occurs inside the DVT unit.

Once the coordinates are determined the vision system unit transmits them in a 48-character string to the robot's controller. Then the controller decides if the bags are close enough together and the skew minimal enough to permit three bags to be picked up at once. If not the robot picks bags one at a time guided by the DVT unit's coordinates. Then another attempt is made with the next group of three.

As Packaging World went to press Pioneer had just finished the acceptance test. With vision the robot performed flawlessly during the test. "We went through twenty-two pallets of product with sixty-six bags on a pallet and we had only two instances where the robot was forced to pick up three bags individually" says Burgess. "Our hit rate was just phenomenal." Even better the system did not "alarm out" due to poorly built loads says Burgess unlike before.

Although it's not necessary to display the DVT camera images in order for the system to work Pioneer opted to connect a laptop computer to the system just so the operator can view the images in case the system shuts down due to an error. That does happen acknowledges Burgess when the vision system encounters bags that are obscured by dust wrinkles or tears. The operator will either try to brush off or straighten the bag so it's visible to the system or if that's not possible remove the three bags by hand.

Programming and integration time wasn't excessive according to Automated Concepts and DVT. "The coordinate translation tool is one of the built-in capabilities of the system" says DVT's Phil Heil applications engineer.

For its part Automated Concepts wrote the program for the Fanuc controller that would translate the DVT unit's coordinates into positioning instructions for the robot. But still says Turner it was a fairly straight-forward process. Even the special lighting often required with machine vision wasn't necessary in this situation. Pioneer simply added another metal halide light fixture for extra illumination the exact same type of light that's used in the rest of the plant.

"We've worked with vision before" Turner says "but I've got to tell you this is one of the cleanest most cost-effective systems that we've come across."

The current speed of the vision-guided robot is close to six cycles/min or 18 bags/min a figure that's acceptable to Pioneer. That speed is expected to improve with future refinements to the system says Burgess.

Future plans

Though the system is up and running Burgess says it's just the start. Two more Pioneer facilities are scheduled to receive similar robots next year. Plus he believes additional speed can be picked up by changing the way the robot acquires the height dimension. Now the arm swiftly returns to the previous picking height then goes into a slow creep mode until it taps against the next bag layer.

"I'd much rather have another camera then we could look at all sorts of different things and know the exact location of that bag" says Burgess. That will be addressed early next year.

Burgess says both the robotics integrator and vision company put in extra effort to make a success out of what proved to be an incredibly difficult application. "I spent two years having [other vendors] telling me I can't possibly lift bags with vacuum at this rate saying that the vacuum wouldn't hold that the acceleration would knock it loose that the deceleration would knock it loose that no one had ever done it a whole gamut of excuses" Burgess says. "To ACI's credit they decided it would be a good project to take on. I've received superb support from both ACI and DVT" he concludes.

The 2014 Packaging Trends Report
Sign up to receive timely updates from our editors and download this e-book consisting of our editors' picks of most notable package designs. Updated for 2014!


Don't miss intelligence crucial to your job and business!
Click on any newsletter to view a sample. Enter your email address below to sign up!
Each newsletter ranges in frequency from once per month to a few times per month at most.