Alexander Ouellet, Innovation Engineer at Harpak-ULMA Packaging, discussed the many “flavors” of augmented reality at PACK EXPO Connects’ Innovation Stage presentation this afternoon, and said that for the packaging industry, AR can guide workers with performing product line changeovers, getting machines up and running, or, it can alert them to performance issues.
“For the first time,” he said, “AR is bringing computing and on-demand information to our front-line workforce. AR is making it possible to more quickly train or provide expert guidance to our workforce and provide them the information they need at the right time and with the correct context. That's where most of the value is, especially as producers look for ways to further optimize production processes."
Ouellet said that AR technologies are best understood as a spectrum of overlapping technologies, rather than distinct items, and that the defining trait of each is how the user is going to experience it. (This article addresses AR technologies used in packaging manufacturing today. For a description of all AR technologies described by Ouellet, see the full presentation here. Available through March 31, 2021.)
Assisted Reality projects an extra layer of information onto the user’s peripheral vision, such as Google Glasses. Ouellet said of assisted reality, “It differs from Augmented Reality in that it doesn't change what the user is seeing nor is it aware of the geometry of the user's environment.”
Augmented Reality (AR) is an overlay of computer-generated content onto the physical world, that can superficially interact with the environment in real-time. With AR, the end-user uses headsets or mobile devices.
Mixed Reality (MR) is an overlay of synthetic content anchored to and interacting with real-world objects in real-time. Said Ouellet, “When implemented well, MR should be a perfect union of digital and physical content interacting seamlessly.”
Ouellet also said there are three distinct types, or flavors, of AR. The one most people are familiar with is Video See-Through (VST) technology, which allows the user to view through a tablet or phone screen. One example is the World Lens app, where the user holds up their phone and it translates signs and language.
The second type of AR is lens-based applications that employ Obstacle See-Through (OST) technology, such as the Microsoft HoloLens. Ouellet said in this technology, “AR information is projected in the lens and your brain puts two and two together and stitches it together to create an augmented view of reality.”
Thirdly is Projection AR, where a projected image creates a direct overlay onto a specially designed work surface. “A projection-based augmented reality system,” said Ouellet, “can provide user instructions or assistance in a variety of media. Projection AR reduces or eliminates the need for computer monitors and screens as the instructions appear directly in the task space. It can be useful for assembly, disassembly, inspections, kitting, tool changeovers, training, and more.”
Ouellet said that AR is an immature market, and “future AR deployment will be driven primarily by industrial applications because that's where the biggest bang for the buck lies.”
Even as a developing market, however, the value proposition of AR is delivering substantial productivity gains to human labor – as much as 30% to 60% depending on the application.
One long-form study at Iowa State found that AR made training faster for complex tasks, and “virtually eliminated human error in those processes,” said Ouellet.
He also cited a study at Lockheed, where AR was adopted as a guidance and instruction tool to attach fasteners to spacecraft and resulted in an 85% reduction in overall training time, a 90+% reduction in the time that it took front-line workers to comprehend tasks, and a 40% increase in overall productivity. “In the end,” said Ouellet, “Lockheed concluded that they saved $38 per fastener in a program that buys over 2 million fasteners every year, that's a staggering value proposition.”
AR does come with deployment challenges, and Ouellet said, “I think it's safe to say the majority of producers today would struggle with a do-it-yourself AR approach.” Infrastructure issues such as Wi-Fi and access to devices, good 3D data, and building out the AR experience in a way that is appealing to the user are some of the challenges mentioned by Ouellet.
Here are a few current use cases for augmented reality mentioned by Ouellet:
AR for remote service: Allows an expert resource to on-demand interactively guide people through a support task, such as being guided through a repair process on a machine with a mobile device or headset, consequently saving travel costs and repairing the machine faster.
Virtual Factory Acceptance Tests (vFATs): Techs use AR headsets to capture key data and processes while the machine is running, including HMI outputs and necessary validation criteria, which is then shared with the customer via live video sessions or AR chalk sessions.
“This approach,” said Ouellet, “can be particularly useful in an automation engagement where the packaging machine may already be in the customer facility and the automation components are being assembled outside of the facility. We can use AR to recreate a combined view if you will.”
AR training solutions: Using AR technology and a process called expert capture, techs create training content around a specific machine - an on-demand library of AR experiences - so that it can be delivered with the machine. Training sequences might include how to perform a film or a tooling change, or guide a tech through diagnosing common faults, or troubleshooting procedures, etc.
Service work instructions: AR is used to provide a real-time worker guidance during operations, which can reduce the chance for error by enforcing checkpoints and confirming that the task was performed properly.
IoT service diagnostics. “This allows staff to view key performance data as the machine is running,” said Ouellet, “including more complex data such as OEE metrics or simple operating sensor data like ceiling pressures or temperatures that could be indicating the onset of a component failure.”
Combining digital twin technology with AR also allows technicians to see how and where internal components are designed to operate without opening the machine up.
Ouellet sees a big picture payoff for AR in the area of predictive maintenance, and points to industries who are already using this technology, such as jet engines. “To accomplish this,” he said, “we'll aggregate massive volumes of IoT data into cloud-based data lakes, that will enable us to benchmark any individual machine's performance against the entire data-set, a useful analysis in its own, but it also means the data can be mined using a combination of machine learning and artificial intelligence to develop the predictive algorithms that will minimize unplanned machine downtime.”
