Augmented Reality Transforms Press Repairs

It’s your busiest time of the year. You’ve got a mountain of jobs in the queue. Your operator fires up the press to get started — and the warning lights come on. A sensor is out. A part needs replacing. Production screeches to a halt. And your mind slowly fills with dread as you think about all the angry calls you’re about to get from customers as you wait for a technician to come in and fix your press.

Thanks to augmented reality (AR), those days are virtually over. A growing number of digital and offset press manufacturers are moving to AR-based servicing to guide operators expertly through system repairs, without the need for a service technician to travel to the site. They are utilizing smartphones, tablets, and headsets to show users, through arrows, annotations, and animations, what to do and which parts to replace, so they can get up and running as soon as possible, and avoid losing money by waiting for service.

“With the remote support, you’ll be able to get going instantly,” says Matthias Naumann, remote maintenance manager at Koenig & Bauer, which offers Visual ServiceSupport to link users with technicians. “In about 80% of the cases when a customer calls us, they will be able to resume production in some way.”

"It saves time for the customer. It allows them to be more productive," notes Todd Miller, Ricoh’s director of centralized services. “They don’t have to wait a couple days for one of our specialists to arrive.” Ricoh partners with Help Lightning to provide remote service using AR.

“It’s about getting that customer up and running on their device faster. Not having to send somebody onsite,” says Abby Robinson, director of customer success with CareAR, which was acquired by Xerox in 2021 to provide remote AR visual support.

“This is a technology that will be adopted by everyone in … a couple of years time,” says Ayush Jain, Ideas Incubator, Engineering Systems (xRServices) at HP, which uses AR headsets to provide mixed reality visual guidance to users of HP xRServices. “It is the future of services.”

AR Headsets

AR is transforming the servicing and maintenance of equipment. Some manufacturers include it in service contracts, others offer it as a subscription service, but the reality is it quickly pays for itself by getting machines running again so they can continue making money for their owners.

While most manufacturers that offer this service utilize smartphones and tablets to communicate, many are adding smart glasses to keep operators’ hands free while they follow instructions. HP xRServices uses headsets exclusively for its subscription service (about $400 per month). Jain says using the headsets is a very simple process. “If you have a problem, all you need to do is put on the headset, press a button, and you’re ready,” he says.

If the HP press shows an error code, the user scans the associated QR code and holographic instructions appear as a ghost image over what the operator is seeing, directing him or her to manipulate a control and remove a part. Arrows appear to indicate which direction to turn a knob. Animations overlay the physical image. The remote support engineer on the other end can send pictures to show the correct position of a valve.

“It’s fully animated so he can just follow the steps,” remarks Jain. “Because the guidance is so immersive, it almost feels like the remote engineer is onsite.”

If there’s no error code, the remote support engineer talks with HP technicians to troubleshoot. They see what the user is seeing and guide him by circling controls they want him to manipulate. Parts that are customer replaceable and onsite can be installed with guidance, courtesy of the headset.

“If, in the troubleshooting, they identify a part that’s not available, all they need to do is open a case to order the part,” Jain says. “When the customer receives the part, he can resume the procedure.” By inputting the part number, the instructions automatically come up in the headset. “Any procedure that is not a safety hazard is now game for xRServices,” Jain adds.

‘Back Up and Running Faster’

Other companies are using smart glasses to connect operators and support staff. Xerox CareAR uses products from RealWear and Lenovo (following the discontinuation of Google Glass), in addition to mobile devices.

“When a press user calls into the Xerox welcome center for this, they use CareAR Assist to see the customer’s environment … and help troubleshoot an issue, all in the hopes of solving it remotely to get the customer back up and running faster,” says Robinson, who adds that the service is included in a Xerox service contract.

Technicians can draw arrows or circle a part, and that image is anchored in place even as the customer moves the device. If the problem can’t be fixed by the operator because a part is needed, the technician will know which part to bring because he or she has seen the problem.

“Seeing something is worth 1,000 words,” she says. “It’s just a better collaborative experience.”

Because customers must purchase the smart glasses, many opt to use their phones or a tablet, although this does limit their ability to manipulate the press. Accessing the service is very simple, Robinson says.

“You don’t have to log in as a guest, you don’t have to have a username or password, or create an account,” If they don’t have the app they can join a session by clicking on a link sent in a text message.

“They get connected right away,” she adds. “No data is shared, and there’s high security and privacy.”

Customers who want to solve problems themselves can opt for the CareAR Instruct platform. Scanning a QR code launches AR-based, step-by-step guidance. Graphical indicators draw attention to “hotspot” locations where text or video detail is presented when tapped. Animations guide users through clearing paper jams, for example. Holographic images float over the video image of a machine part and can include prescribed motion.

Through Ricoh’s Help Lightning technology, an onsite person makes an adjustment guided by a remote technician, who can see what the person is seeing through smart glasses. | Credit: Ricoh

Ricoh’s partnership with Help Lightning for AR-based service also makes use of smart glasses (from Vuzix), however they’re used mostly for communication between a remote expert and an on-site technician. But as customers continue to show interest in a shared services model, Miller notes, Ricoh wants to start providing them with glasses. “We see that as an offering coming shortly,” he says.

Currently, customers use a smartphone or tablet to access service. The remote service person can insert his or her hands into the image the customer sees on-screen to point out a problem area and demonstrate how to manipulate a control.

“We can point out things, we can draw on it, we can freeze the frame and zoom in,” Miller explains. “The biggest benefit is working on the customer-replaceable units. We can watch them pull it out of the box, show them which part goes first, how to do the calibrations, and move onto the next one.”

AR in the Offset World

Offset press manufacturers are using AR, as well. For example, Koenig & Bauer’s Visual ServiceSupport lets a virtual technician see what the customer sees through a mobile device. The technician can apply markings to the images, for example, which bolt to turn or which direction to pull a part. Operators can also share images showing certain press behaviors that only occur sporadically.

“We strive for getting the customer as far along as possible without having an onsite visit,” says Naumann. If a part is needed, it will be shipped. “They can call us back the next day and we can help them put it in,” he adds.

Even if a technician has to visit the plant to make a repair, Visual ServiceSupport provides a head start; the service tech knows what part to bring and doesn’t have to re-diagnose the problem upon arrival, drastically cutting service time, Naumann points out.

AR for Site Prep, Training

Some press manufacturers are using their AR services for more than just remote support. HP xRServices headsets are also used to prepare sites for new equipment (customers can place an image of the new press in their shop, get measurements, and create a blueprint); provide training (using visual guided instructions); and handle maintenance (replacing manuals). Jain notes HP xRServices has an installed base of 300 headsets with a target of 700 by the end of the year.

Miller says Ricoh is also looking into using AR to do customer training. “We can start the training quicker,” he remarks.

Providers say AR services did not come about because of a shortage of technicians, but more so out of a desire to get customers’ equipment back up and working quickly.

“For every hour the press is not printing, the customer is losing revenue, and HP is losing revenue,” points out Jain. “So, for us it is important that the client doesn’t lose time in getting back up.”

What’s more, not having to transport a technician to the site of each breakdown brings savings for everyone. “This is helping us reduce a lot of travel,” affirms Jain — thus reducing the carbon footprint for both HP and the customer.

At Ricoh, the use of AR for service started as a way to support less experienced technicians out in the field. “Once we started using it, and saw its benefits and how successful it was, we’re like, ‘This can definitely help customers that want to do more self-maintenance,’” Miller says.

Though HP xRServices is a subscription service that costs about $400 per month, Jain insists that cost is made up very quickly.

“If your press is down for an hour more than it should be, your revenue loss equates to the entire cost of the service for a year,” he points out. “The first time you use it, you’ve recovered ROI for the entire program for the year.”