Retail Assistance Systems for visually impaired people

Zeichnung einer Frau, die etwas in ihr Handy diktiert.


Together with our technology partner Microsoft, we have considered how our software can support blind people with a self-sufficient shopping experience. In many conversations, also with people who master their lives self-confidently and self-determined without sight, the following – initially fictitious – case came into being.

We accompany Mia on her shopping trip to downtown Frankfurt:

Spring has arrived; the days are getting longer and warmer, and Mia is drawn back out to the Frankfurt shopping mile to treat herself to a new dress. A visit to her favorite department store is a must on Mia's tour. Mia has had a digital customer card for some time. Together with additional features from Bütema AG, it was integrated into Microsoft's "Seeing AI" app. Customers with a disability enjoy the advantages of the Seeing AI App with the extended customer card functions.

Zeichnung einer Frau, die einen QR-Code auf einer Stele mit ihrem Handy abscannt.

Assistant points have begun to spring up in every corner of the branch. A floor guidance system leads Mia to the next Assistant Point. A short text in Braille on the steles explains that the Assistant Points can send personalized offers and information directly to the customers' smartphones by scanning the QR code. Mia pulls out her cell phone, calls up the Microsoft Seeing AI app and allows the Assistant Point to identify her. Since the app's AI can view Mia's purchase history - both offline and online - the app immediately describes products to Mia that match her profile. Discounted items from the weekly promotion are also included:

"Hello Mia, welcome to our branch on the Zeil. We have great new offers for you that you might like. For example: tunic dress in light blue by comma”.

"Please give me more information," Mia replies. The bot reads Mia the article text about the blue dress. Mia asks the app to call her personal shopping assistant who will pick her up at assistant point number 7.

A few minutes later, Mr. Grünwald, the branch's shopping assistant, comes to meet with Mia. He recognized Mia from a photo that Mia shared with Mr. Grünwald using the MS Seeing IA App using ISA functions.

"Could you please lead me to this dress?" Mia asks Mr. Grünwald and shows him the article on her mobile phone.

Zeichnung einer blinden Frau, die einem Verkäufer ein Handy zeigt.

Having arrived in the women's department, Mr. Grünwald finds the blue dress for Mia as expected. Mia asks for size 38. Mr. Grünwald accompanies Mia to the next changing room. Recently, a holder for smartphones with a flexible "swan neck" has also been installed in the cabin. Mia puts her mobile phone in the holder and first tries on the dress. The fabric and cut feel good, but Mia isn't sure if the dress really suits her. The MS Seeing IA app is still open on her smartphone. "Please call me a styling consultant," Mia tells the app's chatbot.

"Please call me a styling consultant," Mia tells the app's chatbot.

"Hello Mia, I'm Anne," the styling consultant reports shortly after via live video. "How can I help you?".

"Hi Anne. Can you please tell me if the dress suits me?"

"That suits you perfectly, Mia."

"Thank you, Anne."

Zeichnung einer Frau, die ein Kleid hochhält und sich in einem Spiegel betrachtet. Auf einem Handy, das am Spiegel angebracht ist, berät sie eine Modeberaterin per Video-Chat.

After the video chat has ended, Mia asks the MS Seeing IA app to fetch her Mr. Grünwald again. Using the mobile service desk, Mr. Grünwald's ISA and mobile payment, Mia can conveniently pay for her purchase directly with Mr. Grünwald without having to queue at the checkout. Mia pays for the tunic dress and leaves the branch.

Zeichnung einer Frau, die in einem Modeladen lächelnd eine Tragetasche entgegennimmt,
Scroll to Top