Using Audio Augmented Reality to Support Decision Making

Prateek Jain, Soussan Djamasbi
In proceedings of the Eighteenth Annual Workshop on HCI Research in MIS, ICIS2019, Munich, Germany.

Consumers rely on nutrition facts label on a food package to get nutrition information. Previous research shows that nutrition facts labels are confusing and hard to understand. We created a decision aid in the form of mobile application which uses audio augmented reality to help consumers in making a healthy decision and improve the overall user experience. Our app scans the nutrition facts label with OCR, uses FDA’s 5-20 rule to determine healthy/unhealthy nutrients, and gives the feedback in either audio augmented reality, static pop-up, or their combination. Our preliminary study with 17 participants shows that app significantly helped users in selecting healthy products. The app is significantly easier to use and helpful in making the decision as well. While comparing results of the three feedbacks, the combination of audio AR and the static pop-up was rated higher. However, the difference is only significant in overall experience ratings.