With Augmented Reality (AR), Virtual Reality (VR) and Mixed Reality (MR) becoming more prevalent in modern society, the need for advancements in MR training and education is increasing. Commercial industries are investing in the area with the MR Market forecast to hit €115.18 Billion in 2020 (Statista, 2018). Collaborating with global engineering company ABB and using the current leading AR Head-Mounted Display (HMD), Microsoft’s HoloLens (Microsoft HoloLens, 2018), we have created an augmented training scenario system that uses gamification to improve the educational experience.
The aim of this project was to create specialised training solutions to real-world problems experienced by ABB (ABB, 2018) engineers. AR was used to improve the efficiency in the completion of tasks without limiting the trainee’s ability to interact with the real world. The project aimed to replace traditional paper/digital manuals with 3D Holograms that allowed the operator to access contextual information on demand. Gamification of simple repair procedures was also implemented, providing positive reinforcement and encouraging a competitive environment between trainees.
In the article we will report on the use of MR technologies such as the Microsoft HoloLens, speech synthesis service Amazon Polly (Amazon Web Services, Inc., 2018) and speech recognition AI Windows Speech Engine. The article will discuss how these technologies were used to create enhanced user interaction and feedback. Technologies that were initially investigated such as Vuforia (Vuforia, 2018), Leap Motion (Leap Motion, 2018) and ARCore (Google Developers, 2018) will also be mentioned along with best practices found to generate 3D assets in mixed reality.
While the findings outline key technical difficulties of implementing AR in real-world environments, positive results include increases in health and safety awareness and instant access to diagnostics that are imperative to operations. We identified that efficiency in the completion of complex procedures can be improved through augmentation. Through analytics and machine-based evaluation, the capability of trainees can be automatically assessed and responded to in real-time, and gamification improves learning through motivation (Aparicio, A.F. et al., 2012).
References
Statista. (2018). Global augmented/virtual reality market size 2016-2021 | Statistic. [online] Available at: https://www.statista.com/statistics/591181/global-augmented-virtual-reality-market-size/ [Accessed 13 Apr. 2018].
Microsoft HoloLens. (2018). Microsoft HoloLens. [online] Available at: https://www.microsoft.com/en-ie/hololens [Accessed 13 Apr. 2018].
Abb.com. (2018). ABB Group - Leading digital technologies for industry. [online] Available at: http://www.abb.com/ [Accessed 13 Apr. 2018].
Amazon Web Services, Inc. (2018). Amazon Polly. [online] Available at: https://aws.amazon.com/polly/ [Accessed 13 Apr. 2018].
Vuforia.com. (2018). Vuforia | Augmented Reality. [online] Available at: https://www.vuforia.com/ [Accessed 13 Apr. 2018].
Leap Motion. (2018). Leap Motion. [online] Available at: https://www.leapmotion.com/ [Accessed 13 Apr. 2018].
Google Developers. (2018). ARCore - Google Developer | ARCore | Google Developers. [online] Available at: https://developers.google.com/ar/ [Accessed 13 Apr. 2018].
Aparicio, A.F., Vela, F.L.G., Sánchez, J.L.G. and Montes, J.L.I., (2012, October). Analysis and application of gamification. In Proceedings of the 13th International Conference on Interacción Persona-Ordenador (p. 17). ACM.
Topics: Digital Technologies in Disciplinary Contexts , Topics: Gamification
