Purpose
This paper details a comparative perceived usability study of two comparable natural language processor (NLP) based question answering (QA) systems. The two systems operate over two different modalities – voice and text. The study was completed with two custom built question answering systems, specifically designed for a proposed target user - the Financial Manager.
Design/Methodology/Approach
The study was completed in an exploratory manner within a semi-formal testing environment. An industry and academia accepted measure for perceived usability was used as the measurement method, in the System Usability Scale(SUS).
Research Limitations / Implications
The study was completed with seven participants, along with an additional four participants who acted as key informants. There is scope for expanding these numbers in future studies. The study only examined perceived usability, and any future testing should attempt to measure usability from amore objective standpoint.
Practical Implications
In the business context it is important for organisations to understand which mode of communication their employees prefer, and why, as employees may begin to interface with systems similar to the ones proposed more within their working environment and their business processes.
Originality / Value
Studies have been completed that analyse and understand single communicational methods of distinct systems, but very little research has been completed that compares two systems. There is business and research value in such a study, and this exploratory primary study should lead onto a deeper and more thought provoking studies in the future.
Keywords
Usability; Natural Language Processors; Question Answer Systems; Amazon Alexa; IBM Watson Assistant.
