Rachel Bury
Edge Hill University, Learning Services
I am working in a role within Learning Services which manages Quality, Communication and Marketing. The quality aspect of the role involves use of KPIs and liaison with the University Academic Quality Unit regarding the validation of new programmes, re-validations, and collaborative provision.
Purpose: The purpose of the paper is to outline and share Edge Hill University (Learning Services) journey in relation to collecting, using and interpreting a wide range of information and data to inform service developments in terms of the delivery of TEL (Technology Enhanced Learning). A survey can be a blunt instrument and can actually produce more questions than answers which is where other methods have been used to triangulate data and comments and feedback.
The Student eLearning Survey was introduced in December 2008 as a means of exploring the learner perspective of TEL at Edge Hill University. The survey is an important tool used to identify and examine trends and system performance where there are direct comparisons to past surveys. The intention is to improve the student experience through a better understanding of what makes a good experience of TEL, and more importantly, what does not. It also provides an indicator of the extent to which the institutional VLE, Learning Edge and embedded library and information resources are integral to the learning experience of Edge Hill students.
Approach: In the paper we will outline our approach which includes the very rich, longitudinal data and comments we receive via the survey but also how we triangulate using user experience tools and also system management data.
The survey is a rich source of intelligence and insight into students’ use of technology in support of their learning; their views, likes, dislikes and preferences on tutor use of the virtual learning environment and associated technologies (including library) are captured, analysed and considered. Feedback on institutional systems and personal ownership and use of devices is also collected to form part of picture, and as time passes the survey’s value has grown incrementally.
The survey is one part of the portfolio of evidence gathered by Learning Services, and the along with the user experience it absorbs a considerable amount of staff time and effort to administer, run and analyse the responses and UX recordings, but the effort is more than rewarded by the intelligence gained.
Findings: Now in its 7th iteration, it is possible to draw a correlation between action taken in response to issues raised in earlier surveys, and the usability testing (2 periods of testing have been undertaken) and noticeable improvements in satisfaction today. Strategically, the survey responses have formed a valuable evidence base, which have prompted a number of follow-on actions. These have ranged from business cases for additional investment, conversations with individual departments on their use of technology to support student engagement, and changes to practice, for example, how the Library supports students who bring their own devices.
Research: We will outline how we have attempted to triangulate the survey data against other sources e.g. system data, reported issues and usability data. We will describe how we have used this information to inform our decision making in terms of improvements and change to the VLE and library support for mobile apps for learning. In July 2016 as a result of the 6th eLearning survey, a number of high level changes were made to the VLE interface and in particular the library resources and support embedded therein and these were further tested with questions in the 7th survey. User experience testing of the new interface for the library ‘tab’ informed by the survey’s findings will be undertaken in March 2017 and will be presented at the July conference.
Conclusions: We will share our experiences of using a rich set of data and usability observations with the audience and how this approach to performance measurement and impact has allowed us to make a number of high level improvements to the universities VLE including access to library resources via the library tab. The now bi-annual survey, clearly provides signposts to both where we may want to investigate further or to highlight experiences which are having impact.
Value of the proposal: We feel that this paper will be useful for attendees who are thinking about using usability tools to triangulate other qualitative and quantitative data they may have. Words 675
