The operational, pedagogical and technical challenges of introducing an automated assessment system into computer science laboratory sessions

This paper describes the establishment of an automated assessment and feedback system for programming assignments within the Maynooth University Moodle VLE. The system developed has provided an enhanced learning experience by... [ view full abstract ]

This paper describes the establishment of an automated assessment and feedback system for programming assignments within the Maynooth University Moodle VLE. The system developed has provided an enhanced learning experience by using an interactive assessment tool while subsequently reducing the demonstrating and grading overhead.

Recent increases in class sizes in Computer Science programmes have introduced significant challenges including timetabling, consistency in delivery across sessions, recruitment of demonstrators and provision of prompt feedback. A trial using automated assessment within the Computer Architecture module has shown that effective delivery can be achieved by replacing multiple smaller laboratory sessions with a single large class by the inclusion of an automated assessment mechanism.

Virtual Programming Laboratory (VPL) is a Moodle activity plug-in that helps to manage programming assignments and delivers immediate feedback to the students. The main operational advantage with VPL is that it separates the running of the source code submitted by students from the main Moodle server.

The system was first tested using MASM (Microsoft Assembly Language) which ran on a DOS-Emulator built into a Linux based server. Students could develop the software within the Moodle webpage as if it was an interactive integrated development environment (IDE) and then have it graded automatically.

The Centre for Teaching and Learning has supported this project by bringing together IT Services, Computer Science Technical Support and the lecturers required to implement the system.

A student survey of the use of VPL showed very positive feedback with 78% of respondents saying they would like to see it used in other modules and rating it as 3.75 on a scale of 1=Don't Like to 5=Really like.

A blended approach to assessment using a combination of VPL and quizzes was used in each laboratory session. The approach of using automated interactive feedback has improved engagement in the module and allowed both demonstrators and lecturers to have more time to interact with the students during the laboratory session.