Development of an Irish tool to assist in self and peer review of online programme and module design: Stage 2 Delphi study results

Key words: Review; Programme and Module Design; Delphi Study; Enhancement; Quality. Context Blended and online activities are becoming more mainstreamed in the changing higher education environment, however many staff are new... [ view full abstract ]

Key words: Review; Programme and Module Design; Delphi Study; Enhancement; Quality.

Context
Blended and online activities are becoming more mainstreamed in the changing higher education environment, however many staff are new to the process of their design at module and programme level. There have been some international tools available that have been used to assist academic staff and educational technologists/developers to learn from the self and peer review of their designs. However, some of the language in these tools, and the processes of how they are used, may not suit the Irish context. Therefore, we carried out a two-stage study to design both a tool and a process that would be more suited to the Irish context and informed by those working in it.

Methodology and Findings
The first stage of the study (based on Whiting et al, 2003), presented in EdTech 2015, engaged 18 experts across Irish higher education institutions to develop the conceptual aspect and the initial item generation. As a result, 100 programme and 80 module design statements were created. These experts highlighted the importance of using the tool in an enhancement process incorporating 'collegial' self and peer dialogue. They also suggested that it should be flexible and adaptable for local contexts, drawing on a core set of statements and an optional bank (O’Neill & Cashman, 2015).
Stage 2 of the study used a two-round Delphi methodology in order to gain further consensus on the extent to which these statements should be included in the tool, drawing from a wider expert group in Ireland. There was a separate sample used for the programme design (n= 53; n=30) and the module design (n= 17; n= 24) statements. Based on a series of data analysis steps, the statements numbers were reduced. For example, as a starting point statements below 80% agreement were discarded (Bok et al, 2011; Holey et al, 2007).
The programme design statements were reduced to 48 and the module design to 31 statements. Although many of the high scoring statements related specifically to the digital aspect (‘The online learning environment is well organised, consistent and easy to navigate’), many of the statements presented in stage 1 and scoring highly in stage 2 were more general teaching and learning statements, for example ‘The programme has a coherent structure’. This implies that the tool could also be valuable in programmes with no online aspect.

Benefits to Teaching and Learning
This study has produced a valid tool and process for blended, online and face-to-face programme and module design. The tool should assist staff working in this area to self and peer review their designs at the early or post implementation stages. Their designs can be reflected on and where appropriated changes made. ‘The Programme and Module Review Tool for Online, Blended and Face-to-Face Contexts’ (http://www.ucd.ie/t4cms/UCDTLP0069.pdf) is now available for all to use, aligning with Whiting et al’s (2003) final pilot phase for design of a quality tool.