Honour thesis assessment: the role of guidelines in achieving inter-rater agreement
conference contributionposted on 2023-05-23, 04:35 authored by Alan HendersonAlan Henderson, Guijt, M, Michael BreadmoreMichael Breadmore, Anna CarewAnna Carew, Rosanne Guijt
Engineering honours projects are often self selected or negotiated by students to be in areas of their own interest. While this encourages motivation and engagement in a self directed research project, it also considerably increases the diversity of honours project types. Such project diversity raises questions about the most suitable form of guidelines to provide good inter-rater agreement. Project diversity can also result in academics assessing projects outside their primary area of research specialisation. This is particularly true of transdisciplinary projects that cross over conventional discipline boundaries. To investigate these issues, a team of academics from two Schools at the University of Tasmania assessed a collection of engineering theses using a variety of different guidelines. All guidelines were found to produce poor inter-rater agreement, however inter-rater agreement was improved when both assessors were of the same discipline. An account of the academics’ comments on use of the guidelines reveals conflicting opinions of good and bad features. Guidelines that were viewed as easy to use and less subjective were found not to substantially improve inter-rater agreement. The implications of these findings are discussed and suggestions made in relation to improving assessment guidelines for honours theses.
Publication titleProceedings of 20th Annual Conference for the Australasian Association for Engineering Education
EditorsKestell, Grainger, Cheung
Department/SchoolSchool of Engineering
PublisherThe School of Mechanical Engineering, The University of Adelaide, Adelaide, South Australia
Place of publicationAdelaide
Event VenueThe University of Adelaide
Date of Event (Start Date)2009-12-06
Date of Event (End Date)2009-12-09
Rights statementCopyright 2009 The authors.