There has been a continuing trend for educational institutions to focus on e-learning, this has been accompanied by a shift towards online assessment, either of a formative or summative nature. This paper is a critical analysis of online assessment methods introduced into a first year unit in the Faculty of Commerce at the University of Tasmania. It reflects on the effectiveness on the change from a paper-based assessment method to an online system over a period of two years as well as on the advantages and disadvantages of the change for students, lecturers and tutors. The performance of students in undertaking the unit in Semester 2, 2002 and Semester 2, 2003 was measured using a common assessment test; in 2002 using a paper-based system and then in 2003 for an online system. Analysis was then performed on a cohort within a semester. The results show there is no significant difference in performance between the groups, however there existed a number of administrative benefits associated with the online assessment method.