A toolkit for appraising the long term usability of a text editor

Ronny Cook, Judy Kay, Greg Ryan, Richard C Thomas?

Basser Department of Computer Science,
University of Sydney, 2006 Australia

? Department of Computer Science,
The University of Western Australia, Nedlands, WA 6009, Australia

Abstract

We describe a large scale, low cost, project that has examined the way people develop their skill in using fundamental software tools. The study involved over two thousand users during a three-year period of use of the sam text editor. The work took place while the editor was being employed in normal day to day work - it was not a laboratory experiment.

Our main contributions are first to demonstrate very long-term, low-cost monitoring with collections of simple analysis tools. Second, we have started to develop an understanding of how usability changes in the long term. Third, studies of usability often concentrate on assessment before a system is released for widespread use, whereas ours can help inform the long term design of new tools - a different dimension of usability. In addition we have mixed snap-shot studies with descriptions of long-term, gradual change. We can track the full development of the user, even though the quality of the data is lower than that normally associated with usability studies.

1. INTRODUCTION

The most highly skilled individuals have usually acquired their distinctive level of expertise over years or even decades, be they cathedral stone-masons or medics. As the computer becomes all-pervasive, it will be essential not to inhibit this long term self-development of skill. It is important, therefore, to understand the usability of fundamental software tools. One of the most basic of such tools supports users in creating and modifying texts: a text editor. Accordingly, we believe that learnability and usability testing of text editors, over the long term, is important.

There have been many studies of the ways that users learn about and actually employ text editors. This is natural given their importance. For example, see Roberts and Moran [25] , Allwood and Eliasson [1] , Kay and Black [15] , Mack, Lewis and Carroll [19] , and Sebrechts, Marsh and Furstenburg [27]. There have also been many studies of the usability of various text editing tools, for example: Rosson [26] , Poller and Garter [23] and surveys such as Bosser [3].

Usability has several components. Classically, it is viewed in terms such as those suggested by Shackel [28] and Chapanis [5] with factors like: effectiveness; flexibility; ease of use and learnability; and the user's attitude to it. They assess these aspects of usability for a given user population, fixed tasks, fixed support and a fixed environment. This definition, and much of the work in the literature cited above, leans towards a view of usability that can be assessed in laboratory settings.