Shanmugam Rajasekaran



Shazeeye Kirmani

 

Measuring Heuristic Evaluation Skills

   
 

Heuristic Evaluation is a discount usability engineering method for quick evaluation of a user interface. It involves 3 or more evaluators examining the interface and judging its compliance with a set of recognized usability principles (the "heuristics").

This study is a first attempt to quantify heuristic evaluation skills. Quantification is based on the number of unique issues identified by the evaluators as well as the severity of each issue. Unique issues are categorized into 8 user interface parameters (Interaction Design, Navigation, Visual Design, Functionality, Information Architecture, Labeling, Content, Other) and severity is categorized into three: showstopper, major issue and irritant.

An overall benchmark computed from the collated evaluations is used to compare skills across applications as well as within applications. The result of this skill measurement divides the evaluators into bands of expertise. The Indian Railways website was used to measure the heuristic evaluation skills of 30 evaluators.

Measurement of heuristic evaluation skills has varied applications. It can identify the strengths and weaknesses of evaluators in specific areas of the User Interface. Skills of different evaluators can be compared as well performance improvement of an evaluator can be measured. Further studies will help identify experts from novices based on a determined standard skill level.

    

Speaker Background
Shanmugam Rajasekaran is Head, Experience Design at Infosys Technologies Limited. In the last financial year, the User Experience team delivered more than 250 projects for 90 Infosys customers across several domains. He is also actively involved in establishing quantitative processes in usability. He graduated in Product Design from National Institute of Design, Ahmedabad, in 1992.

Shazeeye Kirmani is a Senior Usability Engineer at Infosys Technologies Limited. She received an M.S. in Human Factors from the State University of New York at Buffalo in 2004. She has worked extensively in the area of user testing and heuristic evaluations. She has anchored more than 100 heuristic evaluations from July 2005 to March 2006.