Understanding HCI

Performance Measurement

What is Performance Measurement?

This technique is to used to obtain quantitative data about test participants' performance when they perform the tasks during usability test. This will generally prohibit any interaction between the participant and the tester during the test that will affect the quantitative performance data. The technique can be used in combination with retrospective testing, post-test interview or questionnaires so that both quantitative and qualitative data are obtained.

Why Use it?

Participants Needed

Experts

1 expert needed

Users

To obtain dependable results, at least 5 user participants are needed, while 8 or more participants would be more desirable

Task List

  1. Define the goals for the usability testing in terms of usability attribute
    1. easy to learn
    2. efficient to use
    3. easy to remember
    4. few errors
    5. subjectively pleasing
  2. Balance the various components of the goals and decide on their relative importance.
  3. Quantify these usability issues by measurements such as those given below:
    1. The time users take to complete a specific task.
    2. The number of task of various kinds that can be completed within a give time limit.
    3. The Ratio between successful interactions and errors.
    4. The time spent recovering from errors.
    5. The number of user errors.
    6. The number of commands or other features that were never used by the user.
    7. The number of system features the user can remember during a debriefing after the test.
    8. The frequency of use of the manuals and/or the help system, and the time spent using them.
    9. The proportion of users who say that they would prefer using the system over some specified competitor.
    10. The proportion of users using efficient efficient working strategies in case there are multiple ways of performing the tasks.
  4. Analyze the data to draw conclusions
    1. To compare with a benchmark value calculate the mean, median or mode and the standard deviation http://www.physics.csbsju.edu/stats/cstats_NROW_form.html

Conditions required

  1. There can't be any unexpected interruption during the test.
  2. Conduct pilot test (dummy run) to make sure that the tools and the techniques for data collection work well.
  3. When possible, the test should be video-recorded to support data collection, so that some data can be collected or verified after the test by reviewing the video recording.
  4. Even though this technique is aimed to collect quantitative data, it should be noticed that it's very important to collect qualitative data to uncover the user's mental process and other information behind the quantitative data and take them into account while drawing the conclusions.

Examples

  1. example of mean, median and mode calculation

Limitations Of methodc

Exercise

Reading