Structure Evaluation

Download 'printer-friendly' PDF version (File size: 14 KB)

- Gerry Gaffney

Observing users attempt to find items in a program, web site, supermarket or other structured environment reveals considerable variation between individual search patterns. While you can use various techniques (such as card sorting) to develop such structures, the process is very dependent on opinion and educated guesswork.

However, you can evaluate and refine structures using simple paper-based techniques.

Structure evaluation:

• Is easy and cheap to conduct
• Enables you to evaluate whether users can find items
• Is flexible enough to accommodate 'on-the-fly' changes
• Does not require a working system, as it can be conducted entirely on paper
• Provides solid data you can use to refine the structure.

When is Structure Evaluation appropriate?

Structure evaluation is appropriate whenever you need to test whether people can locate categorized items - for example, within a web site.

Structure evaluation is best conducted at the early stages of design, when changes are relatively easy and inexpensive.

Conducting Structure Evaluation

Assume we want to evaluate the following simple web site structure. (Note that each item is numbered.)

We can represent the high-level structure of this site by laying four sheets of paper on a table:

On the reverse of each sheet is the list of items it contains. For example, the reverse of the 'Support' sheet contains the numbers '12', '3', and '23' (since these are the items that belong in that category).

A participant is presented with an index card representing an item:

The participant attempts to locate the item in the structure. The reason for using a number is that this reduces the ability of participants to remember items already seen. Each item is presented in turn (in a random sequence) until the participant has attempted to locate each one.

All attempts to locate items are logged. Allow participants a pre-defined number of attempts to locate each item. Remember that like usability testing, this sort of evaluation can be stressful for participants.

Once all participants have completed the exercise, enter the data in a spreadsheet, and examine the groupings.

You can use cluster analysis to get a pictorial representation of the resultant groupings. An easy way to do this is using IBM's EZSort program (free from www.ibm.com/easy).

Pay attention to items that are particularly difficult to locate, and consider whether they should be renamed or relocated.

Who should participate?

Involve as broad a range of participants as possible from within the intended user group. Avoid using participants who within the development team or client organization, as they will have an unrealistic familiarity with terminology.

If possible, use at least 8 participants to get a spread of results. Be aware that the more participants you have, the more data you will need to analyze.

Variations

Since evaluations are quick, consider combining them with other forms of usability activities to maximize the benefit of having users present.

The free software program 'Classified' provides the ability to test simple structures without the need for extensive preparation, and also automates the logging process (available from http://www.infodesign.com.au/usabilityresources/classified/).

Download a 'printer-friendly' PDF version of this sheet (File size: 14 KB)

Download Adobe's free Acrobat Reader (TM) for PDF files