Chapter 8 Barker Conducting Usability Tests
This chapter covers three basic types of tests
1. Tests for task performances (procedures)
2. Tests for skills and understanding (tutorials)
3. Tests for access to information (references)
A ten-step test plan is recommended to cover main tasks needed to perform when conducting usability test for large and small projects.
Test Forms
Figure 8.1 – a procedure test form
Ø You can adapt this form to fit your needs
Figure 8.2 – Guidelines for testing documentation
Ø This form can be used when testing procedures
1. Decide when to test
There are nine stages of documentation development
The best time to test is during three stages:
Ø Phase 3 design
Ø Phase 5 writing/drafting
Ø Phase 9 Field evaluation
2. Select the test points
Test points fall into two areas
Ø Problems with content
Ø Problems with document design
Select procedures for testing
Determine which tasks you will need to test based on where you think a mistake or error may occur. Keep in mind that you are not testing the user – you are testing the documentation.
Look at testing procedures that are complex, meaning they have ten or more steps, or a one-time installation, users can not learn from experience when they are doing these tasks only once.
Certain tasks create confusion and high failure errors; look for specific places that mistake will occur in your documentation – this can reduce efficiency and cost money. There are certain information related tasks that should be error free, they are
Ø Importing information from other programs
Ø Creating, naming, and formatting files
Ø Exporting information for other programs or other program formats
Ø Creating printouts and reports
Select Design Strategies for Testing
You should look at the following design elements:
ü Terminology
ü An index
ü Cuing patterns
ü Headings/layout
ü Navigation
ü Extraordinary document formats
Choose the Type of Test
Ø Table 8.2 indicates three types of tests in relation to the test points
Set Performance and Learning Objectives
Ø You want your test to measure the actual behavior.
Ø Performance does not always mean, getting the test done in the shortest amount of time
Ø Table 8.3 demonstrates types of performance objectives
Objectivity and Testing
You need to set up tests in a way that you are not prejudicing the outcome. No test can be 100% objective, but you cannot have your document passing with no problems what so ever, that is not reality. Bias can creep into your test without your intending it to do so.
Select Testers and Evaluators
§ The tester is the person administering the test
§ The evaluator is the person taking the usability test
Prepare the Test Materials
For testing in a Test Lab, more materials are needed and can be complex. But for purposes, we will be performing informal field evaluations. Check Tables 8.4, 8.5 and 8.6 for more information.
Pilot Test
It is probably a good idea to run a test pilot on your written material, you will check for the following:
Ø Are your instructions written so that your evaluators understand and will they be able to perform the instructions correctly?
Ø Check to be sure you are not using terms that the users can understand.
Ø Will the user be able to perform the test in the time you have allowed?
Set Up the Test Environment
You can set up the test at the user’s work environment or a controlled lab. To learn in the context of the user’s work it is best to perform the test in their work environment. A lab offers control, but is not always a choice that you have. Researchers recommend using a combination of test environments.
Record Information Accurately
Ø You need to use accurate methods to record what you see and hear during usability testing. Recorders can be used to record voice, also can used video recorders to record every movement of the evaluator.
Interpret the Data
Ø The data should tell you something about your users.
Incorporate the feedback
Ø Testing should be incorporated into the document design. You should re-test a few times and analyze the data.
Friday, March 7, 2008
Chapter 8 Campbell: Did I Forget Anything?
by Mary Dobbins
Creating a Perfect Document
There are 5 different types of review which should be done sequentially so most critical issues are addressed first:
Verification – Reviewing in the proper order saves time. Combining reviews for format, grammar, punctuation, accuracy, etc., is much more demanding and leaves room for error unless multiple reviewers are used.
Validation – Checking for accuracy involves making sure information is up to date, dimensions given are accurate, requirements are correct, etc. This can sometimes be combined with editing but usually works best as a separate task. User errors require altering wording to make document clearer; policy/procedure errors require change in content or sequence. Check for usability using ideally one or two other reviewers by: trying to gauge by their reactions if something is unclear, soliciting comments, or observing.
Editing – Be careful not to alter the meaning, edit for format, wording, consistency, flow, cohesion, layout, and visual appeal. Narrow the scope using a format edit, language edit, style edit, etc. Use your knowledge about the audience to guide your editing. Use consistent marking and a style sheet. Read aloud and use different colored pen marks. Review page layout, design elements, consistency and logic.
Proofreading – Credibility is the most important reason for proofreading. Decontextualizing, meaning taking letters, numbers and words out of context instead of grouping them together, is an effective proofreading tool. You can also read backwards, real aloud into a tape recorder or to a partner, or read diagonally. Look for typos, punctuation, spacing, spelling, agreement, page breaks, titles, misplaced words and phrases, alignment, names, numbers, typestyle, typesize, and margin errors. Don’t skip proofreading!
Approval – Communicating with approvers throughout the document process is important to avoid a possible rewrite. The approval process should be a formal procedure which includes time frames and encourages approvers and managers to solicit comments from their own areas. To facilitate quick approval, use a form that’s easy to complete. For lengthy documents, include a summary memo. Coordinate conflicting responses among approvers.
If possible, have others review your document rather than trying to do it yourself. Review the checklists at the end of the chapter for step-by-step guidance in completing the review types above.
by Mary Dobbins
Creating a Perfect Document
There are 5 different types of review which should be done sequentially so most critical issues are addressed first:
Verification – Reviewing in the proper order saves time. Combining reviews for format, grammar, punctuation, accuracy, etc., is much more demanding and leaves room for error unless multiple reviewers are used.
Validation – Checking for accuracy involves making sure information is up to date, dimensions given are accurate, requirements are correct, etc. This can sometimes be combined with editing but usually works best as a separate task. User errors require altering wording to make document clearer; policy/procedure errors require change in content or sequence. Check for usability using ideally one or two other reviewers by: trying to gauge by their reactions if something is unclear, soliciting comments, or observing.
Editing – Be careful not to alter the meaning, edit for format, wording, consistency, flow, cohesion, layout, and visual appeal. Narrow the scope using a format edit, language edit, style edit, etc. Use your knowledge about the audience to guide your editing. Use consistent marking and a style sheet. Read aloud and use different colored pen marks. Review page layout, design elements, consistency and logic.
Proofreading – Credibility is the most important reason for proofreading. Decontextualizing, meaning taking letters, numbers and words out of context instead of grouping them together, is an effective proofreading tool. You can also read backwards, real aloud into a tape recorder or to a partner, or read diagonally. Look for typos, punctuation, spacing, spelling, agreement, page breaks, titles, misplaced words and phrases, alignment, names, numbers, typestyle, typesize, and margin errors. Don’t skip proofreading!
Approval – Communicating with approvers throughout the document process is important to avoid a possible rewrite. The approval process should be a formal procedure which includes time frames and encourages approvers and managers to solicit comments from their own areas. To facilitate quick approval, use a form that’s easy to complete. For lengthy documents, include a summary memo. Coordinate conflicting responses among approvers.
If possible, have others review your document rather than trying to do it yourself. Review the checklists at the end of the chapter for step-by-step guidance in completing the review types above.
Subscribe to:
Posts (Atom)