Chapter 8 Barker Conducting Usability Tests
This chapter covers three basic types of tests
1. Tests for task performances (procedures)
2. Tests for skills and understanding (tutorials)
3. Tests for access to information (references)
A ten-step test plan is recommended to cover main tasks needed to perform when conducting usability test for large and small projects.
Test Forms
Figure 8.1 – a procedure test form
Ø You can adapt this form to fit your needs
Figure 8.2 – Guidelines for testing documentation
Ø This form can be used when testing procedures
1. Decide when to test
There are nine stages of documentation development
The best time to test is during three stages:
Ø Phase 3 design
Ø Phase 5 writing/drafting
Ø Phase 9 Field evaluation
2. Select the test points
Test points fall into two areas
Ø Problems with content
Ø Problems with document design
Select procedures for testing
Determine which tasks you will need to test based on where you think a mistake or error may occur. Keep in mind that you are not testing the user – you are testing the documentation.
Look at testing procedures that are complex, meaning they have ten or more steps, or a one-time installation, users can not learn from experience when they are doing these tasks only once.
Certain tasks create confusion and high failure errors; look for specific places that mistake will occur in your documentation – this can reduce efficiency and cost money. There are certain information related tasks that should be error free, they are
Ø Importing information from other programs
Ø Creating, naming, and formatting files
Ø Exporting information for other programs or other program formats
Ø Creating printouts and reports
Select Design Strategies for Testing
You should look at the following design elements:
ü Terminology
ü An index
ü Cuing patterns
ü Headings/layout
ü Navigation
ü Extraordinary document formats
Choose the Type of Test
Ø Table 8.2 indicates three types of tests in relation to the test points
Set Performance and Learning Objectives
Ø You want your test to measure the actual behavior.
Ø Performance does not always mean, getting the test done in the shortest amount of time
Ø Table 8.3 demonstrates types of performance objectives
Objectivity and Testing
You need to set up tests in a way that you are not prejudicing the outcome. No test can be 100% objective, but you cannot have your document passing with no problems what so ever, that is not reality. Bias can creep into your test without your intending it to do so.
Select Testers and Evaluators
§ The tester is the person administering the test
§ The evaluator is the person taking the usability test
Prepare the Test Materials
For testing in a Test Lab, more materials are needed and can be complex. But for purposes, we will be performing informal field evaluations. Check Tables 8.4, 8.5 and 8.6 for more information.
Pilot Test
It is probably a good idea to run a test pilot on your written material, you will check for the following:
Ø Are your instructions written so that your evaluators understand and will they be able to perform the instructions correctly?
Ø Check to be sure you are not using terms that the users can understand.
Ø Will the user be able to perform the test in the time you have allowed?
Set Up the Test Environment
You can set up the test at the user’s work environment or a controlled lab. To learn in the context of the user’s work it is best to perform the test in their work environment. A lab offers control, but is not always a choice that you have. Researchers recommend using a combination of test environments.
Record Information Accurately
Ø You need to use accurate methods to record what you see and hear during usability testing. Recorders can be used to record voice, also can used video recorders to record every movement of the evaluator.
Interpret the Data
Ø The data should tell you something about your users.
Incorporate the feedback
Ø Testing should be incorporated into the document design. You should re-test a few times and analyze the data.
Subscribe to:
Post Comments (Atom)
7 comments:
Although I've read this chapter before, I have picked up on some additional information. As a result, I am contemplating some things on our usability project.
1) Is it wise to have a separate documentation tool and activity, or would it better to incorporate them into one so that the user isn't looking to one and then the other. The argument can be made that that is real world, and that way you use the documentation tool for whatever activity you choose rather than for each specific activity.
2) How will we judge our success or failure? Perhaps that's not the scope of this exercise, but I can't help but wonder how you go about that in the real world. The text does discuss this; however, again, real world, so many variables and subjective info, this won't be an easy task.
This chapter has me re-thinking my documentation project and the usability project. I thought the terms 'user' and 'tester' were the same but now realize the differences. Mary's thoughts on judging success or failure tie in with what I was wondering as I read. I think all documents, not just when new software programs are introduced, should be revised periodically or sooner if the documentation isn't effective or serving its purpose such as users never referring to it when performing the task. I know this check system doesn't happen in my department simply because there just isn't enough time or personnel to do it on a regular basis. This is an interesting point.
The chapter's quote that "the earlier you test the weaker the results but the easier it is to make changes; the later you test the better the results but the harder it is to make changes." Will software such as AuthorIT make it easier to make sweeping changes later in the process? It seems like it could address that issue.
The testing paradox: "The earlier you test the weaker the results but the easier it is to make changes; the later you test the better the results but the harder it is to make changes." This statement really has me thinking about how important testing a document is in any stage of the writing process. It seems to me that if there is time in the schedule it is important to test the document in some way at every stage of the writing. I have incorporated this into my documentation project, because I have two coworkers looking over the information at the beginning of the writing process and one coworker looking over it at the end (before the final revisions will take place). I think this will give me a wide range of options and ways to make the document better. The hardest part of this for me is getting good results and interpreting them into my document. I have not had much experience in this kind of testing and am hoping these projects will help me understand the process better. I will be keeping this in mind as we do usability tests for GoogleEarth as well.
I was intrigued by the section on field testing methods that emphasize task issues, more particularly, the modified Q-Sort tests. By grouping action tasks into categories and asking the user to rank them in order of preference, developers can help determine what documentation tasks require special attention, and which require little or no attention. It seems like such a simple process using the index cards and having the user sort them by preference, but gives a good insight into which tasks a user determines to be important, and will help developers logically organize their documentation – using the order obtained from the sort cards. I was thinking about the action steps that always end up at the end of the user preference. I would take that to mean those steps are used rarely, if at all. Maybe a software documenter would consider those steps important, but after the q-tests determine they are not; that would be a huge time and effort savings by not putting so much emphasis on the lesser-used or preferred steps.
Lastly, scenario-based testing appears to be a good all-around method of testing since it provides so much information about all aspects of a document. Barker said it provides info from what tasks users chose to perform the task to what resources the user referred to in completing the task. Wouldn’t scenario-based testing be a good option for a smaller operation to get a better overall picture of the documentation usability?
I think this chapter points out some useful information about testing a project. I never understood that there were so many tests that took place before something actually got finished. I think this chapter also references an important issue as to when to conduct the testing. It seems the consensus seems to be to test often. I think in this day and age there is a lot of software help that can limit the need for some testing. This is important because there is not always enough time and personnel to conduct the recommended testing.
This chapter will be very useful when conducting our own usability test for Google Docs. Research takes a lot of time and effort, so it is best to be as organized as possible. Having these spelled out in 10 steps is an easy way to stay thorough and not miss anything important. Accurate methods of recording information is the most crucial part I think, because if the information isn't given in a correct manner, the result will need to be examined again, and perhaps even redone. I might need to keep this chapter handy when putting the final touches on our usability project as well.
While repeated testing is certainly ideal, the best results would come from having a new person following the directions each time, and it may be difficult to continuously get a fresh pair of eyes, so to speak.
Post a Comment