SEEP Extranet: Usability Test Moderator's Guide - Final

SEEP Extranet
Usability Test Moderator’s Guide - Final
Prepared by:
Michael Brown
Mediabarn User Experience Lab
1760 Reston Parkway, Suite 508
Reston, VA 20190
March 18, 2010

Moderator’s Guide: SEEP Extranet
Introduction (5 minutes)
Hi, my name is Michael. Thank you very much for your time.
Today, as you know, we are using this software called UserVue/WebEx to conduct a remote
usability study to review the SEEP Members Extranet website. The purpose of this study is
to help us evaluate the website to observe how easy or difficult it is for users to navigate
and use, and if it is easy to understand. To do this I will be asking you some questions and to
complete some tasks using this website.
Let me assure you that everything you do and say during this time is completely
confidential. The information that we are collecting today is for internal purposes only.
Please know that I am not a part of the project team or design team, therefore feel
comfortable to share your thoughts freely. We want to learn how easy or difficult this
website is to use. There are no wrong answers. You are simply helping us understand what
works and what things need improvement.
As you work on these tasks, please think aloud, that is to share with me your thoughts as
you are completing these tasks so I’ll know what you’re thinking and what decisions you
may be making.
Do you have any questions?
Before we get started, I do have a couple questions.
Video Agreement:
To help us conduct this study, we are recording this remote usability session. We are
recording the audio of our conversation and the video of your desktop as you use this
website. Do we have your permission to record our phone conversation and the video of
your use of this website on your computer desktop?
Thank you very much.
Mediabarn User Experience Lab 2

Pre-Task Questions (5 minutes)
If you haven’t already done so, please login to the SEEP members’ website.
Before we begin, let me ask you a few questions:

1. When you go to the members’ website, where are you typically located (at home, at
work, on the road)?
2. Do you use a Windows PC or Apple computer?
3. Which web browser do you most commonly use?
4. With what kind of internet connection do you access the Web (dialup, dsl, cable, T1,
unknown)?
5. Do you know at what screen resolution you have your computer screen set?
a. (Optional) Does your computer monitor have a standard rectangular screen or
is it a wide screen?
6. How often do you use the SEEP Members website?
7. What do you most commonly do when using this website?
8. What is your biggest frustration when using this website?
9. What does this website do well?
Home Page Review (5 minutes)
Now I would just like to point out a few things on the home page…
1. When you come to the homepage what do you typically do here?

2. What information do you look for?
3. What information do you want to get from this page?
4. How would you describe this page?
Now, let’s go to your workgroup page.

Usability Tasks (40 minutes)

Now I am going to share several scenarios with you and I would like you to show
me how you would complete each task.
TASK 1: Contact a Working Group Member

Task 1: Let’s say you have a question about an upcoming online conference; send your
question to one of your working group members?
End Point: Select name from Working Group’s Group Members page.
TASK 2: Open a library document

You are looking for a “Promising Practices” case study about Children and Communities in
Africa published by the HAMED working group. Where would you find that report so you
could read it?
End Point: HAMED Working Group | Library | open PDF “Promising Practices – PC3” report
[IF HAMED member] You are looking for a document about Microfinance Consensus
Guidelines published by the Financial Services working group. Where would you find that
report so you could read it?
End Point: FSWG Working Group | Library | open PDF
TASK 3: Edit a document

Let’s say you were asked add a section to recent working group minutes. How would make a
change to a posted document?
<Note> Moderator will post a document for editing
End Point: Working Documents, Edit button
• What is the difference between “Pending” and “Approved” documents?

• When would you use the Library nav button and when would you use the Working
Documents nav button?
TASK 4: Reply to a discussion

Let’s say you were reading one of the discussions in your working group and you wanted to
post a response to what has been said, what would you do?
<Note> Moderator will set up a discussion area

End Point: Discussion, Reply button
• Let’s say you wanted to include a Power Point slide with your comment, what would
you do? Reply: Attach File
• What would your expectation be after you posted a comment?
• If you wanted to receive posted comments from a group discussion by email, what
would you do? Action Button: Alert Me
• Let’s say, your Working Group is hosting a 3-day online conference; show me how
you would start a discussion about the event. “New” button
• How often have you used the working group discussion area before?


TASK 5: Post a document

Now let’s post a document about the working group conference. How would you do that?
<Note> Moderator will send a .pdf document for posting to work group
End Point: Working Documents, Upload button
• How would you move a document to a different category? Edit: Category menu
TASK 6: Post an event on the Calendar

How would you add the 3-day conference on your Working Group’s calendar?
End Point: Calendar, “New” button
TASK 7: Email Working Group members

Show me how you would email all the members of your working group about this 3-day
conference?
End Point: Site Actions | Site Settings | People & Groups |Select All | Email Users button
TASK 8: Edit the Wiki

Are you familiar with the Wiki page? Show me how you would update the Working Group
Wiki with information about the 3-day conference.
End Point: Wiki | Edit button
• How would you create a new page in your WG’s Wiki?

• ASK: is it clear to you how you can link Wiki pages to one another?
• ASK: What do these buttons mean to you [History, Incoming Links]?
• (Optional) How would you add a table to your page that had 4 columns and 3 rows?
• (Optional) How would you add content to each cell in the table?
TASK 9: Edit the User Profile

You want to edit your personal information on the members’ website so other members
knew where you work and your expertise. Where would you go to make those changes?
End Point: Observe where members go.
TASK 10: Edit the Working Group main page (Facilitator’s Only)
Show me how you would add a link about the conference on the New Resources module on
your Working Group’s main page.
End Point: Site Action, Edit Page

SURVEY
Strongly Strongly
disagree agree
1. I think that I would like to

use this system frequently
1 2 3 4 5
2. I found the system unnecessarily

complex
1 2 3 4 5
3. I thought the system was easy

to use
1 2 3 4 5
4. I think that I would need the

support of a technical person to
be able to use this system 1 2 3 4 5
5. I found the various functions in

this system were well integrated
1 2 3 4 5
6. I thought there was too much

inconsistency in this system
1 2 3 4 5
7. I would imagine that most people

would learn to use this system
very quickly
1 2 3 4 5
8. I found the system very

cumbersome to use
1 2 3 4 5
9. I felt very confident using the

system
1 2 3 4 5
10. I needed to learn a lot of

things before I could get going
with this system 1 2 3 4 5

Closing Questions (3 minutes)
1. Was there anything you liked about the site?

2. Was there anything you didn’t like about the site?
3. Do you have any thoughts on how the website could be improved?
4. Do you have any additional feedback on the site or the test session itself?
That’s all we have for today. Thank you very much for your time. Your feedback has been
very helpful.

SEEP Extranet: Usability Test Moderator's Guide - Final

Uploaded by

Copyright:

Available Formats

SEEP Extranet: Usability Test Moderator's Guide - Final

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

SEEP Extranet: Usability Test Moderator's Guide - Final

Uploaded by

Copyright:

Available Formats

SEEP Extranet

Usability Test Moderator’s Guide - Final

March 18, 2010

Do you have any questions?

Before we get started, I do have a couple questions.

Thank you very much.

Mediabarn User Experience Lab 2

Pre-Task Questions (5 minutes)

Before we begin, let me ask you a few questions:

Home Page Review (5 minutes)

1. When you come to the homepage what do you typically do here?

Now, let’s go to your workgroup page.

Mediabarn User Experience Lab 3

Usability Tasks (40 minutes)

TASK 1: Contact a Working Group Member

TASK 2: Open a library document

End Point: FSWG Working Group | Library | open PDF

TASK 3: Edit a document

• What is the difference between “Pending” and “Approved” documents?

TASK 4: Reply to a discussion

<Note> Moderator will set up a discussion area

Mediabarn User Experience Lab 4

Mediabarn User Experience Lab 5

TASK 5: Post a document

TASK 6: Post an event on the Calendar

End Point: Calendar, “New” button

TASK 7: Email Working Group members

TASK 8: Edit the Wiki

End Point: Wiki | Edit button

• How would you create a new page in your WG’s Wiki?

TASK 9: Edit the User Profile

End Point: Observe where members go.

End Point: Site Action, Edit Page

Mediabarn User Experience Lab 6

1. I think that I would like to

2. I found the system unnecessarily

3. I thought the system was easy

4. I think that I would need the

5. I found the various functions in

6. I thought there was too much

7. I would imagine that most people

8. I found the system very

9. I felt very confident using the

10. I needed to learn a lot of

Mediabarn User Experience Lab 7

Closing Questions (3 minutes)

1. Was there anything you liked about the site?

Mediabarn User Experience Lab 8

You might also like