DATA SET FOR STUDENT PROJECT POWERPOINT
The following are the data that you will need to prepare your individual project PowerPoints. The data are representative of what you would have collected.
You are doing a limited evaluation of the Windows 10 operating system (running on a Dell Latitude 3450 and Apple iOS X operating system running on MacBook Pro (Model MacBookPro12,1). Assume that the laptops are sufficiently similar in hardware and processor speed to make a valid comparison.
You have collected data from four participants, all of whom are GMU undergraduates. (Four participants are sufficient for the project.) You conducted your experiment in the Fenwick Li
ary Reading Area, also known as the MIX. Five tasks were evaluated in this limited study. As discussed in class, you used a “heads down” protocol because you wanted to measure time on task.
The five tasks are:
· Uninstall an application
· Change the wallpape
· Add a Bluetooth device (a headphone)
· Change the time & date
· Zip a file
Note: You do not need to provide any more detail than just the names of the tasks in your PowerPoint. Thus, no details, specifically, the task analysis steps are provided here.
The sequence of events followed for each of the participants you dealt with follow.
1. Participant asked to participate using the introduction I provided. “Hi! My name is Jack Jones and this is Hank Smith. We’re doing a study to determine how easy it is to use two university li
ary web sites for our class, SYST 469. We’d like you to help us out by doing a few tasks that you would do with operating systems. We only need about 5-10 minutes of your time, and all responses will be anonymous. Also, you can’t do anything wrong as we’re assessing the products and not you. Please have a seat and we’ll start.”
2. Participant given the informed consent form. All participants looked at the form and agreed by nodding their heads. As required by anonymity, you did not ask for any participant names. Also, all participants left the form on the table.
SYST469 – Human-Computer Interaction Class Project Informed Consent Form
RESEARCH PROCEDURES
This research is being conducted to support a student class project relating to the ease of use (usability) of different products. If you agree to participate, you will be asked to perform several tasks using a commercial product or products, and give your opinion about product or products usability while being observed by the students doing the project.
RISKS
There are no foreseeable risks for participating in this research.
BENEFITS
There are no benefits to you as a participant other than to help students complete a course requirement for SYST469.
CONFIDENTIALITY
The data in this study will be confidential. Names and other identifiers will not be placed on questionnaires or other research data.
PARTICIPATION
Your participation is voluntary, and you may withdraw from the study at any time and for any reason. If you decide not to participate, or if you withdraw from the study, there is no penalty.
CONTACT
This research is being conducted the following students: Jack Jones & Hank Smith.
The faculty advisor is Jack Laveson of the System Engineering and Operations Research Department at George Mason University. He may be reached at XXXXXXXXXXfor questions, or to report a problem. You may contact the George Mason University Office of Research Subject Protections at XXXXXXXXXXif you have questions or comments regarding your rights as a participant in the research.
This research has been reviewed according to George Mason University procedures governing your participation in this research.
CONSENT
I have read this form and agree to participate in this study.
3. Participant given specific task instructions. “You will perform a series of tasks using the two operating systems. Do these tasks as you would do them at home following the steps on the sheets that we will give you. Remember that we are evaluating the operating systems, not you. Your first task is to uninstall an application listed on this paper.” You then handed the participant the task analysis steps for uninstalling an application for one of the operating systems. This was followed by the other tasks for the same operating system. Note that the first and third participant performed the tasks first for Windows 10 and then iOS X; the second and forth participant performed the tasks first for iOS X and then Windows 10.”
4. Participant performed tasks, during which you recorded time (from start of task to end of task). No e
ors were noted so e
ors will NOT be part of the results and conclusions. Report that no e
ors occu
ed.
5. Participant completed SUS after performing all tasks for the first operating system just dealt with. “For each of the following statements, mark one box that best describes your reactions to the tasks that you just performed with the operating system.”
6. Participant repeated the tasks for the other operating system followed by the completion of another SUS. “For each of the following statements, mark one box that best describes your reactions to the tasks that you just performed with the second operating system.”
7. At end of tasks for the second operating system, participant answered the opinion question, “Which operating system was easier to use, and why?”
8. Participant was thanked for participation. “Thank you for your time and effort. We really appreciate your help.”
Here are the time on tasks for the participants. Note that because the time was recorded in tenths of seconds, only report averages in tenths of seconds.
PARTICIPANT
TASK
Win 10 Time (sec.)
iOS X Time (sec.)
#1
Uninstall
29.4
36.2
Wallpape
20.0
12.0
Bluetooth
23.7
17.7
Time & Date
35.8
23.1
Zip Folde
19.1
10.4
#2
Uninstall
67.3
60.0
Wallpape
51.0
15.7
Bluetooth
30.8
42.8
Time & Date
32.1
41.2
Zip Folde
51.3
29.5
#3
Uninstall
37.5
14.5
Wallpape
25.7
5.9
Bluetooth
21.4
9.3
Time & Date
29.3
18.7
Zip Folde
20.9
7.3
#4
Uninstall
45.9
24.1
Wallpape
20.3
12.2
Bluetooth
18.5
20.8
Time & Date
23.3
25.5
Zip Folde
16.1
16.0
Each participant completed two SUS forms, one for Win 10 and the other for iOS X. The SUS forms are in the pdf file, completed SUS forms. You will need to score each of the eight forms. I suggest that you use the scoring technique found in the Lecture 9 PowerPoints, slide 30 (single version of PowerPoint lecture).
Lastly, the participant responses to the question, “Which operating system was easier to use, and why?” are as follows.
Participant #1 – iOS X because icons easier to understand.
Participant #2 – iOS X without any comments.
Participant #3 – iOS X because of better layout.
Participant #4 – Win 10 although iOS X also worked well.
PowerPoint Presentation
Student Project Report Format
PowerPoint report format (1/7)
Executive summary –
iefly, what did you do and what are your results?
What did you do: a limited usability field study of two products/apps specifying what these are
Who were the users/participants (no names, but OK to state students)
What tasks did you evaluate? List short titles, e.g., Enable “Rhythmic backlight alert”
Which product/app has better usability
This format follows that of a research report
2
Red a
ows XXXXXXXXXXindicate problem areas with prior student project submissions
PowerPoint report format (2/7)
Introduction – What are you doing, and why?
This is a usability test as a class project for SYST469
State definition of usability, and purpose of field study
Usability is the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use. [ISO XXXXXXXXXX: Usability (1998)]
The project is an evaluation of two existing interactive products based on data obtained from participants during a field study.
State the products/apps to be evaluated including specific models, operating system versions, etc. in sufficient detail that someone could buy exactly the same product/app
3
PowerPoint report format (3/7)
Method – describes the details of what you did
What tasks did you evaluate? List titles and, if needed,
ief descriptions – e.g., Enable “Rhythmic backlight alert” – makes screen flash on incoming call
Where did you do the testing?
Who were the users? Remember, no names or other personally identifying information as users are anonymous.
How were the tasks presented?
What was the presentation order?
Were task procedures given to the users / participants
A heads down protocol was used
4
PowerPoint report format (4/7)
Method – continued
Specifically, what did the user do?
User asked to participate
User advised of informed consent requirements – no need to include informed consent form
User give specific task instructions
User performed task, during which you recorded time and possibly e
ors
User completed SUS after performing all tasks for a product
Remember there will be two SUS scores/user, one for each product
Do not include SUS questionnaire as this is well known
At end of all tasks, user completed opinion question(s)
User thanked for participation
5
6
Hint:
- The numbers above the boxes were added after the user completed the SUS, but before scoring to make the job easier.
- It is OK for you to do this as well.
- Also remember to multiply the sum (example = 29) by 2.5 to get the SUS score.
PowerPoint report format (5/7)
Computed data – the information you collected about users performing the tasks
Provide computed data (not collected / raw data) in table format
Raw data will be submitted separately
Graph the computed data analysis in a way that you consider useful, however:
SUS has a range of 0 to 100 (so format axis manually from 0 – 100 to give the viewer the co
ect range of values)
TOT has a range of 0 to maximum time (so format the axis manually from 0 – maximum time)
Examples of computed data tables & graphs on next 3 slides
7
Raw / collected is NOT to be submitted; this was applicable only if you were collecting the raw data that you are not doing this semeste
Seconds Download & Install App Take Photo & Set As Wallpaper Create Group & Add 2 Apps Add A Contact Change Ringtone
Android 17.7 31.4 15.6 37.4 11.2
iOS 13.7 27.1 12.2 31.9 14.4
Mean Time/Task
Seconds
Format Vertical Axis so range is 0 to maximum time
Mean Time/Task
TOT by Task and Product
8
Android Download &