I encourage you to watch Sanskriti's blog for the final results, but I wanted to share a view into her excellent work. You might treat this as a preview of Sanskriti's results.
Sanskriti's usability test included about equal men and women, and about equally divided between "high" and "low" mobile OS exposure (but none at "moderate," which is interesting). Most were Windows OS users, and the testers were equally distributed in expertise. Her test participants had a good distribution in age groups. Click to view a larger version of each chart:
This is important, because GNOME wants to be useful to what I call "average users with average experience." And Sanskriti's usability test participants represent that.
These testers required an average of 38 minutes (±11 minutes) to do all scenario tasks in the full usability test. The median test time was 37.5 minutes. Click to see a larger version of the chart:
I asked Sanskriti to use the heat map method to display her usability test results. This is a method I developed during my master's program research, and refined in later usability testing. In a usability heat map, each scenario task in the usability test is displayed in rows, and each tester is shown as columns. Each tester's experience for every scenario task is represented using a colored block: green if the tester completed this task successfully with little or no problems, orange if the tester experienced some problems but was still able to complete the task successfully, red if the tester encountered great difficulty but still completed the task. And black if the tester was completely unable to finish the task.
Sanskriti's heap map shows some very useful and interesting results. Click to view a larger version of Sanskriti's data:
I can make a few initial observations from this data. Looks like testers had the most difficulty with tasks Gedit.6 and Photos.3 and Photos.4, with noticeable difficulty in tasks Notes.1 and Photos.2. There's some interesting data around tasks Gedit.1 and Music.1 that might reflect testers 9, 11, and 12.
This is only a preview of Sanskriti's results. I encourage you to watch Sanskriti's blog for the final results, which I hope to see in the next week as she wraps up her work in the internship.
image: Outreach Program for Women