Opened 3 years ago

Closed 3 years ago

#21221 closed task (fixed)

Perform usability testing for the mobile security slider

Reported by: lnl Owned by: linda
Priority: Medium Milestone:
Component: Webpages Version:
Severity: Normal Keywords: orfox, tor browser, tbb-mobile
Cc: isabela, gk, n8fr8, brade, mcs, fdsfgs@… Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

Background

Tor is broadly aiming to 1) prioritize mobile users and 2) make the desktop version of Tor browser and Tor browser mobile (currently orfox, but will be rebranded) consistent. The security slider allows the user to choose their security level while using tor browser. Since this is not on the mobile application yet, adding this feature accomplishes both of the aforementioned goals. While adding the feature to the mobile version, we made some changes from the desktop version to improve the interface and make the UI more mobile-appropriate. If you are curious about how and why we made the changes to the security slider you can look at the project page.

Scope

The purpose of this test is to see if the security slider is usable. Specifically, the metrics for success are:

  • Control: the user understands how to change their settings.
  • Communication: the user understands what each setting does (literally understand--“to make the setting from safe to safer, I slide this button over”, not that they understand what each setting means or how it is technically implemented).
  • Comprehension: the user knows how the settings will affect their browsing (again, this along the lines of “this is what I want,” “this makes this feature go away and I do/don’t like that” not that they understand how JS is a vector for exploits or even is aware of the difference between http and https websites).
  • Decision: the user is able to make the decision that they want.

The ideal case is that the user will see the interface, and be able to say, something along the lines of: “when I move the slider, the javascript turns off, which will break some functionality. I don’t want that because I visit a lot of sites using Tor as my everyday browser.” | “when I toggle this, the browser goes into secure mode, which is the bare minimum of functionality to operate. It’s conservative and breaks a lot, but I’m okay with that because I want to feel safe.”

We are not testing: if the Tor browser mobile is easy to find on the app store, if people are able to install it on their phone, if it is easy to use, or if the security slider is easy to find in the menu.

Methodology

We are running a small-scale, short, qualitative, open-ended, qualitative user test.

  • Small-scale: we are testing with 5 users. Read about when it is appropriate if you are curious.
  • Short: aim for a maximum of 5 minute interview. The purpose of the interview is to get their impression of the UI (does the user take a second before they can explain how to use it? Are they confused about the settings?) rather than their understanding (can they explain why we used a horizontal slider versus a vertical one? Do they understand what turning off javascript does?). The longer the interview, the more it tends to lean towards their understanding, which is not what we want.
  • Open-ended: users are free to answer the question in their own words. This is to allow the users to speak more freely and to communicate naturally. (the opposite would be closed ended, and would involve asking the user to rank/rate/label the interface, “how hard is it from 1 to 10?. This works better for larger groups, but not small groups.)
  • Qualitative: we are focusing on user attitude and comprehension (do they like it, will they use it, are they able to use it). We are not concerned with user behavior (how many people use which setting, how many clicks, how many seconds) for this phase in testing.

Materials

What we will give to our partners as background and instructions:
https://drive.google.com/open?id=1aE1txcRdJ9u5Der2-k9i-eq8mzvyPyAdmrPGlVKyPxo

Interview questions and feedback form:
https://drive.google.com/open?id=1yYCLZtZuvK_0EICCiXbxQTHFJUhRHJ_rJgdMO5AR-uA

Child Tickets

Change History (7)

comment:1 Changed 3 years ago by lnl

Summary: perform usability testing for the mobile security sliderPerform usability testing for the mobile security slider

comment:2 Changed 3 years ago by gk

Cc: gk added; geko removed
Keywords: tbb-mobile added

comment:3 Changed 3 years ago by mcs

Cc: brade mcs added

comment:4 Changed 3 years ago by tokotoko

Cc: fdsfgs@… added

comment:5 Changed 3 years ago by linda

Owner: changed from lnl to linda
Status: newassigned

comment:6 Changed 3 years ago by linda

The testing has been completed; the summary of the experiment, recommendations on what to do next, and the next action items are here: https://trac.torproject.org/projects/tor/wiki/doc/UX/OrfoxSecuritySlider#Testing

comment:7 Changed 3 years ago by linda

Resolution: fixed
Status: assignedclosed
Note: See TracTickets for help on using tickets.