Version 38 (modified by lnl, 2 years ago) (diff)


A Usability Evaluation of Tor Launcher

This project evaluates how easy it is for a user to connect to Tor with qualitative (behavioral and attitudinal) methods, makes design changes according to Tor-specific considerations, and verifies that the changes helped with quantitative (measurement-oriented) methods.

Users take more than 10 minutes to connect to Tor if the public relays are censored and 50% of the users cannot connect to Tor if the interface’s hardcoded bridges are censored. The suggested design changes saves users a lot of time--over 20 minutes if they are in censored environments that censor hardcoded bridges.

Designing changes was especially challenging because of atypical, Tor-specific design considerations. You might think that it would be easiest for the user if the process of setting up the connection to Tor was completely automated, and you're right. But it doesn't account for the fact automating the process can put some users at risk or that the relays that work most reliably are magnitudes more expensive and limited in capacity (to name a few reasons). Anyone interested in doing UX work for Tor would especially benefit from reading about these considerations.


  • Linda Lee (@linda): user experience researcher
  • David Fifield (@dcf): computer security researcher
  • Nathan Malkin: user experience researcher
  • Ganesh Iyer: user interface designer
  • Serge Egelman: project adviser
  • David Wagner: project adviser


  • Oct - Dec 2015: user research
  • Jan - Dec 2016: interface redesigns, experiments, paper writing
  • Jan - Mar 2017: paper acceptance, paper polishing
  • Jul 2017: paper presentation at PETS


  • find out why and where users struggle connecting to Tor
  • make changes to the Tor Launcher interface to address those problems
  • measure the impact of those changes to evaluate if we should implement the changes

Design considerations

If you're going to only read one section, read this one: these don't require an understanding of the specific UI component being evaluated or our experimental methodologies, but will be helpful to know if you work on Tor.

Third-party knockoffs

It would be easy to ask the user something like, "what country are you in?," and give them advice. Or ask "are you at risk?" or "do you know what you are doing?" to offer manual configuration while defaulting to automatic configuration.

While leveraging user input works well, a software designed for anonymity requiring de-anonymizing information makes make users feel uneasy. But even worse than that is that there are benign and malicious third-party versions of Tor Browser on the internet, and not all users download Tor Browser from the official site or official mirror. Since these replicate the official version, having user input in ours will result in impersonations having user input as well--and they won't be so nice with the information as we would.

Network eavesdroppers

Assuming that there was a sufficiently powerful network adversary, a failed, non-obfuscated attempt to connect to Tor will tell the network eavesdropper that the user is connecting to Tor. Depending on who the user is, this may or may not be dangerous.

Most state-level governments are capable of this level of surveillance. So a user in China or Iran must be careful to really get the connection right the first time, and to use protocols that hide the fact that they are using Tor. If we automated the connection process and assigned everyone relays that run these protocols, those relays would be overloaded. If we tried the most likely thing that would work first, then their eavesdropper will know that they are using Tor.

User Consent

Since being revealed that using Tor comes with some risk (in some countries, encryption is illegal!), it's critical to give users agency over the situation. Letting them configure the components makes them realize, at the minimum, that they are attempting to connect to Tor.

With the current interface (before the proposed changes), over half the people make a mistake connecting to Tor. While it's easy to say that we would make less mistakes and could help them by choosing for them, automatically taking action that could have consequences for the user without proper consent is not ethical.

Financial constraints

There are a set of relays which almost work all the time (meek bridges)--they are hard to block because blocking them would cause a lot of other sites to be blocked, and governments don't want to blanket-block all of them. We could leverage these to ensure that people connect to Tor the first time safely, then assign them something that would work afterward.

But meek relays cost tens of thousands of dollars to run a month at their current capacity (easily the most expensive relays to run), and using them more would cost even more money.

Time constraints

Network environments change all the time--what bridge was reachable in a country an hour ago might not be tomorrow. Keeping up with how and when each country censors Tor would require a lot of time and would require multiple people to do it as their full time job. We measure what is going on after-the-fact, but then we would have made some mistakes on the users' behalf, which may put them at risk without their consent.

Even if we could keep up with how and when each country censors Tor, we would need to push changes to account for this before any of the affected users connect to Tor, which is an unreasonable turnaround time.

User pain points

We found out why and where users struggled with the interface with 1) an interface audit by user experience researchers, 2) live user observations of real-life people using this interface in a controlled setting, and 3) user answers to interview questions following the experiment.

Blanket user targeting

we don't differentiate between high/low risk users, users that do/don't know what they are doing, or users who are using it for nice-to-have security, anonymity, or censorship circumvention.

Missing information

Some transports work in some countries, while others do not. It depends if the country has figured out how the transport works and decide to block it. Some transports cost a lot (meek) while others cost less money.

None of this is really indicated in the interface. We would argue that these are details that should be abstracted for the user, so that they shouldn't need to know. But currently, they would be required to know to make the correct decision, which is not good.

Nonexistent mental model

tor-specific concepts, complicated, internet isn't even clear to them

Design changes and results

With the Tor-specific design considerations in mind, we made a changes according to usability heuristics, such as increasing visibility of system status, leveraging recognition rather than recall, and matching the system with the real world. We also try to reduce text, make it aesthetically pleasing, and played with some graphics.

Proposed UI changes

10 main changes here.

Resulting Improvements

People saved a lot of time.

Future UI changes

We believe that these changes will help, but don't prescribe them exactly. We did not test our design changes independently, so we cannot (and do not) claim the effects each change. Instead of specific changes, we offer suggest these ways of alleviating user pain points:

  • Hiding the proxy screen and less-used transports: 1/200+ participants configured a proxy correctly and 0/200+ participants chose flashproxy, fte, fte-ipv6, or scramblesuit transports.
  • Discourage users from configuring optional components: many participants configured proxies and bridges that they did not need.
  • Redirect to relevant screens on error: i.e., redirect users to the bridge screen if the bridge is unreachable, instead of requiring users to diagnose the source of the problem and navigate manually.
  • Take advantage of the progress screen: we can use the minutes users spend here to educate users about what Tor does, or introduce Tor Browser.

We plan to work on making changes to Tor Launcher to improve its usability in 2017!


The PETS paper (to come in July 2017!) documents the usability testing with academic rigor, and explains this project more in depth.

I've summarized the most relevant supplementary information in these sections below.

Scope and context

We restricted ourselves to making design changes that would not ignore any of the design considerations and not require any changes to existing infrastructure.

Although Tor Browser was originally designed for Internet anonymity, many now use Tor Browser to circumvent Internet censorship. Tor Launcher mainly exists so that people who face Internet censorship can configure bridges, transports, and proxies to connect to Tor.

Technical background

Tor is an anonymity network that routes Internet traffic through a series of relays that make it difficult to observe the source and destination. Tor Browser is a modified Firefox browser with a built-in Tor client that is the recommended way to use Tor. Tor Launcher is a Tor Browser component that starts, stops, and otherwise controls the underlying Tor processes. Tor Launcher’s graphical user interface asks the user to configure bridges, pluggable transports, and proxies to make a connection to Tor. This is the object of our study.

There are many valid configuration settings to connect to Tor. A user who does not need a bridge or proxy can connect with a bridge, proxy, or both, provided that they are configured correctly.

Users rarely need proxies, and need bridges if they are circumventing censorship. The user must first successfully connect to the Internet through their proxy before trying to connect to a Tor relay or bridge.

When users choose a a hard-coded transport/bridge option in the interface, they use pre-configured “bridge lines,” a bridge specification that includes its IP address, transport type, and other metadata officially configures Tor Browser to use bridges. They can enter this manually in a the "custom bridge" option, but this requires the user to get the formatting exactly right (and there are no directions).


We performed a cognitive walkthrough by working tasks from the perspective of the user and assessing its learnability for new or infrequent users. We then conducted qualitative and quantitative usability tests of the interface. We believed that both were necessary for a complete evaluation. Qualitative testing tells you why or how it is or is not usable, while quantitative testing tells you how much it is or is not usable.

We simulated three environments of increasing severity in censorship. We chose to simulate environments for the stability and reproducibility of the experiment, as real censored networks are volatile and complex. These are not intended to imitate any particular country’s censorship environment, but is sufficient for testing the configuration interface since these environments require the users to take the all interface paths we wanted to test.

We used these industry standard metrics to measures users’ abilities to complete tasks:

  • Completion rate: percentage of users that con- nect to Tor in an experimental condition.
  • Connection time: time from Tor Browser startup to a successful connection to Tor.
  • Configuration time: time users spent in Tor Launcher configuring bridges and proxies, not counting time spent passively waiting.

Funding and collaboration

This work was Linda's master's thesis while she was getting her master's degree in computer science at UC Berkeley. This work was funded by the National Science Foundation (only because she was an NSF fellow) and Intel (because her academic advisers were funded by Intel). Neither funding source requested criteria or changes to this research, but did make sure that Linda had money to work on this. This work was done in collaboration with SCRUB (secure computing research for users' benefit) Laboratory and BLUES (Berkeley laboratory for usable and experimental security) Laboratory, because that's where she worked while studying at Berkeley. Neither research institution requested criteria or changes to this research, but supported this research with its testing resources (computers, participants, etc).

Attachments (7)

Download all attachments as: .zip