PageOutline

= Redesigning the Tor Metrics Page =

We made Tor metrics page easier to use and more useful more people. You can find background about the page and its common use cases here.

We inspected the website, brainstormed design changes, drafted wireframes, built a prototype, and will push changes to the Tor metrics page soon.

People:

• Linda, UX Team
• Karsten, Metrics Team
• iwakeh, Metrics Team
• Isabela, UX Team
• Raphael Bergmann, web designer (contracted)
• hiro, admin team

Timeline:

• Nov 2016: usability analysis, brainstorming, wireframing, setting things up for the web designer
• Dec 2016: prototyping and iterating the website until we're happy with it

Goals for Sponsor X:

• Conduct a usability analysis of the current Tor Metrics website to identify the most pressing usability issues (deliverable 6.1)
• Improve usability of the most pressing issues identified in the earlier usability analysis as long as they are within scope to make it easier for users to find, compare, and interpret Tor usage and performance metrics (deliverable 6.2)

The scope of the project was to make design changes. We would like to add any new metrics or new data manipulation features, but reserve that for future work.

Pain points

Lots of text

Currently, the metrics site is a single page with a list of all of the metrics that Tor metrics collect. Users need to read through each metric to locate what you want to find.

There are options to filter and sort them in the lefthand side, but this requires that the user read all of those options, choose, update, and re-read an updated list.

Bad navigation

Having to go back and forth to click on different links to see different metrics is annoying.

Once you click on a particular metric, you are redirected to its graph page.

To go to another metric, you would need to hit the back button, go back, read the whole list of all the metrics (lots of text, again..), find your other metric, and click on that.

Single graph page

Currently, there is only one metric is displayed at a time. To compare them, users open up two tabs.

But each subpage often had different-length descriptions at the top, so it is annoying to switch back and forth to compare the two.

Solutions

Screenshots are from the prototype, which can be found here.

=== Simplified navigation ===

Users now choose from a simple menu about the specific topic that they are curious about.

No long list of metrics to read and no settings for filtering, selecting, and displaying metrics.

Users see related metrics in the tabs above the graph. To see metrics on a different topic, they select from the menu again.

=== Level tabs ===

Once the users select a topic, they can see the graphs of related metrics by clicking back and forth these subtabs. We fixed the graph window so they can compare the graphs easier.

(We also wanted to display multiple graphs on a page, update the graphing engine, and cool stuff! But that's out of scope for this particular project.)

= Additional changes =

=== New subpages ===

We added subpages to the metrics page based on usability inspections and brainstorming.

* Home: A home page that greets users and explains subpages. 
* About: Metrics team methodology, philosophy, contact, faq, and glossary. 
* News: Recent events that were recorded with Tor metrics (such as a censorship event). 
* Data: A place to see, manipulate, and download metrics taken by the team. 
* Sources: How and with what software the metrics were measured with. 
* Tools: Other Tor-related tools that provide some other measurements. 
* Research: Research papers and references that are useful to know or use tor metrics. 

We attached some screenshots of some of the pages that did not exist before:

_ the home page! We haven't decided what to put here exactly._

_ the news page, that explains real life events that were measured._

_ the research page, with its recommendations._

We hope that this informs users of what's going on, provides transparency into the metrics, and encourages use of these metrics for research.

UI makeover

We followed standard UX practices, to build a responsive, less text-heavy, visual website that looked modern.

The prototype was styled with respect to this style guide.

= Appendix =

Background

Tor metrics collects information about Tor in a safe manner and displays the results.

The metrics website is started out as static website with pre-selected graphs which were updated in a background process and displayed. Graphs were generated with user inputs from HTML forms (i.e. as date, country). Later, data tables were introduced to people could see the raw data.

Later, the metrics page had a start page with a side bar that allowed users to select which metrics they wanted to see. Graphs were separated into individual pages, graphs and tables that used the same data were linked together, and data had about pages that explained what they were. This still how the website is today.

A dump of past iterations of the metrics page that show its evolution:

• January 2016
• March 2015
• October 2014
• October 2014
• June 2014

For the consideration of our users, we want to:

• preserve functionality even if javascript is turned off
• make things run on Debian in a stable way

Currently, the top 3 graphs for the metrics page according to webstats are:

1. Direct users by country (17.92%)
2. Bridge users by country (11.34%)
3. Relays and bridges in the network (4.56%)

We placed the things that were most viewed by the current users at the top in our new design.

Usability Inspections

Metrics Team's Usability Report

The metrics team, or more precisely Karsten and iwakeh, have summarized the most pressing issues that he sees with the website. They have written a report on the current usability of the metrics website. Here are issues they found:

• Issue 1.1: Little introduction of Tor and Tor Metrics
• Issue 1.2: No links to related services
• Issue 1.3: Links to metrics subpages should not be text
• Issue 1.4: Links to technical reports are missing
• Issue 1.5: Website seriously lacks web design
• Issue 2.1: There are far too many graph subpages
• Issue 2.2: There are no explanations of events in the data
• Issue 2.3: Additional information is distributed all over
• Issue 2.4: Graphs are not as interactive as they could be
• Issue 3.1: Tables should be combined with graphs
• Issue 4.1: Distinction to official graphs is unclear

Linda's Usability Inspection

Linda identified seven types of users: casual visitors, curious outsiders, journalists, researchers, relay operators, Tor developers, and Tor authorities. The respective user types user stories, tasks, and cognitive walkthrough are below.

User profiles:

• Casual visitor: a person who wants to know a bit about Tor (i.e. Tor Browser users/sponsors).
• Curious outsider: a person who is affected by Tor users (i.e. website operators, police).
• Journalist: a person who wants to write about Tor (i.e. internet freedom activist).
• Researcher: a person who has a lot of technical skills and knowledge (i.e. academics).
• Relay operator: a person who runs Tor relays and bridges.
• Tor developer: a person who works on Tor (i.e. devs on the network team).
• Tor authority: a person who runs a core part of Tor network infrastructure (i.e. directory authority/BridgeDB operators).

User stories:

• Casual visitor: I want to see high-level facts that summarize Tor so I can see what's going on.
• Curious outsider: I want to know how Tor works so that I can respond to how Tor users affect me.
• Journalist: I want to see evidence of the latest events so that I can write about it.
• Researcher: I want data, tools, and resources so that I can properly research Tor.
• Relay operator: I want to know about Tor relays so that I can help make Tor bigger, faster, and more diverse.
• Tor developer: I want to know how my changes affected Tor and what to work on next.
• Tor authority: I want detailed logs of events so that I can keep the network stable and safe.

User tasks:

• Casual visitor: find a summary of users, relays, throughput, and countries.
• Curious outsider: find how Tor works with respect to the internet (exit nodes).
• Journalist: find censorship events, find censorship history of my country, find what works in my country.
• Researcher: find all raw data, measurement and support tools, any research references.
• Relay operator: find relay details (throughput, capacity, diversity) and what is the best way to help the network.
• Tor developer: find system overview, data related to my service (transport, onion service, etc)
• Tor authority: find system overview, any unusual events (spikes/dips, relays down)

Isabela's Industry Examination

Isabela reviewed twitter's analytics tool to see how industry standard user interfaces were. Her comments are here. Here were some of her observations and suggestions.

• Observation 1: There is a dashboard-like feel to the interface, which gives a quick overview of the latest metrics over the last day, trending tweets, and other interesting information.

• Suggestion 1: Give users a summary of the analytics that they are intersted in (we tabled this for a later iteration. we want to make a dashboard-like feature, but not this time.)

• Observation 2: Users can quickly jump between different information and manipulate the timeframe of the data.

• Suggestion 2: Improve the flow for users to see different metrics on the metrics website.

• Observation 3: The analytics platform performs basic analyses for the user, such as sorting by popularity, converting to percents, and showing changes over time.

• Suggestion 3: Use visualizations such as sparklines, pie charts, bars, etc. to illustrate basic information visually.

=== Brainstorming ===

Linda's ideas to help with user tasks:

• Splash page: a page to give an overview of Tor and metrics.
• Navigation banner: redirect users to specific topics (overview, research tools, etc.)
• Explain: a page that explicitly states what is not collected for metrics
• Compression: being able to see multiple graphs on one screen, plot multiple things on a graph
• Tagging: explain events, missing data, and anomalies and displaying those tags
• Grouping: group information by specific country, transport/hidden service, event (tagged event)
• Using tagging: using tagged events as inputs instead of manual dates (i.e. "5 days before and after this version release")
• Downloadable graphs: data available as a csv/json file (for researchers?)
• Transparency: links and references to underlying data collection services
• Help: links and references to measurement tools for allowing users to collect their own metrics
• Dashboard: customize the use of the metrics page

Karsten's navigation ideas (from suggestions above):

• News: new confirmed censorship events, new releases, etc.
• Graphs: overview of graphs
• Dashboard: configurable set of sparklines
• Tools: overview and links to ExoneraTor, Atlas, Onionoo clients
• Data: overview and links to CollecTor, also documentation
• Development: parsing libraries like metrics-lib, Onionoo API
• Research: how to cite us, selected papers, tech reports
• About: about the project, full Schneier quote, FAQ, contribute

=== Berlin Meeting Agenda===

Linda and the metrics team met face to face in Berlin (Nov 8th, 2016)!

Agenda:

1. Review Karsten's, iwakeh's, Linda's, and Isabela's usability inspections.
2. Discuss Karsten's, iwaheh's, Linda's, and Isabela's suggested changes, and decide on which we should follow through on.
3. Use pencil and paper to draft through potential metrics pages, experimenting with layout and features.
4. Prioritize the list of changes to make, so that we can tell the web designer to work from the top down.
5. Come up with a usability evaluation (so we can say something about how our improvements made impact).
6. Discuss how to apply the style guide to the website.

Summary:

• We decided to only implement changes that improved the usability of the website. Specifically, this includes: page organization/redesign to help various users, visualizations that do not require another graphing engine. (notes)
• We sketched out how we want the new metrics website to look like. (sketches)
• We brainstormed functional changes that could be made, and reserve this for future work. We talked about: additional graph customizations, additional data formats, a user dashboard, event tagging, and additional metrics. (notes on graphing functionalities, notes on additional features)

Todos:

• Karsten will investigate how to create sparklines in R.
• Karsten will create some content for the new metrics page.
• Linda will create a content page to iterate through the text to be put on the new metrics page.
• Linda will make a rough mock up of the tor metrics website for the web designer to see before the IRC meeting (Nov 17th, 2016).

Wireframes

Using Balsamiq, we wireframed mockups that reflected a bunch of the ideas that we came up with during brainstorming. This is what we gave the web designer had to work with, and you will notice that the prototype looks very similar to these wireframes.

Here are some thumbnails of the prototype. You can click on them to see the full sized version.

[[Image(about-new.png, 100px)]

=== Prototypes ===

version 1: throwing out ideas that worked. screenshots, comments

version 2: iteration from comments above, changes to homepage, site structure, and content grouping screenshots, comments

version 3: iteration from comments above, added a new secondary menu bar, finalized content, debugged some stuff. screenshots, general feedback, design feedback

After incorporating the public and design feedback, we launched the new version of Tor Metrics.