wiki:doc/UX/MetricsRedesign

Version 27 (modified by lnl, 2 years ago) (diff)

explain better

Redesigning the Tor Metrics Page

The ​Tor metrics page is the primary place to learn interesting facts about the Tor network, displaying safely measured things.

Currently, the metrics page is functional but not usable. All measured Tor metrics can be found on the website, but the organization, visualization, and explanation need work. The metrics team has been wanting to make the metrics website more usable for a while now; specifically, to "make visualizations more intuitive, but also to reduce the click count to reach information and to make or keep information accessible to users who rely on screen readers." Karsten has even experimented with alternative ways to display information here and here.

Let's make the metrics page as useful as possible to a wide range of users (casual Tor Browser users, journalists, developers, and researchers)!

Goals (wrt Sponsor X):

  • Conduct a usability analysis of the current Tor Metrics website to identify the most pressing usability issues (deliverable 6.1)
  • Improve usability of the most pressing issues identified in the earlier usability analysis as long as they are within scope to make it easier for users to find, compare, and interpret Tor usage and performance metrics (deliverable 6.2)

People:

  • Karsten, Metrics Team
  • Iwakeh, Metrics Team
  • Isabela, UX Team
  • Linda, UX Team
  • <contracted web designer>

Process:

  1. Karsten, Linda, and Isabela independently inspect the usability of the metrics page
  2. Linda, and Isabela mock up alternative designs of the metrics page
  3. Karsten, Iwakeh, and Linda meet in Berlin to discuss the results of the inspection and iterate through potential designs
  4. Karsten, Iwakeh, and Linda decide on what changes should be made
  5. Karsten and Linda talk to the designer with list of changes and sketches of potential features
  6. Karsten, Linda, and the designer iterate through the page design
  7. Karsten, Linda, and the designer decide on the final design
  8. Linda evaluates the usability of the metrics site, before and after
  9. The designer makes the appropriate changes

Timeline:

  • 1st week Nov: Finish inspecting the metrics page
  • 2nd week Nov: Meet in person to brainstorm ideas
  • 3rd week Nov: Iterate through ideas and designs
  • 4th week Nov: Finalize design and start evaluation
  • December: Designer starts making changes

Background

A brief history of the metrics page, to explain why it looks the way it does today. Summarized from Karsten:

The metrics website started out as static website. Pre-selected graphs were updated in a background process and displayed.

The first major improvement was to let users customize what they see. Graphs were generated with user inputs from HTML forms (i.e. as date, country). Along with this, simple data tables were introduced to people could see the raw data.

The next major improvement was to separate graphs and tables into individual pages. This improved the search functionality on the start page. To compensate for this, data pages were created to show the underlying data and graphs and tables that used the same data were linked together.

Things we want to preserve:

  • avoid javascript as much as possible
  • make everything run on Debian stable

An incomplete dump of past iterations of the metrics page:

Top 10 graphs for the metrics page (from here):

  1. Direct users by country (17.92%)
  2. Bridge users by country (11.34%)
  3. Relays and bridges in the network (4.56%)
  4. Relays with Exit, Fast, Guard, Stable, and HSDir flags (3.74%)
  5. Total relay bandwidth in the network (3.29%)
  6. Unique .onion addresses (2.89%)
  7. Hidden-service traffic (2.46%)
  8. Bridge users by transport (2.33%)
  9. Relays by platform (1.83%)
  10. Relays by version (1.83%)

Inspection

Metrics Team's Usability Report

The metrics team, or more precisely Karsten and iwakeh, have summarized the most pressing issues that he sees with the website. They have written a report on the current usability of the metrics website. Here are issues they found:

  • Issue 1.1: Little introduction of Tor and Tor Metrics
  • Issue 1.2: No links to related services
  • Issue 1.3: Links to metrics subpages should not be text
  • Issue 1.4: Links to technical reports are missing
  • Issue 1.5: Website seriously lacks web design
  • Issue 2.1: There are far too many graph subpages
  • Issue 2.2: There are no explanations of events in the data
  • Issue 2.3: Additional information is distributed all over
  • Issue 2.4: Graphs are not as interactive as they could be
  • Issue 3.1: Tables should be combined with graphs
  • Issue 4.1: Distinction to official graphs is unclear

Linda's Usability Inspection

Linda identified seven types of users: casual visitors, curious outsiders, journalists, researchers, relay operators, Tor developers, and Tor authorities. The respective user types user stories, tasks, and cognitive walkthrough are below.

User profiles:

  • Casual visitor: a person who wants to know a bit about Tor (i.e. Tor Browser users/sponsors).
  • Curious outsider: a person who is affected by Tor users (i.e. website operators, police).
  • Journalist: a person who wants to write about Tor (i.e. internet freedom activist).
  • Researcher: a person who has a lot of technical skills and knowledge (i.e. academics).
  • Relay operator: a person who runs Tor relays and bridges.
  • Tor developer: a person who works on Tor (i.e. devs on the network team).
  • Tor authority: a person who runs a core part of Tor network infrastructure (i.e. directory authority/BridgeDB operators).

User stories:

  • Casual visitor: I want to see high-level facts that summarize Tor so I can see what's going on.
  • Curious outsider: I want to know how Tor works so that I can respond to how Tor users affect me.
  • Journalist: I want to see evidence of the latest events so that I can write about it.
  • Researcher: I want data, tools, and resources so that I can properly research Tor.
  • Relay operator: I want to know about Tor relays so that I can help make Tor bigger, faster, and more diverse.
  • Tor developer: I want to know how my changes affected Tor and what to work on next.
  • Tor authority: I want detailed logs of events so that I can keep the network stable and safe.

User tasks:

  • Casual visitor: find a summary of users, relays, throughput, and countries.
  • Curious outsider: find how Tor works with respect to the internet (exit nodes).
  • Journalist: find censorship events, find censorship history of my country, find what works in my country.
  • Researcher: find all raw data, measurement and support tools, any research references.
  • Relay operator: find relay details (throughput, capacity, diversity) and what is the best way to help the network.
  • Tor developer: find system overview, data related to my service (transport, onion service, etc)
  • Tor authority: find system overview, any unusual events (spikes/dips, relays down)

Isabela's Industry Examination

Isabela reviewed twitter's analytics tool to see how industry standard user interfaces were. Her comments are here. Here were some of her observations and suggestions.

  • Observation 1: There is a dashboard-like feel to the interface, which gives a quick overview of the latest metrics over the last day, trending tweets, and other interesting information.
  • Suggestion 1: Give users a summary of the analytics that they are intersted in (we tabled this for a later iteration. we want to make a dashboard-like feature, but not this time.)
  • Observation 2: Users can quickly jump between different information and manipulate the timeframe of the data.
  • Suggestion 2: Improve the flow for users to see different metrics on the metrics website.
  • Observation 3: The analytics platform performs basic analyses for the user, such as sorting by popularity, converting to percents, and showing changes over time.
  • Suggestion 3: Use visualizations such as sparklines, pie charts, bars, etc. to illustrate basic information visually.

Design

We made changes to improve the usability, but chose not to add any new metrics or new data manipulation related features until another time.

Pain points

Pain point 1: lots of text makes it hard to locate what you want to find.

Currently, the metrics site is a single page with a list of all of the metrics that Tor metrics collect.

There are options to organize them in the lefthand side of the website, but this requires that the user read all of those things, choose what they want to see, click update, re-read the updated list, and choose their metric. This takes too long, and users literally need to read everything on the webpage.

Pain point 2: having to go back and forth to click on different links is annoying.

Once you click on a particular metric, you are redirected to a graph page where you can get a rough visualization of the metric.

To go compare with similar metrics, you would need to hit the back button, go back, read the text again, find your other metric, and click on that.

Pain point 3: only one metrics is displayed at a time so you couldn't compare similar metrics

There was no way to display multiple related metrics side by side to compare them. There are lots of metrics on that users would want to compare, such as the number of relays and the number of bridges. But to do that, you would need to open up two tabs.

No image "metric2" attached to doc/UX/MetricsRedesign

and since the tabs often had different-length descriptions at the top, the graphs wouldn't be at the same spot on the screen so shifting back and forth between tabs (which is inconvenient enough already) is not a good way to compare graphs unless users spend time adjusting the graphs to be at the same spot on their screens.

Changes made

Screenshots are from the prototype, which can be found here. We're still hacking away at it, so it may look a little different than the screencaptures we inserted below.

Change 1: fixed user workflow

There is no long list of metrics for users to read. Users now choose from a simple menu about the specific topic that they are curious about.

Once the users are there, they can see the graph, can click to other related graphs with a single click, and compare these graphs by clicking back and forth these subtabs.

(We would eventually even want to plot multiple metrics at the same time, or allow multiple graphs on a single page, but we saved that for a later time)

Change 2: added new pages

We added subpages to the metrics page.

  • Home: A home page that greets users and explains subpages.
  • About: Metrics team methodology, philosophy, contact, faq, and glossary.
  • News: Recent events that were recorded with Tor metrics (such as a censorship event).
  • Data: A place to see, manipulate, and download metrics taken by the team.
  • Sources: How and with what software the metrics were measured with.
  • Tools: Other Tor-related tools that provide some other measurements.
  • Research: Research papers and references that are useful to know or use tor metrics.

Here are screenshots of some of the pages that did not exist before. We hope that these help the diverse group of users that visit the metrics website to find more of what they are curious about, provide transparency into the metrics, and encourage use of these metrics or other related tools for research.

Change 3: Interface Makeover

We followed standard UX practices, to build a responsive, less text-heavy, visual website that looked modern. The prototype was styled with respect to this style guide.


Summary

A tl;dr blurb about the project with select links to resources (a prototype that summarizes our work?).

Implementing changes

say when these changes will be in effect and when the site will launch

For future projects

other things that we were going to do, such as making a dashboard, being able to see multiple screens, switching to a new graphing engine, etc.


Misc Notes and Logs

Brainstorming Changes

Linda's ideas to help with user tasks:

  • Splash page: a page to give an overview of Tor and metrics.
  • Navigation banner: redirect users to specific topics (overview, research tools, etc.)
  • Explain: a page that explicitly states what is not collected for metrics
  • Compression: being able to see multiple graphs on one screen, plot multiple things on a graph
  • Tagging: explain events, missing data, and anomalies and displaying those tags
  • Grouping: group information by specific country, transport/hidden service, event (tagged event)
  • Using tagging: using tagged events as inputs instead of manual dates (i.e. "5 days before and after this version release")
  • Downloadable graphs: data available as a csv/json file (for researchers?)
  • Transparency: links and references to underlying data collection services
  • Help: links and references to measurement tools for allowing users to collect their own metrics
  • Dashboard: customize the use of the metrics page

Karsten's navigation ideas (from suggestions above):

  • News: new confirmed censorship events, new releases, etc.
  • Graphs: overview of graphs
  • Dashboard: configurable set of sparklines
  • Tools: overview and links to ExoneraTor, Atlas, Onionoo clients
  • Data: overview and links to CollecTor, also documentation
  • Development: parsing libraries like metrics-lib, Onionoo API
  • Research: how to cite us, selected papers, tech reports
  • About: about the project, full Schneier quote, FAQ, contribute

Wireframes

Using Balsamiq, we wireframed mockups that reflected our changes. This is what the web designer had to work with. There are little thumbnails attached inline, but you can download these in full size from the attachments section of this wiki page.

Berlin Meeting

Linda and the metrics team met face to face in Berlin (Nov 8th, 2016)!

Agenda:

  1. Review Karsten's, iwakeh's, Linda's, and Isabela's usability inspections.
  2. Discuss Karsten's, iwaheh's, Linda's, and Isabela's suggested changes, and decide on which we should follow through on.
  3. Use pencil and paper to draft through potential metrics pages, experimenting with layout and features.
  4. Prioritize the list of changes to make, so that we can tell the web designer to work from the top down.
  5. Come up with a usability evaluation (so we can say something about how our improvements made impact).
  6. Discuss how to apply the style guide to the website.

Summary:

  • We decided to only implement changes that improved the usability of the website. Specifically, this includes: page organization/redesign to help various users, visualizations that do not require another graphing engine. (notes)
  • We sketched out how we want the new metrics website to look like. (sketches)
  • We brainstormed functional changes that could be made, and reserve this for future work. We talked about: additional graph customizations, additional data formats, a user dashboard, event tagging, and additional metrics. (notes on graphing functionalities, notes on additional features)

Todos:

  • Karsten will investigate how to create sparklines in R.
  • Karsten will create some content for the new metrics page.
  • Linda will create a content page to iterate through the text to be put on the new metrics page.
  • Linda will make a rough mock up of the tor metrics website for the web designer to see before the IRC meeting (Nov 17th, 2016).

IRC Meetings

Attachments (23)