wiki:doc/UX/MetricsRedesign

Version 25 (modified by lnl, 19 months ago) (diff)

--

Redesigning the Tor Metrics Page

The ​Tor metrics page is the primary place to learn interesting facts about the Tor network, displaying safely measured things.

Currently, the metrics page is functional but not usable. All measured Tor metrics can be found on the website, but the organization, visualization, and explanation need work. The metrics team has been wanting to make the metrics website more usable for a while now; specifically, to "make visualizations more intuitive, but also to reduce the click count to reach information and to make or keep information accessible to users who rely on screen readers." Karsten has even experimented with alternative ways to display information here and here.

Let's make the metrics page as useful as possible to a wide range of users (casual Tor Browser users, journalists, developers, and researchers)!

Goals (wrt Sponsor X):

  • Conduct a usability analysis of the current Tor Metrics website to identify the most pressing usability issues (deliverable 6.1)
  • Improve usability of the most pressing issues identified in the earlier usability analysis as long as they are within scope to make it easier for users to find, compare, and interpret Tor usage and performance metrics (deliverable 6.2)

People:

  • Karsten, Metrics Team
  • Iwakeh, Metrics Team
  • Isabela, UX Team
  • Linda, UX Team
  • <contracted web designer>

Process:

  1. Karsten, Linda, and Isabela independently inspect the usability of the metrics page
  2. Linda, and Isabela mock up alternative designs of the metrics page
  3. Karsten, Iwakeh, and Linda meet in Berlin to discuss the results of the inspection and iterate through potential designs
  4. Karsten, Iwakeh, and Linda decide on what changes should be made
  5. Karsten and Linda talk to the designer with list of changes and sketches of potential features
  6. Karsten, Linda, and the designer iterate through the page design
  7. Karsten, Linda, and the designer decide on the final design
  8. Linda evaluates the usability of the metrics site, before and after
  9. The designer makes the appropriate changes

Timeline:

  • 1st week Nov: Finish inspecting the metrics page
  • 2nd week Nov: Meet in person to brainstorm ideas
  • 3rd week Nov: Iterate through ideas and designs
  • 4th week Nov: Finalize design and start evaluation
  • December: Designer starts making changes

Background

A brief history of the metrics page, to explain why it looks the way it does today. Summarized from Karsten:

The metrics website started out as static website. Pre-selected graphs were updated in a background process and displayed.

The first major improvement was to let users customize what they see. Graphs were generated with user inputs from HTML forms (i.e. as date, country). Along with this, simple data tables were introduced to people could see the raw data.

The next major improvement was to separate graphs and tables into individual pages. This improved the search functionality on the start page. To compensate for this, data pages were created to show the underlying data and graphs and tables that used the same data were linked together.

Things we want to preserve:

  • avoid javascript as much as possible
  • make everything run on Debian stable

An incomplete dump of past iterations of the metrics page:

Top 10 graphs for the metrics page (from here):

  1. Direct users by country (17.92%)
  2. Bridge users by country (11.34%)
  3. Relays and bridges in the network (4.56%)
  4. Relays with Exit, Fast, Guard, Stable, and HSDir flags (3.74%)
  5. Total relay bandwidth in the network (3.29%)
  6. Unique .onion addresses (2.89%)
  7. Hidden-service traffic (2.46%)
  8. Bridge users by transport (2.33%)
  9. Relays by platform (1.83%)
  10. Relays by version (1.83%)

Inspection

Metrics Team's Usability Report

The metrics team, or more precisely Karsten and iwakeh, have summarized the most pressing issues that he sees with the website. They have written a report on the current usability of the metrics website, which states the following problems:

  • Issue 1.1: Little introduction of Tor and Tor Metrics
  • Issue 1.2: No links to related services
  • Issue 1.3: Links to metrics subpages should not be text
  • Issue 1.4: Links to technical reports are missing
  • Issue 1.5: Website seriously lacks web design
  • Issue 2.1: There are far too many graph subpages
  • Issue 2.2: There are no explanations of events in the data
  • Issue 2.3: Additional information is distributed all over
  • Issue 2.4: Graphs are not as interactive as they could be
  • Issue 3.1: Tables should be combined with graphs
  • Issue 4.1: Distinction to official graphs is unclear

Their suggestions to the above problems (summarized from the report):

  • Issue 1.1: give a brief overview on the start page and link to interesting things
  • Issue 1.2: link to CollecTor, Onionoo, ExonerTor, and other sources that metrics depends on
  • Issue 1.3: use thumbnails of graphs (rather than words) to indicate various subpages
  • Issue 1.4: add links to technical reports in a non-noisy way for researchers
  • Issue 1.5: make the website look more Tor-like
  • Issue 2.1: reduce the number of subpages; give users a 'dashboard' of with graphs they are interested in.
  • Issue 2.2: provide an interface to add explanations of events in the data
  • Issue 2.3: provide a high-level summary of important things
  • Issue 2.4: generate graphs on the client side for speed and add functionalities
  • Issue 3.1: put tables and graphs together
  • Issue 4.1: distinguish third-party visualizations from official metrics visualizations

Linda's Usability Inspection

As a UX researcher, Linda inspected the usability of the website with respect to various users.

The purpose of this inspection was to identity who would use the metrics website, for what reason, and if the metrics site is usable to that user for that purpose. If the user 1) knows what to do at the page to accomplish the task, 2) knows if they did the right thing to accomplish that task, and 3) can finish the task, then the metrics site will be considered usable for that type of user.

We identified seven types of users: casual visitors, curious outsiders, journalists, researchers, relay operators, Tor developers, and Tor authorities. The respective user types user stories, tasks, and cognitive walkthrough are below.

User profiles:

  • Casual visitor: a person who wants to know a bit about Tor (i.e. Tor Browser users/sponsors).
  • Curious outsider: a person who is affected by Tor users (i.e. website operators, police).
  • Journalist: a person who wants to write about Tor (i.e. internet freedom activist).
  • Researcher: a person who has a lot of technical skills and knowledge (i.e. academics).
  • Relay operator: a person who runs Tor relays and bridges.
  • Tor developer: a person who works on Tor (i.e. devs on the network team).
  • Tor authority: a person who runs a core part of Tor network infrastructure (i.e. directory authority/BridgeDB operators).

User stories:

  • Casual visitor: I want to see high-level facts that summarize Tor so I can see what's going on.
  • Curious outsider: I want to know how Tor works so that I can respond to how Tor users affect me.
  • Journalist: I want to see evidence of the latest events so that I can write about it.
  • Researcher: I want data, tools, and resources so that I can properly research Tor.
  • Relay operator: I want to know about Tor relays so that I can help make Tor bigger, faster, and more diverse.
  • Tor developer: I want to know how my changes affected Tor and what to work on next.
  • Tor authority: I want detailed logs of events so that I can keep the network stable and safe.

User tasks:

  • Casual visitor: find a summary of users, relays, throughput, and countries.
  • Curious outsider: find how Tor works with respect to the internet (exit nodes).
  • Journalist: find censorship events, find censorship history of my country, find what works in my country.
  • Researcher: find all raw data, measurement and support tools, any research references.
  • Relay operator: find relay details (throughput, capacity, diversity) and what is the best way to help the network.
  • Tor developer: find system overview, data related to my service (transport, onion service, etc)
  • Tor authority: find system overview, any unusual events (spikes/dips, relays down)

Ideas to help with user tasks:

  • Splash page: a page to give an overview of Tor and metrics.
  • Navigation banner: redirect users to specific topics (overview, research tools, etc.)
  • Explain: a page that explicitly states what is not collected for metrics
  • Compression: being able to see multiple graphs on one screen, plot multiple things on a graph
  • Tagging: explain events, missing data, and anomalies and displaying those tags
  • Grouping: group information by specific country, transport/hidden service, event (tagged event)
  • Using tagging: using tagged events as inputs instead of manual dates (i.e. "5 days before and after this version release")
  • Downloadable graphs: data available as a csv/json file (for researchers?)
  • Transparency: links and references to underlying data collection services
  • Help: links and references to measurement tools for allowing users to collect their own metrics
  • Dashboard: customize the use of the metrics page

Karsten's navigation ideas (from suggestions above):

  • News: new confirmed censorship events, new releases, etc.
  • Graphs: overview of graphs
  • Dashboard: configurable set of sparklines
  • Tools: overview and links to ExoneraTor, Atlas, Onionoo clients
  • Data: overview and links to CollecTor, also documentation
  • Development: parsing libraries like metrics-lib, Onionoo API
  • Research: how to cite us, selected papers, tech reports
  • About: about the project, full Schneier quote, FAQ, contribute

Isabela's Industry Examination

As an ex-industry developer, Isabela provides recommendations based on industry best practices.

Isabela's reviews:


Discussion

Berlin Meeting

Linda and the metrics team met face to face in Berlin (Nov 8th, 2016)!

Agenda:

  1. Review Karsten's, iwakeh's, Linda's, and Isabela's usability inspections.
  2. Discuss Karsten's, iwaheh's, Linda's, and Isabela's suggested changes, and decide on which we should follow through on.
  3. Use pencil and paper to draft through potential metrics pages, experimenting with layout and features.
  4. Prioritize the list of changes to make, so that we can tell the web designer to work from the top down.
  5. Come up with a usability evaluation (so we can say something about how our improvements made impact).
  6. Discuss how to apply the style guide to the website.

Summary:

  • We decided to only implement changes that improved the usability of the website. Specifically, this includes: page organization/redesign to help various users, visualizations that do not require another graphing engine. (notes)
  • We sketched out how we want the new metrics website to look like. (sketches)
  • We brainstormed functional changes that could be made, and reserve this for future work. We talked about: additional graph customizations, additional data formats, a user dashboard, event tagging, and additional metrics. (notes on graphing functionalities, notes on additional features)

Todos:

  • Karsten will investigate how to create sparklines in R.
  • Karsten will create some content for the new metrics page.
  • Linda will create a content page to iterate through the text to be put on the new metrics page.
  • Linda will make a rough mock up of the tor metrics website for the web designer to see before the IRC meeting (Nov 17th, 2016).

IRC Meetings


Design

We made changes to improve the usability, but chose not to add any additional functionalities (such as new metrics or new visualizations).

Design Changes

What we're trying for:

  • help all user profiles mentioned above (casual visitor, curious outsider, journalist, researcher, relay operator, Tor developer, Tor authority) to complete some tasks
  • add any helpful information that does not require any additional backend work or a new graphing engine.
  • make it visually appealing!

Specific changes:

  • add additional navigational pages to group information together (i.e. all the research-related resources in a page).
  • add sources, services, research, and news pages.
  • add a summary of the most popular metrics on the homepage to give casual visitors an idea of Tor's status that day.
  • make the metrics about page more visible to get to, and add paragraphs introducing the team, what data is not collected, etc.
  • reducing information on a given page to prevent information overload.
  • rewording text on the metrics page.

Wireframes

Using Balsamiq, we wireframed mockups that reflected our changes. This is what the web designer will have to work with.

home: shows an overview of Tor and directs users to other pages.

about: an about page that introduces the team, explains terms, and answers faqs.

data: graphs, tables, csvs, links, and more! (the main part of metrics)

sources: a description of where and how Tor metrics gets their data.

services: links to additional services that may be helpful.

research: a page to help researchers gather their own data and find references.

news: where blog posts and metrics graphs are linked together.

Prototype

The prototype was styled with respect to this style guide.

Check out the website in progress here.

Some screenshots (at some point during the dev process) are included down below.


Summary

A tl;dr blurb about the project with select links to resources (a prototype that summarizes our work?).

Attachments (23)