wiki:org/teams/MetricsTeam

Metrics Team

Overview

Welcome to the Metrics Team page. The Metrics Team is a group of Tor people who care about measuring and analyzing things in the public Tor network. There are currently three other teams in Tor: the Network Team writes the software to keep the Tor network running, the Applications Team produces usable applications for end users, and the Community Team focuses on the non-technical aspects of growing a healthy Tor community. But if you're interested in measurements and analyses, the Metrics Team is what you're looking for. Welcome!

People

The Metrics Team currently consists of between half a dozen and a dozen developers, depending on how and when you count. We're not listing names here to keep the team open to everyone. You're on the team if you're participating in discussions and development, and you're not part of the team anymore if you decide you want to move on (which we hope won't happen).

Communication

Our synchronous medium of communication is IRC meetings that happen every Thursday at 14:30 UTC in #tor-dev on OFTC.

If you want to reach someone from the team between these meetings to ask a development-related question, just go to #tor-dev and mention the magic word "metrics", and somebody from the team might either be around or appear later and get back to you.

Our asynchronous medium of communication is the metrics-team@ mailing list. This list is public in the sense that anyone can subscribe and read archives. But it's moderated on first post, meaning that your first post will be reviewed to make sure it's not spam and on topic and all further posts will go directly to the list. Feel free to subscribe and just listen if you want, and feel free to post if you have a question that you think is on topic.

Products

The Metrics Team is not selling any products, don't worry.

Product is a codebase of software maintained by the Team. Not all products of the Metrics Team are mentioned in this section (for a more complete list see roadmap draft). Instead, we list some long-term products here that are or will be released in way that a third party can decide to run a mirror of this type of service.

Here some links to these products' pages (in alphabetical order):

  • CollecTor the friendly data collecting service. A wiki subsection for CollecTor mirror operators mirror operating wiki.
  • ExoneraTor helps to find out whether that address was used as a Tor relay.
  • Metrics the primary place to learn interesting facts about the Tor network.
  • metrics-lib a Java library that fetches and parses Tor descriptors.
  • Onionoo is a web-based protocol to learn about currently running Tor relays and bridges. A wiki subsection for Onionoo mirror operators mirror operating wiki

Releases and Milestones

Releases

August 8, 2017 milestone Onionoo 4.0-1.3.0 download Onionoo 4.0-1.3.0
July 13, 2017 milestone CollecTor 1.2.0 download CollecTor 1.2.0
June 28, 2017 milestone metrics-lib 2.0.0 download metrics-lib 2.0.0
June 20, 2017 milestone metrics-lib 1.9.0 download metrics-lib 1.9.0
June 16, 2017 milestone metrics-lib 1.8.2 download metrics-lib 1.8.2
June 8, 2017 milestone metrics-lib 1.8.1 download metrics-lib 1.8.1
June 6, 2017 milestone metrics-lib 1.8.0 download metrics-lib 1.8.0
May 17, 2017 milestone metrics-lib 1.7.0 download metrics-lib 1.7.0
February 28, 2017 milestone Onionoo 4.0-1.2.0 download Onionoo 4.0-1.2.0
February 17, 2017 milestone metrics-lib 1.6.0 download metrics-lib 1.6.0
January 27, 2017 milestone Onionoo 3.2-1.1.0 download Onionoo 3.2-1.1.0
January 17, 2017 milestone CollecTor 1.1.2 download CollecTor 1.1.2

older Metrics releases

Planned Milestones

CollecTor 1.3.0 CollecTor 2.0.0
metrics-lib 2.1.0 metrics-lib 3.0.0
ExoneraTor 1.0.0 ExoneraTor 2.0.0
MetricsWeb 1.0.0 MetricsWeb 2.0.0
Onionoo 1.4.0 Onionoo 2.0.0

Documentation

An FAQ by and about the Metrics Team. If something you want to know is not answered yet, check back in a few days. Or address your question directly to us via the communication channels listed in the previous section.

The 32C3 State of the Onion talk covered work of the Metrics Team by giving example of how bad relays are detected, investigated, and excluded from the Tor network. The entire talk is worth watching, but if you're short on time, be sure to watch minutes 20 to 28.

More documentation

Censorship Watch

MetricsTimeline lists incidents that might help in interpreting metrics graphs.

Here's a table with tickets related to censorship incidents:

Results (1 - 7 of 20)

1 2 3
Ticket Summary Status Priority Severity Reporter Modified
#22369 Increase of users in Ukraine due to block of Russia-based services new Medium Normal dcf 3 weeks ago
#21013 We need a platform for users to report censorship events new High Normal mrphs 2 months ago
#20348 Allot Communications blocking of vanilla Tor, obfs4, and meek in Kazakhstan, starting 2016-06 reopened Medium Normal dcf 2 months ago
#20785 Block of some direct users in Saudi Arabia, 2016-11-20 new Medium Normal dcf 2 months ago
#6140 Kazakhstan uses DPI to block Tor assigned Medium Normal runa 2 months ago
#21014 Turkey blocking of direct connections, 2016-12-12 new Medium Normal mrphs 8 months ago
#20907 Blocking of public relays in Belarus, 2016-12-01 new Medium Normal dcf 8 months ago
1 2 3

Roadmap from October 2016 to June 2017

The metrics team put together the following roadmap for the time from October 2016 to June 2017. This list was created for the Seattle meeting in September 2016 and will be updated on a regular basis and revisited at the next meeting around March 2017.

  • 2016-10: Provide user-friendly documentation that empowers users to independently operate CollecTor instances. (Sponsor X 1.2. CollecTor)
  • 2016-10: Enable CollecTor to synchronize Tor network data from other CollecTor instances. (Sponsor X 1.3. CollecTor)
  • 2016-10: Set up a second CollecTor instance and enable synchronization with the first CollecTor instance. (Sponsor X 1.4. CollecTor)
  • 2016-11: Define a release process for Onionoo and put out at least one release. (Sponsor X 3.1. Onionoo)
  • 2016-11: Provide user-friendly documentation that empowers users to independently operate Onionoo instances. (Sponsor X 3.2. Onionoo)
  • 2016-12: Conduct a usability analysis of the current Tor Metrics website to identify the most pressing usability issues. (Sponsor X 6.1. Metrics)
  • 2017-01: Add a visualization of Tor Browser downloads to Metrics. (Sponsor X 6.3. Metrics)
  • 2017-02: Improve usability of the most pressing issues identified in the earlier usability analysis as long as they are within scope to make it easier for users to find, compare, and interpret Tor usage and performance metrics. (Sponsor X 6.2. Metrics)
  • 2017-03: Write user-friendly tutorials for metrics-lib/DescripTor that empower users to independently analyze large amounts of network data. (Sponsor X 2.3. metrics-lib/DescripTor)
  • 2017-03: Improve all usability issues of Atlas that are classified as High priority or above in the issue tracker as long as they are within scope and reported at least six months before the end of the award. (Sponsor X 3.3. Atlas)
  • 2017-04: Replace all existing Torperf instances gathering current Tor network performance measurements with OnionPerf instances. (Sponsor X 5.1. OnionPerf)
  • 2017-04: Perform an analysis on reducing the amount of sensitive, potentially personally identifying data stored in memory of Tor relays and bridges or reported to the directory authorities. (Sponsor X 4.1. Tor daemon)
  • 2017-05: Develop and deploy at least one more user model in addition to the current model. (Sponsor X 5.2. OnionPerf)
  • 2017-05: Reduce the amount of sensitive, potentially personally identifying data stored in memory of Tor relays and bridges by implementing at least one suggestion from the earlier analysis document. (Sponsor X 4.2. Tor daemon)
  • 2017-05: Obfuscate data that gets reported by Tor relays and bridges to the directory authorities by implementing at least one suggestion from the earlier analysis document. (Sponsor X 4.3. Tor daemon)
  • 2017-06: Make a wider audience aware of the Tor descriptor parsing library metrics-lib/DescripTor by writing a blog post about it that explains how to use it. (Sponsor X 2.1. metrics-lib/DescripTor)

Task without specific sponsor or specific time:

  • Compare the Tor Metrics website code to RStudio's Shiny Server and other web frameworks.
  • Write an outline of further improvements or prototypes for all metrics-related Java products. (no specific sponsor)

Objectives and Key Results for the Metrics Team in Q2 2017

  • 1 Finish external deliverables (1/6 * (3/3 + 1/1 + 1/1 + 0/1 + 0/1 + 1/1))
    • 1.1 Complete replacing the 3 existing Torperf instances gathering current Tor network performance measurements with OnionPerf instances. (Sponsor X 5.1. OnionPerf) (3: deployed op-hk, op-nl, and op-us)
    • 1.2 Develop and deploy at least 1 more user model in addition to the current model. (Sponsor X 5.2. OnionPerf) (1: deployed onion service measurements model)
    • 1.3 Complete and publish 1 technical report on reducing the amount of sensitive, potentially personally identifying data stored in memory of Tor relays and bridges or reported to the directory authorities. (Sponsor X 4.1 Tor daemon) (1: published report)
    • 1.4. Reduce the amount of sensitive, potentially personally identifying data stored in memory of Tor relays and bridges by implementing new directory-request statistics based on requests by country, transport, and IP version and removing existing directory-request statistics based on unique IP addresses by country, transport, or IP version (Sponsor X 4.2. Tor daemon) (0: deferred because of simulation results)
    • 1.5. Obfuscate data that gets reported by Tor relays and bridges to the directory authorities by implementing noise for new directory-request statistics based on requests by country, transport, and IP version. (Sponsor X 4.3. Tor daemon) (0: deferred because of simulation results)
    • 1.6. Make a wider audience aware of the Tor descriptor parsing library metrics-lib/DescripTor and ideally increasing release tarball downloads by 50% by writing a blog post about it that explains how to use it. (Sponsor X 2.1. metrics-lib/DescripTor) (1: wrote blog post)
  • 2 Raise new funds to keep a head count of 2 developers after Sponsor X (1/2 * (1/1 + 1/1))
    • 2.1 Support next stages of current proposal if applicable. (1: done)
    • 2.2 Support other proposals if the current proposal doesn't work out. (1: not required)
  • 3 Reduce product backlog (1/4 * (0/1 + 5/5 + 4/5 + 0/5))
    • 3.1 Rewrite the webstats service as CollecTor module. (0: not done yet)
    • 3.2 Resolve 5 enhancement tickets opened by March 15 in components CollecTor, Metrics website, metrics-lib, and Onionoo. (5: resolved even 9 enhancements, but only the first 5 count here)
    • 3.3 Resolve 5 defect tickets opened by March 15 in components CollecTor, Metrics website, metrics-lib, and Onionoo. (4: resolved 4 defects)
    • 3.4 Resolve 5 tickets in needs_review opened by March 15 in components CollecTor, Metrics website, metrics-lib, and Onionoo. (0: resolved 3 defects in needs_review, but they're also counted above)
  • 4 Share operation of services (1/3 * (1/1 + 0/1 + 0/1))
    • 4.1 Share operation of CollecTor service among more than one person (currently run by karsten, mirror run by iwakeh). (1: done)
    • 4.2 Share operation of Onionoo service among more than one person (currently run by karsten, mirror run by karsten). (0: not done yet)
    • 4.3 Share operation of Metrics website among more than one person (currently run by karsten). (0: not done yet)
  • Evaluation: 1/4 * ((1/6 * (3/3 + 1/1 + 1/1 + 0/1 + 0/1 + 1/1)) + (1/2 * (1/1 + 1/1)) + (1/4 * (0/1 + 5/5 + 4/5 + 0/5)) + (1/3 * (1/1 + 0/1 + 0/1))) = 0.6125

Open Tickets

The following table lists all open tickets, from recently modified to not touched in years, of products that are maintained by Metrics Team people or otherwise related. Not all of these products are actively maintained, so be sure to reach out to the team before starting to hack on any of these and also take a look at the Volunteer's Guide. Also note that not all metrics-related tools use Tor's Trac as their bug tracker, so you might also want to ask for bug trackers of other metrics tools.

Results (1 - 20 of 308)

1 2 3 4 5 6 7 8 9 10 11
Ticket Summary Component Status Owner Priority Severity Created Modified Reporter
#16596 Change database queries towards making only a single query per request Metrics/ExoneraTor needs_review karsten Medium Normal 2 years ago 32 hours ago karsten
#23255 Fix a bug while sanitizing bridge network statuses without entries Metrics/CollecTor merge_ready metrics-team Very High Normal 3 days ago 3 days ago karsten
#21515 Add auxiliary data on Tor relays and bridges to CollecTor Metrics/CollecTor new metrics-team Medium Normal 6 months ago 3 days ago karsten
#23244 Onionoo documents should be the same accross all tp.o instances Metrics/Onionoo new metrics-team Medium Normal 4 days ago 3 days ago iwakeh
#23243 write a spec for web-server-access log descriptors Metrics/Metrics website needs_information metrics-team Medium Normal 4 days ago 4 days ago iwakeh
#22983 add a descriptor interface and implementation for web-logs Metrics/metrics-lib needs_revision metrics-team Medium Normal 4 weeks ago 4 days ago iwakeh
#21759 Add persistence for torperf/onionperf Metrics/CollecTor needs_review iwakeh Medium Normal 5 months ago 5 days ago iwakeh
#23211 limit Onionoo default listening to localhost Metrics/Onionoo new metrics-team Medium Normal 8 days ago 7 days ago iwakeh
#23215 set annotation from descriptor during sync-runs Metrics/CollecTor needs_information metrics-team Medium Normal 8 days ago 8 days ago iwakeh
#22428 add webstats module to collector Metrics/CollecTor needs_review iwakeh Medium Normal 3 months ago 9 days ago iwakeh
#23169 metrics site should explain why metrics are important and what we do to make sure they're safe Metrics/Metrics website new metrics-team Medium Normal 10 days ago 9 days ago arma
#21615 Use hashed fingerprint in all lookups Metrics/Atlas reopened irl Medium Normal 6 months ago 10 days ago cypherpunks
#23160 Adding flag makes list empty while seach result contains the flag Metrics/Atlas new irl Medium Normal 10 days ago 10 days ago qbi
#21427 allow to filter for tor_version Metrics/Onionoo needs_review metrics-team Medium Normal 6 months ago 12 days ago cypherpunks
#14201 Configure out/ directory path somewhere else than in web.xml. Metrics/Onionoo accepted iwakeh Medium Normal 3 years ago 12 days ago karsten
#15813 onionoo instances have distinct contact string encoding Metrics/Onionoo needs_review Medium Normal 2 years ago 12 days ago cypherpunks
#21138 add javadoc overview page to Onionoo Metrics/Onionoo new metrics-team Medium Normal 8 months ago 12 days ago iwakeh
#21370 Onionoo should be more robust against unexpected data Metrics/Onionoo new metrics-team Medium Normal 7 months ago 12 days ago iwakeh
#22287 Switch from custom CollecTor downloader to metrics-lib's DescriptorCollector Metrics/Onionoo needs_review metrics-team Medium Normal 3 months ago 12 days ago karsten
#15508 bandwidth sorting should be numeric and take the actual unit into account Metrics/Atlas new phw High Normal 2 years ago 12 days ago cypherpunks
1 2 3 4 5 6 7 8 9 10 11

Last modified 11 days ago Last modified on Aug 8, 2017, 2:25:20 PM