wiki:org/teams/MetricsTeam

Metrics Team

Overview

Welcome to the Metrics Team page. The Metrics Team is a group of Tor people who care about measuring and analyzing things in the public Tor network. There are currently three other teams in Tor: the Network Team writes the software to keep the Tor network running, the Applications Team produces usable applications for end users, and the Community Team focuses on the non-technical aspects of growing a healthy Tor community. But if you're interested in measurements and analyses, the Metrics Team is what you're looking for. Welcome!

People

The Metrics Team currently consists of between half a dozen and a dozen developers, depending on how and when you count. We're not listing names here to keep the team open to everyone. You're on the team if you're participating in discussions and development, and you're not part of the team anymore if you decide you want to move on (which we hope won't happen).

Communication

Our synchronous medium of communication is IRC meetings that happen every Thursday at 14:30 UTC in #tor-dev on OFTC.

If you want to reach someone from the team between these meetings to ask a development-related question, just go to #tor-dev and mention the magic word "metrics", and somebody from the team might either be around or appear later and get back to you.

Our asynchronous medium of communication is the metrics-team@ mailing list. This list is public in the sense that anyone can subscribe and read archives. But it's moderated on first post, meaning that your first post will be reviewed to make sure it's not spam and on topic and all further posts will go directly to the list. Feel free to subscribe and just listen if you want, and feel free to post if you have a question that you think is on topic.

Products

The Metrics Team is not selling any products, don't worry.

Product is a codebase of software maintained by the Team. Not all products of the Metrics Team are mentioned in this section (for a more complete list see roadmap draft). Instead, we list some long-term products here that are or will be released in way that a third party can decide to run a mirror of this type of service.

Here some links to these products' pages (in alphabetical order):

  • CollecTor the friendly data collecting service. A wiki subsection for CollecTor mirror operators mirror operating wiki.
  • ExoneraTor helps to find out whether that address was used as a Tor relay.
  • Metrics the primary place to learn interesting facts about the Tor network.
  • metrics-lib a Java library that fetches and parses Tor descriptors.
  • Onionoo is a web-based protocol to learn about currently running Tor relays and bridges. A wiki subsection for Onionoo mirror operators mirror operating wiki

Releases and Milestones

Releases

March 15, 2018 Onionoo 5.1-1.11.0, cf. announcement downloads
December 20, 2017 Onionoo 5.0-1.9.0, cf. announcement downloads
November 28, 2017 Onionoo 4.4-1.8.0, cf. announcement downloads
November 20, 2017 Onionoo 4.3-1.7.1, cf. announcement, preparing major version jump downloads
October 27, 2017 Joint release of milestones CollecTor 1.4.1, Onionoo 4.2-1.6.1, cf. announcement downloads
October 10, 2017 Joint release of milestones CollecTor 1.4.0, metrics-lib 2.1.1, Onionoo 4.2-1.6.0, cf. announcement downloads
September 15, 2017 Joint release of milestones CollecTor 1.3.0, metrics-lib 2.1.0, Onionoo 4.1-1.5.0, cf. announcement downloads
August 31, 2017 milestone Onionoo 4.1-1.4.0 and 4.1-1.4.1 announcement download Onionoo 4.1-1.4.1
August 18, 2017 announcement download CollecTor 1.2.1
August 8, 2017 milestone Onionoo 4.0-1.3.0 download Onionoo 4.0-1.3.0
July 13, 2017 milestone CollecTor 1.2.0 download CollecTor 1.2.0

older Metrics releases

Planned Milestones

CollecTor 1.5.0 CollecTor 2.0.0
metrics-lib 2.2.0 metrics-lib 3.0.0
ExoneraTor 1.0.0 ExoneraTor 2.0.0
MetricsWeb 1.0.0 MetricsWeb 2.0.0
Onionoo 2.0.0 Onionoo 2.1.0

Documentation

An FAQ by and about the Metrics Team. If something you want to know is not answered yet, check back in a few days. Or address your question directly to us via the communication channels listed in the previous section.

The 32C3 State of the Onion talk covered work of the Metrics Team by giving example of how bad relays are detected, investigated, and excluded from the Tor network. The entire talk is worth watching, but if you're short on time, be sure to watch minutes 20 to 28.

More documentation

Censorship Watch

MetricsTimeline lists incidents that might help in interpreting metrics graphs.

Here's a table with tickets related to censorship incidents:

Results (1 - 7 of 10)

1 2
Ticket Summary Status Priority Severity Reporter Modified
#20348 Allot Communications blocking of vanilla Tor, obfs4, and meek in Kazakhstan, starting 2016-06 reopened Medium Normal dcf 3 days ago
#30794 Create lightweight censorship analyser for users assigned Medium Normal phw 7 days ago
#30500 Can the GFW still do DPI for "new" vanilla Tor? assigned Low Normal phw 3 weeks ago
#30872 Test BridgeDB's distribution channels in controlled experiment new Medium Normal phw 5 weeks ago
#29855 TOR was blocked by my company. needs_information Medium Normal wanje 4 months ago
#28898 Huge drop of users in UAE new Medium Normal anadahz 7 months ago
#28748 Казахстан new Medium Normal Ridvanz 8 months ago
1 2

Roadmap from October 2017 to September 2018

We put together a roadmap for 2017/18 in Berlin in September 2017, refined it at the Tor meeting in Montreal in October 2017, and updated it at a team-internal meeting in Aberdeen in April 2018. The following contains a very short version of goals for the time until September 2018 and will be updated over time:

Short-term goals (Q4/2017)

Code
  1. Update all code to use Java 8 features (M; 50% of 50% done, deferred)
  2. Switch ExoneraTor and metrics-web from Tomcat to embedded Jetty, and harmonize directory structures (M; 100% done)
Operation
  1. Deploy better notification system for operational issues (M; 30% of 30% done, deferred)
  2. Make sure that each service has at least two operators (M; 30% done)
Sources
  1. Add web server logs as new data source (M; 100% done)
News
  1. Provide metrics timeline events as both a table on Tor Metrics pages and as an RSS/Atom feed that is also syndicated via Twitter to increase community engagement (M; 95% done)
Services
  1. Make Atlas part of the Tor Metrics website (M; 100% done)
  2. Make Compass part of Atlas in order to provide aggregated search results and details pages on Atlas and to finally shut down Compass (M; 100% done)
  3. Resolve ExoneraTor database slowness (M; 100% done)
  1. Extend Onionoo to use JNDI for rDNS lookups (M; 100% done)

Visualizations
  1. Add OONI graphs to Tor Metrics based on pre-aggregated data from vanilla Tor test and bridge reachability test (M; 45% done)
  2. Add IPv6 relay graphs to Tor Metrics based on existing descriptor contents (M; 100% done)
  1. Add a few more performance graphs based on existing OnionPerf data (M; 100% done)
About
  1. Rename Tor Metrics components to make them easier to understand for people outside of the team (M; 45% of 45% done, deferred)

Medium-term goals (until Q3/2018)

Research
  1. Document aggregated statistics files better (Sponsor 13) (8/5 L; 100% of originally planned 80% done)
  2. Put external research data on the website, including Rob's phantomtrain OnionPerf data and Yixin's BGP monitoring data (M; 30% done)
Sources
  1. Re-process bridge descriptor tarballs to retain “contact” lines in server descriptors and “fingerprint” lines in statuses (M; 25% of 25% done, deferred)
  2. Support PrivCount development and integrate its data as successor to statistics reported by relays and bridges (M; 100% done)
  3. Help the Core Tor and Tor Browser teams with ongoing measurements (M; 100% done)
About
  1. Document the Tor Metrics system/pipeline, including explicitly writing down our requirements (Sponsor 13) (4/5 L; 20% of originally planned 10% done, deferred)

Burndown Chart

Draft roadmap from October 2018 to March 2019

In Mexico City, we started a draft roadmap for the time until March 2019. The following tasks are still subject to change.

  1. Extend OnionPerf to measure performance of Tor with mobile networks and/or PTs (requires #26673). (2M)
  2. Rewrite CollecTor relaydescs module using Stem/txtorcon, at least consensuses and server descriptors as proof of concept (#28320). (2M, Sponsor 13; 100% done)
  3. Write technical report documenting the metrics code base, the CollecTor prototype, the OONI comparison, and recommendation (#25644). (M, Sponsor 13; 75% done)
  4. Evaluate at least one other (not MaxMind) GeoIP database that could be used to improve data analysis and visualisation within Tor Metrics. (S)
  5. Review information architecture for Tor Metrics (#24422) and update to Bootstrap 4. (M)
  6. Write an initial CollecTor module for PrivCount including format specification discussions. Make some internal graphs (not on Tor Metrics) with: number of relays supporting each statistics version in each consensus; number of relays actually reporting statistics in each day; number of tally reporters processing statistics. (M)
  7. Finish operational issues notifications (#28322). (M; 30% done)
  8. Add a metrics-web module to add the consensus weight per bwauth graph from #25459. Requires using votes in metrics-web. (M; 100% done)
  9. Run arthur's DNS timeout scanner, archive it in CollecTor, and add it to Onionoo. (2M)
  10. Finish OONI vanilla Tor data. (M)
  11. Create an additional information lookup service using remote APIs (RIPEstat) and local databases (MaxMind). Consider adding back the relays by country graph to Tor Metrics. (2M)
  12. Finish having 2 operators for each service (#28327). (S)
  13. Make a plan for adding external research data to either Tor Metrics or the research portal. (S; 15% done)
  14. Rewrite or extend TorDNSEL with Stem to support IPv6. (2M)
  15. Extend dir-spec to serve recent, non-current consensuses and votes (#21378) and CollecTor to fetch them (#28324). Requires help by the network team. (M)
  16. Review CollecTor's use of file system storage and explore alternatives, potentially ZFS. (M; 10% done)
  17. Finish adding Java 8 date/time (#23752). (M; 30% done)
  18. Add sbws data to CollecTor (#21378) and to Onionoo (#28321). (M)
  19. Improve the graph from #25459. (S; 100% done)
  20. Clean up metrics-web codebase in order to move away from Ant to just Java (2M, 90% done)

Roadmap from February 2019 to June 2019

In Brussels 2019 we reviwed the roadmap, added new projects and shape it into a kanban board that will be revisited weekly: https://storm.torproject.org/shared/TsgBadLSfM8uh_aftPjxmR_rm0a8E-4UQ2HEa_l0DHv

Priorities from July 2019 to December 2019

  1. exit scanner (MVP is the priority)
  2. performance metrics
  3. complete the bandwith file archive (with Tom's file)
  4. snowflake and PT related work
  5. better notification system
  6. irtf draft for guidelines for safe internet measurement

Open Tickets

The following table lists all open tickets, from recently modified to not touched in years, of products that are maintained by Metrics Team people or otherwise related. Not all of these products are actively maintained, so be sure to reach out to the team before starting to hack on any of these and also take a look at the Volunteer's Guide. Also note that not all metrics-related tools use Tor's Trac as their bug tracker, so you might also want to ask for bug trackers of other metrics tools.

Results (1 - 20 of 324)

1 2 3 4 5 6 7 8 9 10 11
Ticket Summary Component Status Owner Priority Severity Created Modified Reporter
#25625 Make CollecTor's file structure description part of Metrics-Web's CollecTor docs Metrics/Website assigned metrics-team Medium Normal 16 months ago 6 hours ago iwakeh
#27980 Missing server descriptors in recent/ but not in out/ Metrics/CollecTor assigned metrics-team Medium Normal 9 months ago 6 hours ago karsten
#29425 Write integration tests for data-processing modules Metrics/Statistics assigned metrics-team Medium Normal 5 months ago 6 hours ago karsten
#29507 Evaluate existing OnionPerf data regarding worst-case performance Metrics/Analysis assigned metrics-team Medium Normal 5 months ago 6 hours ago karsten
#30303 Add "archived" badge to graphs that are not updated anymore Metrics/Website assigned metrics-team Medium Normal 3 months ago 6 hours ago karsten
#30602 Extend TorperfResult to return parsed new PARTIAL* and ERRORCODE fields Metrics/Library assigned metrics-team Medium Normal 8 weeks ago 6 hours ago karsten
#30610 Extend onionperf module to import new PARTIAL* and ERRORCODE fields Metrics/Statistics assigned metrics-team Medium Normal 8 weeks ago 6 hours ago karsten
#31071 Add a notice if we're missing data for a lookup Metrics/ExoneraTor assigned metrics-team Medium Normal 2 weeks ago 6 hours ago karsten
#25217 metrics-base: Update checkstyle to debian stable Metrics assigned karsten Low Minor 17 months ago 7 hours ago iwakeh
#31204 Extend file objects in index.json to include descriptor types, publication times, and file digests Metrics/CollecTor new metrics-team Medium Normal 13 hours ago 13 hours ago karsten
#29461 Add a Snowflake module Metrics/CollecTor new metrics-team Medium Normal 5 months ago 26 hours ago irl
#31102 Produce guidelines for safe internet measurement Metrics/Analysis new metrics-team Medium Normal 11 days ago 31 hours ago irl
#31172 Tests fail on Debian buster on jenkins Metrics/Library accepted karsten High Normal 2 days ago 31 hours ago irl
#31198 Upgrade ExoneraTor to Debian buster libraries Metrics/ExoneraTor new metrics-team Medium Normal 38 hours ago 38 hours ago karsten
#31197 Upgrade metrics-web to Debian buster libraries Metrics/Website new metrics-team Medium Normal 38 hours ago 38 hours ago karsten
#31196 Upgrade Onionoo to Debian buster libraries Metrics/Onionoo new metrics-team Medium Normal 38 hours ago 38 hours ago karsten
#31195 Upgrade CollecTor to Debian buster libraries Metrics/CollecTor new metrics-team Medium Normal 38 hours ago 38 hours ago karsten
#31193 Upgrade to Debian buster libraries Metrics new metrics-team Medium Normal 38 hours ago 38 hours ago karsten
#31194 Upgrade metrics-lib to Debian buster libraries Metrics/Library needs_review metrics-team Medium Normal 38 hours ago 38 hours ago karsten
#29624 New version of exit list format Metrics/Exit Scanner new metrics-team Medium Normal 5 months ago 3 days ago irl
1 2 3 4 5 6 7 8 9 10 11

Last modified 11 days ago Last modified on Jul 8, 2019, 5:06:44 PM

Attachments (12)

Download all attachments as: .zip