Metrics timeline workshop

Resources

• doc/MetricsTimeline

• ethercalc workspace

• all graphs for a country on one page

• giant graph of relay users for all countries

• giant graph of bridge users for all countries

• motivating graphs (#20216 (moved)):

  • Iran relay users
  • Iran bridge users
  • Iran bridge users by transport
  • all bridge users by transport

• graph of meek users, annotated using entries from the timeline:

• mockups of how metrics graphs could look with annotations:

Objective

The dream behind the doc/MetricsTimeline is to one day add annotations to Tor Metrics graphs—so when a graph shows a sudden drop in users from Belarus, for example, there will also be an annotation and a link to ticket:20907. Basically, to put the metrics graphs in context and aid people in interpreting them.

In the metrics timeline workshop, we will work on expanding the coverage of the metrics timeline, in three ways:

1. Mining other data sources (such as this list of blocking orders from Brazil) for relevant events and transcribing them into the metrics timeline.
2. Visually examining graphs for apparent anomalies and adding them to the doc/MetricsTimeline#Unknown for later investigation.
3. Researching the entries in the "Unknown" category to find any possible explanations.

Notes

Date: 2017-10-12\ Facilitator: David Fifield\ Session wiki: https://trac.torproject.org/projects/tor/wiki/org/meetings/2017Montreal/AgendaIdeas#Metricstimelineworkshop\ Tables with event timelines online at https://trac.torproject.org/projects/tor/wiki/doc/MetricsTimeline

• David shows graphs of connections from Iran

  • Directly-connecting users shows a couple of significant drops
  • Data comes from directory mirrors
  • Adding graph of bridge users indicates users switched to using bridges
  • Adding bridge of bridge users by pluggable transport shows that some specific transports were blocked but others were not
  • Compared to global users, which shows universal drop in obfs3 users, which leads one to realize that the drop was a statistics anomaly caused by the time between the Bridge Authority Tonga went offline and Bifröst came online

• David shows list of events he has been compiling that collectively can help explain patterns and features of metrics graphs

• David shows monster list of per-country Tor users from 2011 to present

  • Each graph shows a noticeable increase in users in 2013, which we know is due to a botnet using Tor but which is frequently misinterprited

• Options for group

  1. Look for interesting patterns in per-country user counts
  2. Search the Internet for events that can explain interesting changes in Tor statistics
  3. Research the entries in the "Unknown" category to find any possible explanations.

• Example of ideal metrics annotation, spike in users from Bangladesh (due to block of Facebook, WhatsApp, and Viber)

• News timeline data already put on metrics website https://metrics.torproject.org/news.html

• No current plan for ongoing updating and maintaining event data related to metrics

• OONI has a similar need of real-world explanations for its measurements

• Analytics packages exist with timeline annotations?

• Group proceeds to identify interesting features of graphs of per-country user numbers 2011-present.

• Things that might be useful to keep in mind when explaining graph features

  • Change in GeoIP database
  • Change in the way that Directory Authorities or Bridge Authorities work
  • Smaller countries have fewer users, and so changes are less likely to be related to any Tor or censorship event, and moreover it is probably harder to find data about events an that country due to less coverage
  • Tor Browser and Orbot releases, which sometimes contain new default bridges

• Interesting methods of analysis

  • Anomaly detector (e.g. from Joss Wright)
  • Joint analysis of relay and bridge users
  • Use Google Trends to search for popularity of Tor-related search terms over time broken down by country

• Discussion of how to get reports explaining metrics observations

  • Everybody is invited to contribute to the wiki at https://trac.torproject.org/projects/tor/wiki/doc/MetricsTimeline. These events will ideally be overlaid/annotating Tor Metrics graphs. The input would ideally be machine readable to allow this annotation to be done automatically. Perhaps this would be better done as a form for users.
  • Present an anomaly to affected users and ask them to give suggestions for explanations. Can operate through existing network of people partnering with OONI.
  • Infolabe anomalies mailing list does send anomalies to subscribed users.