Opened 10 years ago

Closed 3 years ago

#2963 closed task (wontfix)

Instrument Tor client to count directory requests and bridge connections

Reported by: karsten Owned by:
Priority: Medium Milestone:
Component: Metrics/Analysis Version:
Severity: Normal Keywords:
Cc: arma, ahf Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

When estimating user numbers, we have to make assumptions a) on the number of directory requests that an average user makes and b) on the average number of bridge connections that a bridge client opens.

We should instrument a Tor client to log all directory requests and bridge connections, so that we can derive better data from these logs. In fact, the info-level logs of the Tor clients that we use for Torperf may already tell us something about directory requests.

Child Tickets

Attachments (4)

consensus-downloads.csv (7.6 KB) - added by karsten 7 years ago.
consensus-downloads.png (40.5 KB) - added by karsten 7 years ago.
torperf50kb-2013-07-04.log.gz (233.8 KB) - added by karsten 7 years ago.
consensus-downloads.2.png (61.9 KB) - added by karsten 7 years ago.

Download all attachments as: .zip

Change History (13)

comment:1 Changed 10 years ago by karsten

Owner: karsten deleted
Status: newassigned

comment:2 Changed 9 years ago by arma

Component: MetricsAnalysis

comment:3 Changed 7 years ago by karsten

In #7009 we wanted to know how often clients download new consensuses. Changed log level of the torperfs on ferrinii to info, will report some stats here in a week from now.

Changed 7 years ago by karsten

Attachment: consensus-downloads.csv added

Changed 7 years ago by karsten

Attachment: consensus-downloads.png added

comment:4 Changed 7 years ago by karsten

Cc: arma added

Attached a .csv and a .png file containing successfully loaded consensuses by ferrinii's tor clients. In particular, I'm counting occurrences of the following log line:

[info] connection_dir_client_reached_eof(): Successfully loaded consensus.

comment:5 Changed 7 years ago by arma

Excellent.

These data points do not match up with the assumption "clients fetch a new consensus every 2-4 hours"!

So there is either (1) a bug in how you're deciding that we fetched a new consensus, or (2) we are in fact fetching them more often than we thought (and, it would seem, more often than we need to).

Or I guess (3) Torperf's Tors are fetching them more often than regular Tors.

Changed 7 years ago by karsten

comment:6 Changed 7 years ago by karsten

I just attached the info-level log of the 50KiB torperf from July 4. This is Tor version 0.2.2.35 (git-b04388f9e7546a9f). Also, here's the torrc:

DataDirectory .
SocksPort 9020
MaxCircuitDirtiness 1 minute
UseEntryGuards 0
RunAsDaemon 1
Log info file log
ControlPort 10020
CookieAuthentication 1
PidFile tor.pid

Am I looking at the right log lines, or what lines should I look at?

Also, please note that any change we make to the consensus download interval has huge impact on our user-counting algorithm. If we make future tors smarter about downloading consensuses less frequently, we'll have a hard time adapting our user-counting algorithm.

Changed 7 years ago by karsten

Attachment: consensus-downloads.2.png added

comment:7 Changed 7 years ago by karsten

Attached an updated graph and switched back to notice-level logging on ferrinii.

comment:8 Changed 4 years ago by arma

Cc: ahf added
Severity: Normal

ahf: you might find this ticket interesting, as the torperf traces have a bunch of logs that could be reconstructed to answer questions about frequency of dir fetches.

comment:9 Changed 3 years ago by karsten

Resolution: wontfix
Status: assignedclosed

Closing tickets in Metrics/Analysis that have been created 5+ years ago and not seen progress recently, except for the ones that "nickm-cares" about.

Note: See TracTickets for help on using tickets.