Quick analysis of #1944 experiment to answer some of dgoulet's questions on #13509

Line
1Here's a very quick analysis from the #1944 experiment.  Raw logs are available here:
2
3https://people.torproject.org/~karsten/torperf-experiment-2014-10-21.tar.xz (11M)
4
5https://people.torproject.org/~karsten/torperf-info-logs-2014-11-05.tar.xz (14M)
6
7>  * How many HS desc fetch failures do we get everytime we connect to an HS? (and that's #13208)
8
9{{{
10\$ grep -h ^HS_DESC events-*.log | cut -d" " -f2 | sort | uniq -c
11 129 FAILED
139940 REQUESTED
14}}}
15
16That's 1.3% descriptor fetch failures.
17
18>  * How many IPs do we try before succeeding at an introduction?
19
20{{{
21\$ grep HSCI_ events-50kb.log | cut -d" " -f2 | sort | uniq | wc -l
22    9250
23\$ grep "STREAM .* NEW .*onion" events-50kb.log | wc -l
24    7838
25}}}
26
27That's 9250 introduction circuits for 7838 requests, so 1.18 per request.
28
29>  * How many RPs do we try before suceeding at connecting?
30
31{{{
32\$ grep HSCR_ events-50kb.log | cut -d" " -f2 | sort | uniq | wc -l
33    8715
34}}}
35
36That's 8715 rendezvous circuits for 7838 requests, or 1.11 per request.
37
38>  * What fraction of HS fetches here succeed?
39
40I'm not sure what you mean.
41
42>  * What fraction of INTRODUCE1 attempts connect to a real INTRO circuit?
43
44Again, not sure what you mean.
45
46>  * How often do we change our IPs from service-side? How long do we keep a typical IP?
47
48{{{
49\$ grep HSSI_ events-hs.log | cut -d" " -f2 | sort | uniq | wc -l
50     472
51\$ head -n1 events-hs.log | sed 's/.*ARRIVED_AT=\(.*\)/\1/'
522014-09-24T10:45:04.929867
53\$ tail -n1 events-hs.log | sed 's/.*ARRIVED_AT=\(.*\)/\1/'
542014-10-21T16:23:18.969814
55}}}
56
57That's 472 server-side introduction points in 27 days and a few hours, or 17.5 times per day.
58
59>  * What fraction of INTRODUCE2 cells do we decline to answer?
60
61The event logs don't contain anything related, AFAIK, but the info-level logs contain some interesting lines:
62
63{{{
64Sep 29 02:32:36.000 [info] rend_service_introduce(): We received an INTRODUCE2 cell with same first part of Diffie-Hellman handshake 26 seconds ago. Dropping cell.
65[...]
66Oct 13 23:35:04.000 [warn] rend_service_introduce(): Bug: Internal error: Got an INTRODUCE2 cell on an intro circ (for service "kmltuigipr23dk3g") with no corresponding rend_intro_point_t.
67Oct 13 23:35:07.000 [warn] rend_service_introduce(): Bug: Internal error: Got an INTRODUCE2 cell on an intro circ (for service "kmltuigipr23dk3g") with no corresponding rend_intro_point_t.
68Oct 13 23:35:09.000 [warn] rend_service_introduce(): Bug: Internal error: Got an INTRODUCE2 cell on an intro circ (for service "kmltuigipr23dk3g") with no corresponding rend_intro_point_t.
69Oct 13 23:35:12.000 [warn] rend_service_introduce(): Bug: Internal error: Got an INTRODUCE2 cell on an intro circ (for service "kmltuigipr23dk3g") with no corresponding rend_intro_point_t.
70Oct 13 23:35:14.000 [warn] rend_service_introduce(): Bug: Internal error: Got an INTRODUCE2 cell on an intro circ (for service "kmltuigipr23dk3g") with no corresponding rend_intro_point_t.
71Oct 13 23:35:19.000 [warn] rend_service_introduce(): Bug: Internal error: Got an INTRODUCE2 cell on an intro circ (for service "kmltuigipr23dk3g") with no corresponding rend_intro_point_t.
72}}}
73
74The latter 6 look very related, so that's 2 incidents in 7838 requests.
75
76>  * How often do we re-upload?
77
78{{{
79\$ grep "Launching upload for v2 descriptor" info.log | wc -l
807084
81}}}
82
83That's 7084 uploads in 27 days and a few hours, so around 11 per ''hour''.  Feels like something's broken.
84
85>  * How often do we use preemptive circuits?
86>  * How often do we cannibalize circuits?
87
88Is the question: how often do we cannibalize preemptively built circuits?  Or do you mean something else?
89
90{{{
91\$ grep ^CIRC events-50kb.log | grep CANNIBALIZED | wc -l
92   18598
93}}}
94
95That's 18598 cannibalized circuits for 7838 requests, so 2.4 per request.  Which makes sense: first to fetch the descriptor, second for the rendezvous circuit, third, if available, for the introduction circuit.