Opened 8 years ago

Closed 3 years ago

#2714 closed task (fixed)

Does using rephist uptime for calculating HSDir produce too much flapping?

Reported by: arma Owned by:
Priority: Medium Milestone:
Component: Metrics/Analysis Version:
Severity: Normal Keywords: SponsorR, tor-auth, tor-hs
Cc: karsten, dgoulet Actual Points:
Parent ID: #13209 Points:
Reviewer: Sponsor:

Description

This ticket came out of the solution for #2709.

One of our original goals with the "uptime > 24 hours" constraint for hsdir flag assignment was to make sure that the hsdir flag assignment was quite robust over time: a hidden service needs to store its descriptor in the place that a client will fetch it hours later.

Uptime is quite monotonic -- for a relay that stays up. How monotonic is the rephist-calculated uptime?

Child Tickets

Attachments (3)

hsdir-set-instability-2011-partial.gz (27.1 KB) - added by rransom 8 years ago.
hsdir-set-instability intermediate output file for metrics-tasks.git/task-2649
hsdir-set-instability.gz (192.3 KB) - added by rransom 8 years ago.
hsdir-set-instability intermediate output file for metrics-tasks.git/task-2649 (full)
hsdir-set-absolute-instability-graph.pdf (172.3 KB) - added by rransom 8 years ago.
graph of absolute (unscaled) instability of the HSDir set

Download all attachments as: .zip

Change History (17)

comment:1 Changed 8 years ago by Sebastian

A relay that fails to be available to a majority of dirauths counts as a relay that is not up, even if the relay itself believes otherwise IMO.

comment:2 in reply to:  1 Changed 8 years ago by arma

Replying to Sebastian:

A relay that fails to be available to a majority of dirauths counts as a relay that is not up, even if the relay itself believes otherwise IMO.

But if it's away for a short time and then back, it can still be useful at serving the hidden service descriptors that had been assigned to it.

I guess one of the next questions to ask is "what time periods are we talking about here?"

That is, how long does a relay have to be up to be a suitable hidden service directory point?

comment:3 Changed 8 years ago by nickm

Milestone: Tor: 0.2.3.x-final

Changed 8 years ago by rransom

hsdir-set-instability intermediate output file for metrics-tasks.git/task-2649

comment:4 Changed 8 years ago by rransom

I've attached the 2011 portion of the hsdir-set-instability intermediate output file from my task-2649 analysis to this ticket so other people can view the graph interactively with gnuplot more easily.

Based on a zoomed-in view of 2011-04-25 through 2011-05-21 (when I downloaded the consensus tarball for May), instability in the HSDir set (as a fraction of the size of the set) fluctuates on a 24-hour cycle, and that cycle's amplitude increased significantly around 2011-04-28.

comment:5 Changed 8 years ago by rransom

To view the HSDir set instability graph:

  • Uncompress the file attached to this ticket and copy it to task-2649/out/hsdir-set-instability.
  • Start gnuplot in the task-2649 directory.
  • Run load "graph-hsdir-set-instability.gnuplot", then set format x "%Y-%m-%d-%H", then replot.
  • Zoom in on a portion of the graph by right-clicking on opposite corners of the area you want to view.

Changed 8 years ago by rransom

Attachment: hsdir-set-instability.gz added

hsdir-set-instability intermediate output file for metrics-tasks.git/task-2649 (full)

Changed 8 years ago by rransom

graph of absolute (unscaled) instability of the HSDir set

comment:6 Changed 8 years ago by rransom

See hsdir-set-absolute-instability-graph.pdf for a graph of the unscaled instability of the HSDir set; see task-2714 ( git://git.torproject.org/rransom/metrics-tasks.git task-2714 ) for sources.

comment:7 Changed 7 years ago by nickm

Milestone: Tor: 0.2.3.x-finalTor: 0.2.4.x-final

Did what we found out about this let us know anything about why this is happening and what we could do about it?

comment:8 Changed 7 years ago by nickm

Keywords: tor-auth added

comment:9 Changed 7 years ago by nickm

Component: Tor Directory AuthorityTor

comment:10 Changed 6 years ago by nickm

Component: TorAnalysis
Milestone: Tor: 0.2.4.x-final

I'm kicking this out of the Tor component until it represents a Tor task. Please toss it back in if somebody gets a Tor task out of this.

comment:11 Changed 5 years ago by arma

Parent ID: #13209

comment:12 Changed 4 years ago by arma

Cc: dgoulet added

David: you've been gaining some intuition about this topic lately from the hs health measurer. Did you get any closer to answering this ticket? Are there specific queries we could make to the consensus documents to answer it?

comment:13 Changed 4 years ago by arma

Keywords: SponsorR added

comment:14 Changed 3 years ago by dgoulet

Keywords: tor-hs added
Resolution: fixed
Severity: Normal
Status: newclosed

Some basic conclusion out of #13209 experiment. A client getting the latest consensus at every hour, the churn rate was less than 3% and that was only one single HSDir would change out of the set of 6 in that hour. But it never happened over almost a year of running the HS health that a service descriptor was not found in the set.

However, this is not really representing reality as clients have different consensuses and not _only_ the latest one... See #13209 for more information on the issues of this experiment.

Sooooo we can keep that thing opened forever or flag it that "OK, HSDir so far aren't moving around much as we observed and we are happy with this". We should try to measure this a bit better with a "v2.0" of #13209 so we can overtime measure it and act accordingly if shit hit the fan. We already have some good ticket going on about improving stability of HSDir flag and raising the bar as well so those relays stop snooping on .onion (#19162).

I'm closing this ticket and we'll re-open a new one if we think that it's becoming an issue at some point.

Note: See TracTickets for help on using tickets.