Dir auths should choose Fast and Guard flags by consensus weight if they don't measure

In #8435 (moved) we made directory-authorities-that-run-bwauths stop voting Fast or Guard for relays they hadn't measured yet.

But as I pointed out in https://trac.torproject.org/projects/tor/ticket/8435#comment:13, since only a minority of dir auths run bwauths, the majority of dir auths are still voting Fast and Guard based on descriptor bandwidths.

So while the title of ticket #8435 (moved) says "Ignore advertised bandwidths for flags once we have enough measured bandwidths", the ChangeLog entry is more accurate:

    - Directory authorities that have more than a threshold number
      of relays with measured bandwidths now treat relays with unmeasured
      bandwidths as having bandwidth 0. Resolves ticket 8435.

We should at some point actually do the original goal, which is to give Fast to the 7/8s of relays whose consensus weights are highest, and Guard to the 1/2 of relays whose consensus weights are highest and who match the other guard constraints.

changed milestone to %Tor: unspecified

added component::core tor/tor milestone::Tor: unspecified needs-proposal owner::TvdW points::4 priority::high severity::normal sponsor::U-can status::needs-revision tor-dirauth type::defect version::tor 0.2.7 labels

One suggestion is to look at the previous consensus, and use the number out of it, if we don't have a measurement of our own.

There are two versions of that idea: one is that we use the consensus weight iff we don't run a bwauth, and the other is that we use the consensus weight if we don't run a bwauth or if we do but haven't measured that relay yet. I'm ok with either, but the former sounds a bit cleaner.

Trac:
Keywords: tor-auth deleted, tor-auth 026-triaged-1 added

Trac:
Milestone: Tor: 0.2.6.x-final to Tor: 0.2.7.x-final

Trac:
Status: new to assigned

Marking some tickets as triaged-in for 0.2.7 based on early triage

Trac:
Keywords: tor-auth 026-triaged-1 deleted, 026-triaged-1, tor-auth, 027-triaged-1-in added

Trac:
Version: N/A to Tor: 0.2.7
Points: N/A to small/medium
Priority: normal to major
Keywords: N/A deleted, SponsorU added

I just closed #12877 (moved) as an overlap with this one.

Trac:
Cc: N/A to aagbsn

Trac:
Milestone: Tor: 0.2.7.x-final to Tor: 0.2.8.x-final

Bulk-replace SponsorU keyword with SponsorU field.

Trac:
Keywords: SponsorU deleted, N/A added
Sponsor: N/A to SponsorU

https://github.com/TvdW/tor/commits/previous-consensus-for-fast-and-guard

Trac:
Status: assigned to needs_review

I just pushed some more commits that refactor that function and comment it better. They're in my 'ticket11327' branch.

(I think they are on top of tvdw's latest branch, but somebody should check it to be sure.)

And now that I've refactored it, a question becomes clearer:

       if (node->rs && node->rs->bandwidth_kb) {
+        /* Case 3: Not running a bwauth, and we know a weight from the
+         * consensus. Use that one (so e.g. we won't vote Fast on a relay
+         * that the consensus has given very low weight to). */
         bw_kb = node->rs->bandwidth_kb;
       }

Is the 'if' part the right way to check that we have a consensus? It seems like "we have a consensus but it lists this relay's weight as 0" will be mistaken as "no we don't have a consensus". Nick did the ri, rs, etc refactorings long ago, so he is the perfect person to answer/improve this part.

And then once we're set there, I could imagine another refactoring where we take out the bw_kb variable and just do a 'return' for each case.

Trac:
Status: needs_review to needs_revision
Severity: N/A to Normal

setting TvdW as the owner of this needs_revision ticket.

Trac:
Status: needs_revision to assigned
Owner: N/A to TvdW

Trac:
Status: assigned to needs_revision

A while ago someone (I think arma/nickm) mentioned we should look into whether we can simply disable Fast/Guard voting if an authority doesn't have enough measurements yet.

After implementing this thought and testing it it turns out this may not be a good idea: the network would lose all fast/guard flags if there are insufficient bandwidth authorities. Also, Chutney would stop working.

I think this is better handled on a consensus method level, or we could choose to get rid of the Fast flag completely and have the client determine it instead.

In any case, I think having the authorities take data from a previous consensus is not the correct solution to this problem.

These seem like features, or like other stuff unlikely to be possible this month. Bumping them to 0.2.9

Trac:
Milestone: Tor: 0.2.8.x-final to Tor: 0.2.9.x-final

Trac:
Sponsor: SponsorU to SponsorU-can

Quick update from IRC: we talked about alternatives and one thing that came up is to have the consensus generation process take care of this instead, as only there do we have enough information to make the right call about whether to take from all bandwidth authorities or only the ones with bandwidth scanners.

This would effectively introduce first and second class dirauths: first class dirauths vote for these flags, and the second class dirauths' opinion only counts if there aren't enough voters around.

Trac:
Reviewer: N/A to N/A

Trac:
Keywords: N/A deleted, tor-guards-revamp added

Trac:
Points: small/medium to 2

Trac:
Cc: aagbsn to aagbsn, asn

Deferring many tickets that are in needs_revision with no progress noted for a while, where I think they could safely wait till 0.3.0 or later.

Please feel free to move these back to 0.2.9 if you finish the revisions soon.

Trac:
Keywords: N/A deleted, nickm-deferred-20160905 added
Milestone: Tor: 0.2.9.x-final to Tor: 0.2.???

Milestone renamed

Trac:
Milestone: Tor: 0.2.??? to Tor: 0.3.???

Finally admitting that 0.3.??? was a euphemism for Tor: unspecified all along.

Trac:
Keywords: N/A deleted, tor-03-unspecified-201612 added
Milestone: Tor: 0.3.??? to Tor: unspecified

Remove an old triaging keyword.

Trac:
Keywords: tor-03-unspecified-201612 deleted, N/A added

Trac:
Keywords: N/A deleted, 027-triaged-in added

Trac:
Keywords: 027-triaged-in deleted, N/A added

Trac:
Keywords: 027-triaged-1-in deleted, N/A added

Trac:
Keywords: 026-triaged-1 deleted, N/A added

Trac:
Keywords: nickm-deferred-20160905 deleted, N/A added

Turns out that tor-auth is for directory authority so make it clearer with tor-dirauth

Trac:
Keywords: tor-auth deleted, tor-dirauth added

Trac:
Points: 2 to 4
Keywords: tor-dirauth, tor-guards-revamp deleted, needs-proposal tor-dirauth added

See also #8435 (moved) , where some progress was made.

changed time estimate to 32h

mentioned in issue #12877 (moved)

mentioned in issue #24046 (moved)

mentioned in issue #30164 (moved)

moved to tpo/core/tor#11327 (moved)