wiki:org/teams/NetworkTeam/BandwidthAuthority

Bandwidth Authority

Tasks/Tickets

Initially planned SoP tasks

  • [ ] implement features we wanted in torflow in sbws instead (1 week)
    • [ ] bwauth code needs to be smarter about failed circuits #16559
    • [X] assign the 10-second client timeout as the time for failed circuits? (this was fixed by pastly)
    • [ ] at least publish failure rates #7281
    • [ ] Understand how accurate the bandwidth authority estimates are #7177
  • [ ] integration tests (1 week)
  • [X] a practice transition in the test network (1 week, but split up)
    • [X] set up torflow
    • [X] set up sbws
    • [X] compare and switch
  • [ ] work little-t tor needs (3 weeks)
    • [X] report version of bwscanners in votes #3723
    • [ ] stop relays reporting zero observed bandwidths #24250
    • [ ] relays should regularly do a larger bandwidth self-test #22453
    • [ ] bandwidth testing circuits should use guards sometimes #19009
    • [ ] report bandwidth more often on test networks #17036
    • [X] report bandwidth less often on large, public relays #24104
  • [ ] non-essential tickets like https://github.com/pastly/simple-bw-scanner/issues/77 and https://github.com/pastly/simple-bw-scanner/issues/48 (2 weeks) - [ ] if time and motivation

We have created and worked on other tickets since this list was proposed.

SoP parent Ticket:

Ticket Resolution Summary Component Milestone Modified Owner Reporter Cc Parent ID
#28403 Link to bandwidth files from Consensus Health Metrics/Consensus Health 4 weeks ago tom teor metrics-team #25925
#21377 DirAuths should expose bwauth bandwidth files Core Tor/Tor Tor: 0.4.0.x-final 26 hours ago juga tom starlight@… #25925
#27107 Transition plan from Torflow to sbws Core Tor/sbws sbws: 1.1.x-final 2 weeks ago juga #25925
#28547 Monitor relays that are not measured by each sbws instance Core Tor/sbws sbws: 1.1.x-final 12 days ago teor pastly, juga #25925
#28585 Release 1.0.3 Core Tor/sbws sbws: 1.0.x-final 2 weeks ago juga juga juga #25925
#28602 The call to generate the bw file is passing an old argument Core Tor/sbws sbws: 1.0.x-final 10 days ago juga juga juga #25925
#28639 After several days, most of the circuits timeout Core Tor/sbws sbws: 1.0.x-final 5 hours ago juga juga, teor #25925
#7177 Understand how accurate the bandwidth authority estimates are Core Tor/sbws sbws: unspecified 9 days ago karsten mikeperry, aagbsn, ln5, gamambel, cass, juga #25925
#27047 Authorities should keep recent consensuses, votes, and bandwidth files Core Tor/Tor Tor: unspecified 4 months ago teor irl, metrics-team #25925

sbws tickets in https://github.com/pastly/simple-bw-scanner from 2018/05/14 to 2018/08/31:

sbws PRs/issues from 2018/05/14

sbws component tickets:

Ticket Resolution Summary Component Milestone Modified Owner Reporter Cc Parent ID Modified
#27107 Transition plan from Torflow to sbws Core Tor/sbws sbws: 1.1.x-final 2 weeks ago juga #25925 2 weeks ago
#28099 Make a policy for adding new sbws relay keys Core Tor/sbws sbws: 1.1.x-final 2 weeks ago teor pastly, juga, teor 2 weeks ago
#28158 Consider writing bw lines even if the min percentage has not been reached Core Tor/sbws sbws: 1.1.x-final 2 weeks ago juga pastly, juga, teor #28547 2 weeks ago
#28224 Monitor dirauth running sbws in production Core Tor/sbws sbws: 1.1.x-final 2 weeks ago juga micah #27107 2 weeks ago
#28547 Monitor relays that are not measured by each sbws instance Core Tor/sbws sbws: 1.1.x-final 12 days ago teor pastly, juga #25925 12 days ago
#28563 Work out how sbws can report excluded relays in the bandwidth file Core Tor/sbws sbws: 1.1.x-final 2 weeks ago teor pastly, juga #28547 2 weeks ago
#28565 Report excluded results in a relay's bandwidth line Core Tor/sbws sbws: 1.1.x-final 2 weeks ago teor pastly, juga #28547 2 weeks ago
#28566 Report relays that sbws sees in the consensus, but never chooses to test Core Tor/sbws sbws: 1.1.x-final 2 weeks ago teor pastly, juga #28547 2 weeks ago
#28567 Report relays that sbws wants to test, but the test doesn't work Core Tor/sbws sbws: 1.1.x-final 2 weeks ago teor pastly, juga #28547 2 weeks ago
#28582 Document the load-balancing goal for sbws Core Tor/sbws sbws: 1.1.x-final 2 weeks ago teor teor juga #27107 2 weeks ago
#28589 Open trac tickets for every open sbws issue Core Tor/sbws sbws: 1.1.x-final 2 weeks ago teor juga #27107 2 weeks ago
#28590 Replace sbws tests/integration/net.tar with an unpacked version Core Tor/sbws sbws: 1.1.x-final 2 weeks ago teor juga 2 weeks ago
#28652 When sbws stops making progress, log a warning Core Tor/sbws sbws: 1.1.x-final 13 days ago teor juga, teor #28547 13 days ago
#28737 Redesign sbws torrc option configuration Core Tor/sbws sbws: 1.1.x-final 5 days ago teor juga, teor 5 days ago
#28774 Stop the integration tests http server when the tests end Core Tor/sbws sbws: 1.1.x-final 5 days ago juga juga, teor 5 days ago
#28585 Release 1.0.3 Core Tor/sbws sbws: 1.0.x-final 2 weeks ago juga juga juga #25925 2 weeks ago
#28602 The call to generate the bw file is passing an old argument Core Tor/sbws sbws: 1.0.x-final 10 days ago juga juga juga #25925 10 days ago
#28639 After several days, most of the circuits timeout Core Tor/sbws sbws: 1.0.x-final 5 hours ago juga juga, teor #25925 5 hours ago
#28646 Disable adaptive circuit timeouts Core Tor/sbws sbws: 1.0.x-final 8 days ago teor juga, teor #28692 8 days ago
#28647 Update INSTALL.rst and DEPLOY.rst based on Torflow's documentation Core Tor/sbws sbws: 1.0.x-final 3 days ago teor juga, teor 3 days ago
#28661 Change http links to https in the documentation Core Tor/sbws sbws: 1.0.x-final 9 days ago juga juga juga, teor 9 days ago
#28662 Change old broken links in the documentation Core Tor/sbws sbws: 1.0.x-final 9 days ago juga juga juga, teor 9 days ago
#28663 sbws stops accumulating, silently Core Tor/sbws sbws: 1.0.x-final 27 hours ago stefani #28639 27 hours ago
#28670 CI should check for broken links in the documentation Core Tor/sbws sbws: 1.0.x-final 12 days ago juga juga juga, teor 12 days ago
#28692 sbws should set ConnectionPadding 0 Core Tor/sbws sbws: 1.0.x-final 2 days ago teor juga, teor 2 days ago
#28694 When CircuitPadding is implemented in Tor, set it to 0 in sbws Core Tor/sbws sbws: 1.0.x-final 9 days ago teor nickm, juga, teor #28693 9 days ago
#28701 Try enabling predicted circuits in sbws Core Tor/sbws sbws: 1.0.x-final 5 hours ago teor juga, teor #28639 5 hours ago
#28715 Some torrc options don't have an argument Core Tor/sbws sbws: 1.0.x-final 5 days ago teor juga, teor 5 days ago
#28724 sbws doesn't actually read config files given as an argument Core Tor/sbws sbws: 1.0.x-final 5 days ago pastly juga, teor 5 days ago
#28736 Make info level logs more clear and consistent Core Tor/sbws sbws: 1.0.x-final 3 days ago juga juga, teor #28663 3 days ago
#28738 Stop merging multiple torrc options with the same name Core Tor/sbws sbws: 1.0.x-final 5 days ago teor juga, teor 5 days ago
#28741 sbws should send scanner metadata as part of every HTTP request Core Tor/sbws sbws: 1.0.x-final 2 days ago teor juga, teor 2 days ago
#28761 sbws ignores config option `to_syslog` Core Tor/sbws sbws: 1.0.x-final 3 days ago anadahz juga, teor 3 days ago
#28788 Document configuration verify option and disable urllib3 Core Tor/sbws sbws: 1.0.x-final 4 days ago juga juga, teor 4 days ago
#28789 Stop allowing destination serving files without tls Core Tor/sbws sbws: 1.0.x-final 4 days ago juga juga, teor 4 days ago
#7177 Understand how accurate the bandwidth authority estimates are Core Tor/sbws sbws: unspecified 9 days ago karsten mikeperry, aagbsn, ln5, gamambel, cass, juga #25925 9 days ago
#16559 bwauth code needs to be smarter about failed circuits Core Tor/sbws sbws: unspecified 12 days ago juga TvdW s7r@…, starlight.2015q2@…, juga, teor 12 days ago
#25925 bwauth improvements (ex-parent ticket for SoP planned tasks) Core Tor/sbws sbws: unspecified 8 days ago juga juga 8 days ago
#27343 Dockerfile for sbws basic install Core Tor/sbws sbws: unspecified 12 days ago gabe pastly, juga, teor 12 days ago
#27346 Improve sbws bandwidth accuracy Core Tor/sbws sbws: unspecified 12 days ago teor pastly, juga@…, teor, juga 12 days ago
#27361 Tests that launch sbws in a subprocess Core Tor/sbws sbws: unspecified 12 days ago pastly pastly, juga, teor 12 days ago
#27362 (sub-)packages outside of core (cli) should not need to know about confs and args Core Tor/sbws sbws: unspecified 8 days ago juga juga, teor #28684 8 days ago
#27363 Make the sbws node cap a proportion of the capped bandwidth Core Tor/sbws sbws: unspecified 12 days ago teor pastly, juga, teor 12 days ago
#27689 Round bandwidth in bandwidth files based on proposal 276 Core Tor/sbws sbws: unspecified 12 days ago juga pastly, juga, teor 12 days ago
#27786 sbws: use at least 4 measurements that are at least 6 hours apart Core Tor/sbws sbws: unspecified 12 days ago teor pastly, juga@…, teor, juga #27346 12 days ago
#27787 sbws: use at least 3 days of observed bandwidths Core Tor/sbws sbws: unspecified 12 days ago teor pastly, juga@…, teor, juga #27346 12 days ago
#27788 sbws: weight bandwidths based on the time since the last bandwidth Core Tor/sbws sbws: unspecified 12 days ago teor pastly, juga@…, teor, juga #27346 12 days ago
#27789 sbws: use a decaying average for measured and observed bandwidths Core Tor/sbws sbws: unspecified 12 days ago teor pastly, juga@…, teor, juga #27346 12 days ago
#27790 sbws: design and construct bias curves Core Tor/sbws sbws: unspecified 12 days ago teor pastly, juga@…, teor, juga #27346 12 days ago
#27791 sbws: compare relays against other similar relays Core Tor/sbws sbws: unspecified 12 days ago teor pastly, juga@…, teor, juga #27346 12 days ago
#28045 Start supporting python 3.7 Core Tor/sbws sbws: unspecified 12 days ago juga juga pastly, juga, teor 12 days ago
#28087 Investigate circuit timeout times and if sbws is properly cleaning up circuits when it gives up on them Core Tor/sbws sbws: unspecified 12 days ago pastly pastly, juga, teor 12 days ago
#28106 Change integration tests from bash to shell Core Tor/sbws sbws: unspecified 12 days ago juga pastly, juga, teor, egypcio, gman999@…, catalyst 12 days ago
#28282 Refactor bandwidth file generation code Core Tor/sbws sbws: unspecified 8 days ago juga juga pastly, juga #28684 8 days ago
#28355 Disable measurement timing rules on sbws by default Core Tor/sbws sbws: unspecified 12 days ago juga juga 12 days ago
#28447 improve new SBWS rounding to exhibit uniform percent deltas Core Tor/sbws sbws: unspecified 12 days ago starlight pastly, juga 12 days ago
#28452 3. implement rounding gap smoothing as in proposal 276 Core Tor/sbws sbws: unspecified 12 days ago teor teor pastly, juga, teor #27689 12 days ago
#28463 Measure exits as non-exits with 50% probability Core Tor/sbws sbws: unspecified 12 days ago teor pastly, juga 12 days ago
#28545 Use an 11 second download in sbws Core Tor/sbws sbws: unspecified 12 days ago teor pastly, juga #22453 12 days ago
#28667 Obtain the new descriptors and network status documents by signals instead of every 5min Core Tor/sbws sbws: unspecified 12 days ago juga juga, teor 12 days ago
#28684 Make sbws easy to understand and maintain Core Tor/sbws sbws: unspecified 9 days ago juga juga, teor, irl 9 days ago
#28706 Maybe implement resolving destination domain using Tor's RESOLVE and ADDRMAP events Core Tor/sbws sbws: unspecified 8 days ago juga 8 days ago
#28718 Simplify configuration options Core Tor/sbws sbws: unspecified 8 days ago juga juga, teor #28684 8 days ago
#28758 Add requirements files Core Tor/sbws sbws: unspecified 5 days ago anadahz juga, teor 5 days ago
#28759 Upload sbws to PyPI Core Tor/sbws sbws: unspecified 5 days ago anadahz juga, teor 5 days ago

Tickets kit keyword bwauth or scanner or bandwidth description, not SoP parent not sbws component:

Ticket Resolution Summary Component Milestone Modified Owner Reporter Cc Parent ID Modified
#24045 Measure and map overloaded or over-weighted relays Metrics/Relay Search 8 months ago metrics-team teor gk, brade, mcs, arthuredelstein 8 months ago
#24210 Avoid division by zero and excessive ranks in pytorctl's ScanHandler.rank_to_percent Core Tor/pytorctl 13 months ago teor tom #17064 13 months ago
#24499 Bandwidth determination is flawed Core Tor/Torflow 7 months ago tom Hassprediger juga 7 months ago
#24506 Move some bandwidth authority servers to a CDN Core Tor/Torflow 5 months ago tom teor starlight@…, juga #24674 5 months ago
#24674 Bandwidth authorities should use geographically distributed bandwidth servers Core Tor/Torflow 5 months ago tom teor ln5, juga #24499 5 months ago
#24730 Clarify the bandwidth authority spec to include client and server/service paths Core Tor/Torflow 5 months ago tom teor ln5 #24674 5 months ago
#24834 Map consensus weight vs bandwidth for each bandwidth authority's votes Metrics/Relay Search 5 months ago metrics-team teor ln5, tom, juga #24674 5 months ago
#26804 Create a valid bwfile using a chutney script Core Tor/Chutney 5 months ago teor teor #26803 5 months ago
#28807 Ask authority operators to set `MaxAdvertisedBandwidth 0` in their torrcs Core Tor/DirAuth 48 minutes ago wagon arma 48 minutes ago
#4079 Comb bw auth logs and fix or demote frequent ERRORs and WARNs Core Tor/Torflow 12 months ago mikeperry #4445 12 months ago
#4445 Gather WARN and ERROR log lines from bwauth children in aggregate.py Core Tor/Torflow 12 months ago mikeperry 12 months ago
#16843 Add all bwauth measurements (from votes) Metrics/Onionoo 11 months ago metrics-team cypherpunks tyseom #24834 11 months ago
#21378 Archive bwauth bandwidth files Metrics/CollecTor 3 weeks ago irl tom teor, metrics-team, starlight@… 3 weeks ago
#21377 DirAuths should expose bwauth bandwidth files Core Tor/Tor Tor: 0.4.0.x-final 26 hours ago juga tom starlight@… #25925 26 hours ago
#26478 Unify bandwidth related terms in dir-spec and Tor code. Core Tor/Tor Tor: 0.4.0.x-final 7 days ago juga 7 days ago
#26698 Authorities should put a hash of the bandwidth file in their votes Core Tor/Tor Tor: 0.4.0.x-final 7 days ago teor #27047 7 days ago
#28509 Limit relay bandwidth self-tests based on RelayBandwidthRate, not BandwidthRate Core Tor/Tor Tor: 0.4.0.x-final 3 weeks ago teor #22453 3 weeks ago
#28511 Limit the number of open testing circuits, and the total number of testing circuits Core Tor/Tor Tor: 0.4.0.x-final 3 weeks ago teor #22453 3 weeks ago
#28714 "No circuits are opened" on controllers that DisablePredictedCircuits and build 2-hop circuits Core Tor/Tor Tor: 0.4.0.x-final 8 days ago teor 8 days ago
#28547 Monitor relays that are not measured by each sbws instance Core Tor/sbws sbws: 1.1.x-final 12 days ago teor pastly, juga #25925 12 days ago
#28563 Work out how sbws can report excluded relays in the bandwidth file Core Tor/sbws sbws: 1.1.x-final 2 weeks ago teor pastly, juga #28547 2 weeks ago
#28565 Report excluded results in a relay's bandwidth line Core Tor/sbws sbws: 1.1.x-final 2 weeks ago teor pastly, juga #28547 2 weeks ago
#28566 Report relays that sbws sees in the consensus, but never chooses to test Core Tor/sbws sbws: 1.1.x-final 2 weeks ago teor pastly, juga #28547 2 weeks ago
#28567 Report relays that sbws wants to test, but the test doesn't work Core Tor/sbws sbws: 1.1.x-final 2 weeks ago teor pastly, juga #28547 2 weeks ago
#26694 dir-spec: DirAuths should expose bwauth bandwidth files Core Tor/Tor Tor: 0.3.5.x-final 4 weeks ago juga juga #21377 4 weeks ago
#16559 bwauth code needs to be smarter about failed circuits Core Tor/sbws sbws: unspecified 12 days ago juga TvdW s7r@…, starlight.2015q2@…, juga, teor 12 days ago
#9062 Authorities should describe their bwauth version in their votes Core Tor/Tor Tor: unspecified 8 months ago nickm karsten, pastly 8 months ago
#10922 tor connected to bwauth produces lots of pathbias_count_use_attempt BUG messages Core Tor/Tor Tor: unspecified 18 months ago Sebastian mikeperry 18 months ago
#16696 BWauth no-consensus fallback logic may need revision Core Tor/Tor Tor: unspecified 18 months ago starlight aagbsn, karsten 18 months ago
#19009 bandwidth testing circuits should be allowed to use our guards Core Tor/Tor Tor: unspecified 3 weeks ago juga arma #22453 3 weeks ago
#22453 Relays should regularly do a larger bandwidth self-test Core Tor/Tor Tor: unspecified 3 weeks ago juga arma 3 weeks ago
#24110 document control protocol router status format surprises when using microdescriptors Core Tor/Tor Tor: unspecified 7 months ago teor atagar, catalyst #24094 7 months ago
#25687 over-report of observed / self-measure bandwidth on fast hardware -- important to torflow / peerflow Core Tor/Tor Tor: unspecified 2 months ago starlight juga 2 months ago
#26565 Create unit tests for reading measured bandwidths with non-NULL routerstatuses Core Tor/Tor Tor: unspecified 5 months ago juga teor 5 months ago
#26567 Replace ArgumentCharValue with ValueChar in dir-spec and bandwidth-file-spec Core Tor/Tor Tor: unspecified 6 months ago teor 6 months ago
#26797 DirAuths should only read the V3BandwidthsFile once per vote Core Tor/Tor Tor: unspecified 5 weeks ago teor #27047 5 weeks ago
#26798 Write unit tests for reading measured bandwidths with non-NULL routerststuses Core Tor/Tor Tor: unspecified 5 months ago teor juga 5 months ago
#26803 Add a bwfile test to tor's make test-network-all Core Tor/Tor Tor: unspecified 5 months ago teor 5 months ago
#26805 Work out why Tor networks with all-zero bandwidths are unstable Core Tor/Tor Tor: unspecified 5 months ago teor neel@… #26803 5 months ago
#26904 Work out if we need to round scanner measured bandwidths to protect individual client usage Core Tor/Tor Tor: unspecified 5 weeks ago teor teor 5 weeks ago
#26905 Work out if we need to round observed relay bandwidths to protect individual client usage Core Tor/Tor Tor: unspecified 3 weeks ago teor 3 weeks ago
#27047 Authorities should keep recent consensuses, votes, and bandwidth files Core Tor/Tor Tor: unspecified 4 months ago teor irl, metrics-team #25925 4 months ago
#28510 When relays reset bandwidth tests, the test waits until the next directory document is received Core Tor/Tor Tor: unspecified 3 weeks ago teor #22453 3 weeks ago
#28512 Increase NUM_PARALLEL_TESTING_CIRCS to avoid circuit windows Core Tor/Tor Tor: unspecified 3 weeks ago teor #22453 3 weeks ago
#28514 Open NUM_PARALLEL_TESTING_CIRCS when a bandwidth test is initiated Core Tor/Tor Tor: unspecified 3 weeks ago teor #22453 3 weeks ago
#28815 Refactor similar compression buffer code in dircache.c Core Tor/Tor Tor: unspecified 26 hours ago teor 26 hours ago
#28816 Log a bug for uncompressed data on a compressed connection Core Tor/Tor Tor: unspecified 26 hours ago teor 26 hours ago

{{{#! last two queries do not work with change time, but changetime should be added to see tickets only during sop period ,changetime=May+14%2C+2018..Aug+31%2C+2018 }}}

PF

confirmed:

  • [ ] compared total consensus weights across bandwidth authorities (#25459)
  • [ ] understand how accurate bandwidth authority measurements are (#7177)
  • [ ] find over-weighted relays (#24045, #10791)
  • [ ] map consensus weight vs bandwidth in votes (#24834)
  • [ ] provide bwauth votes via the directory protocol (#21377)
  • [ ] make authorities without bandwidth scanners vote flags based on measured bandwidths (#11327)
  • [ ] reset uptimes / bandwidths when a relay changes IP address (#22308 / sbws #154)
  • [ ] analyse and fix a bug in authority time known (#19162)
  • [ ] some bandwidth weight cases not matching the spec (#20284)
  • [ ] fix DirAuth weights for other relays, or stop them using other relays (#13297)

Related pages

Last modified 5 weeks ago Last modified on Nov 8, 2018, 3:10:36 PM