wiki:org/teams/NetworkTeam/BandwidthAuthority

Bandwidth Authority

Tasks/Tickets

Initially planned SoP tasks

  • [ ] implement features we wanted in torflow in sbws instead (1 week)
    • [ ] bwauth code needs to be smarter about failed circuits #16559
    • [X] assign the 10-second client timeout as the time for failed circuits? (this was fixed by pastly)
    • [ ] at least publish failure rates #7281
    • [ ] Understand how accurate the bandwidth authority estimates are #7177
  • [ ] integration tests (1 week)
  • [X] a practice transition in the test network (1 week, but split up)
    • [X] set up torflow
    • [X] set up sbws
    • [X] compare and switch
  • [ ] work little-t tor needs (3 weeks)
    • [X] report version of bwscanners in votes #3723
    • [ ] stop relays reporting zero observed bandwidths #24250
    • [ ] relays should regularly do a larger bandwidth self-test #22453
    • [ ] bandwidth testing circuits should use guards sometimes #19009
    • [ ] report bandwidth more often on test networks #17036
    • [X] report bandwidth less often on large, public relays #24104
  • [ ] non-essential tickets like https://github.com/pastly/simple-bw-scanner/issues/77 and https://github.com/pastly/simple-bw-scanner/issues/48 (2 weeks) - [ ] if time and motivation

We have created and worked on other tickets since this list was proposed.

SoP parent Ticket:

Ticket Resolution Summary Component Milestone Modified Owner Reporter Cc Parent ID
#28403 Link to bandwidth files from Consensus Health Metrics/Consensus Health 3 months ago tom teor metrics-team #25925
#21377 DirAuths should expose bwauth bandwidth files Core Tor/Tor Tor: 0.4.1.x-final 17 hours ago juga tom starlight@… #25925
#27107 Transition plan from Torflow to sbws Core Tor/sbws sbws: 1.1.x-final 3 months ago juga #25925
#28547 Monitor relays that are not measured by each sbws instance Core Tor/sbws sbws: 1.1.x-final 3 months ago teor pastly, juga #25925
#28585 Release 1.0.3 Core Tor/sbws sbws: 1.0.x-final 3 months ago juga juga juga #25925
#28639 After several days, most of the circuits timeout Core Tor/sbws sbws: 1.0.x-final 2 months ago juga juga, teor #25925
#7177 Understand how accurate the bandwidth authority estimates are Core Tor/sbws sbws: unspecified 3 months ago karsten mikeperry, aagbsn, ln5, gamambel, cass, juga #25925
#27047 Authorities should keep recent consensuses, votes, and bandwidth files Core Tor/Tor Tor: unspecified 7 months ago teor irl, metrics-team #25925

sbws tickets in https://github.com/pastly/simple-bw-scanner from 2018/05/14 to 2018/08/31:

sbws PRs/issues from 2018/05/14

sbws component tickets:

Ticket Resolution Summary Component Milestone Modified Owner Reporter Cc Parent ID Modified
#27107 Transition plan from Torflow to sbws Core Tor/sbws sbws: 1.1.x-final 3 months ago juga #25925 3 months ago
#28099 Make a policy for adding new sbws relay keys Core Tor/sbws sbws: 1.1.x-final 3 months ago teor pastly, juga, teor 3 months ago
#28158 Consider writing bw lines even if the min percentage has not been reached Core Tor/sbws sbws: 1.1.x-final 3 months ago juga pastly, juga, teor #28547 3 months ago
#28224 Monitor dirauth running sbws in production Core Tor/sbws sbws: 1.1.x-final 3 months ago juga micah #27107 3 months ago
#28355 Disable measurement timing rules on sbws by default Core Tor/sbws sbws: 1.1.x-final 11 days ago juga juga 11 days ago
#28547 Monitor relays that are not measured by each sbws instance Core Tor/sbws sbws: 1.1.x-final 3 months ago teor pastly, juga #25925 3 months ago
#28563 Work out how sbws can report excluded relays in the bandwidth file Core Tor/sbws sbws: 1.1.x-final 5 weeks ago teor pastly, juga #28547 5 weeks ago
#28565 Report excluded results in a relay's bandwidth line Core Tor/sbws sbws: 1.1.x-final 2 weeks ago juga teor juga #28547 2 weeks ago
#28566 Report relays that sbws sees in the consensus, but never chooses to test Core Tor/sbws sbws: 1.1.x-final 4 days ago juga teor #28547 4 days ago
#28567 Report relays that sbws wants to test, but the test doesn't work Core Tor/sbws sbws: 1.1.x-final 4 days ago juga teor #28547 4 days ago
#28582 Document the load-balancing goal for sbws Core Tor/sbws sbws: 1.1.x-final 3 months ago teor teor juga #27107 3 months ago
#28589 Open trac tickets for every open sbws GitHub issue Core Tor/sbws sbws: 1.1.x-final 6 weeks ago teor juga #27107 6 weeks ago
#28652 When sbws stops making progress, log a warning Core Tor/sbws sbws: 1.1.x-final 6 weeks ago teor juga, teor #28547 6 weeks ago
#28737 Redesign sbws torrc option configuration Core Tor/sbws sbws: 1.1.x-final 3 months ago teor juga, teor 3 months ago
#28774 Stop the integration tests http server when the tests end Core Tor/sbws sbws: 1.1.x-final 6 weeks ago juga juga juga, teor 6 weeks ago
#28983 Work out how long it takes sbws to measure the network Core Tor/sbws sbws: 1.1.x-final 7 weeks ago teor juga 7 weeks ago
#28985 Does sbws need Tor to report observed bandwidths more often? Core Tor/sbws sbws: 1.1.x-final 7 weeks ago teor juga 7 weeks ago
#29046 Remove unused testnets Core Tor/sbws sbws: 1.1.x-final 6 weeks ago juga juga juga #28684 6 weeks ago
#29047 Improve code style following PEP8 and PEP257 Core Tor/sbws sbws: 1.1.x-final 6 weeks ago juga juga juga #28684 6 weeks ago
#29048 Remove unused code Core Tor/sbws sbws: 1.1.x-final 6 weeks ago juga juga juga #28684 6 weeks ago
#29057 Adapt bandwidth file classes to be compatible with stem (descriptors, etc) documents Core Tor/sbws sbws: 1.1.x-final 6 weeks ago juga juga juga #28684 6 weeks ago
#29291 Work out a better way to choose the data size to download Core Tor/sbws sbws: 1.1.x-final 2 weeks ago juga karsten, irl 2 weeks ago
#29292 Document number of threads configuration depending on the machine available bandwidth Core Tor/sbws sbws: 1.1.x-final 10 days ago juga juga 10 days ago
#29299 Include scanner country and Web server country in the bandwidth file header Core Tor/sbws sbws: 1.1.x-final 11 days ago juga juga juga 11 days ago
#29355 Include scanner nickname and UUID in the bandwidth file headers? Core Tor/sbws sbws: 1.1.x-final 2 weeks ago juga juga 2 weeks ago
#29358 Stop overloading the CPU when the test network is run in the integration tests Core Tor/sbws sbws: 1.1.x-final 2 weeks ago juga juga 2 weeks ago
#28585 Release 1.0.3 Core Tor/sbws sbws: 1.0.x-final 3 months ago juga juga juga #25925 3 months ago
#28639 After several days, most of the circuits timeout Core Tor/sbws sbws: 1.0.x-final 2 months ago juga juga, teor #25925 2 months ago
#28646 Disable adaptive circuit timeouts Core Tor/sbws sbws: 1.0.x-final 3 months ago teor juga, teor #28692 3 months ago
#28663 sbws stops accumulating, silently Core Tor/sbws sbws: 1.0.x-final 2 months ago juga stefani #28639 2 months ago
#28692 sbws should set ConnectionPadding 0 Core Tor/sbws sbws: 1.0.x-final 2 months ago teor juga, teor 2 months ago
#28694 When CircuitPadding is implemented in Tor, set it to 0 in sbws Core Tor/sbws sbws: 1.0.x-final 4 weeks ago teor nickm, juga, teor #28693 4 weeks ago
#28701 Try enabling predicted circuits in sbws Core Tor/sbws sbws: 1.0.x-final 2 months ago teor juga, teor #28639 2 months ago
#28864 sbws AsyncResults have no timeout Core Tor/sbws sbws: 1.0.x-final 2 months ago teor juga, teor #28663 2 months ago
#28865 sbws keeps the number of AsyncResults less than the number of threads Core Tor/sbws sbws: 1.0.x-final 2 months ago teor juga, teor #28663 2 months ago
#28866 ResultDump.queue.put() can hang if the queue is full Core Tor/sbws sbws: 1.0.x-final 6 days ago juga teor juga, teor #28663 6 days ago
#28897 Stop running twice destination usability tests Core Tor/sbws sbws: 1.0.x-final 10 days ago juga juga juga, teor #28663 10 days ago
#28933 Run one sbws main loop using the test network as part of integration tests Core Tor/sbws sbws: 1.0.x-final 5 weeks ago juga juga juga, teor #28663 5 weeks ago
#29149 Update/improve documentation on how the scanner/generator work Core Tor/sbws sbws: 1.0.x-final 4 days ago juga juga 4 days ago
#29156 Measure first relays with the lowest number of attempts first. Core Tor/sbws sbws: 1.0.x-final 7 days ago juga juga, teor #28663 7 days ago
#29157 Modify the the threads callback error to store the error Core Tor/sbws sbws: 1.0.x-final 7 days ago juga juga, teor #28663 7 days ago
#29290 Help 2-3 dirauths to deploy sbws Core Tor/sbws sbws: 1.0.x-final 3 weeks ago juga juga 3 weeks ago
#29294 Create an script to automate releases Core Tor/sbws sbws: 1.0.x-final 11 days ago juga juga juga 11 days ago
#29551 Use timeout from the requests' session attribute in the methods Core Tor/sbws sbws: 1.0.x-final 17 hours ago juga juga juga 17 hours ago
#7177 Understand how accurate the bandwidth authority estimates are Core Tor/sbws sbws: unspecified 3 months ago karsten mikeperry, aagbsn, ln5, gamambel, cass, juga #25925 3 months ago
#16559 bwauth code needs to be smarter about failed circuits Core Tor/sbws sbws: unspecified 3 months ago juga TvdW s7r@…, starlight.2015q2@…, juga, teor 3 months ago
#25925 bwauth improvements (ex-parent ticket for SoP planned tasks) Core Tor/sbws sbws: unspecified 3 months ago juga juga 3 months ago
#27343 Dockerfile for sbws basic install Core Tor/sbws sbws: unspecified 3 months ago gabe pastly, juga, teor 3 months ago
#27346 Improve sbws bandwidth accuracy Core Tor/sbws sbws: unspecified 3 months ago teor pastly, juga@…, teor, juga 3 months ago
#27361 Tests that launch sbws in a subprocess Core Tor/sbws sbws: unspecified 3 months ago pastly pastly, juga, teor 3 months ago
#27362 (sub-)packages outside of core (cli) should not need to know about confs and args Core Tor/sbws sbws: unspecified 3 months ago juga juga, teor #28684 3 months ago
#27363 Make the sbws node cap a proportion of the capped bandwidth Core Tor/sbws sbws: unspecified 3 months ago teor pastly, juga, teor 3 months ago
#27689 Round bandwidth in bandwidth files based on proposal 276 Core Tor/sbws sbws: unspecified 3 months ago juga pastly, juga, teor 3 months ago
#27786 sbws: use at least 4 measurements that are at least 6 hours apart Core Tor/sbws sbws: unspecified 3 months ago teor pastly, juga@…, teor, juga #27346 3 months ago
#27787 sbws: use at least 3 days of observed bandwidths Core Tor/sbws sbws: unspecified 3 months ago teor pastly, juga@…, teor, juga #27346 3 months ago
#27788 sbws: weight bandwidths based on the time since the last bandwidth Core Tor/sbws sbws: unspecified 3 months ago teor pastly, juga@…, teor, juga #27346 3 months ago
#27789 sbws: use a decaying average for measured and observed bandwidths Core Tor/sbws sbws: unspecified 3 months ago teor pastly, juga@…, teor, juga #27346 3 months ago
#27790 sbws: design and construct bias curves Core Tor/sbws sbws: unspecified 3 months ago teor pastly, juga@…, teor, juga #27346 3 months ago
#27791 sbws: compare relays against other similar relays Core Tor/sbws sbws: unspecified 3 months ago teor pastly, juga@…, teor, juga #27346 3 months ago
#28045 Start supporting python 3.7 Core Tor/sbws sbws: unspecified 3 months ago juga juga pastly, juga, teor 3 months ago
#28087 Investigate circuit timeout times and if sbws is properly cleaning up circuits when it gives up on them Core Tor/sbws sbws: unspecified 3 months ago pastly pastly, juga, teor 3 months ago
#28106 Change integration tests from bash to shell Core Tor/sbws sbws: unspecified 3 months ago juga pastly, juga, teor, egypcio, gman999@…, catalyst 3 months ago
#28282 Refactor bandwidth file generation code Core Tor/sbws sbws: unspecified 3 months ago juga juga pastly, juga #28684 3 months ago
#28447 improve new SBWS rounding to exhibit uniform percent deltas Core Tor/sbws sbws: unspecified 3 months ago starlight pastly, juga 3 months ago
#28452 3. implement rounding gap smoothing as in proposal 276 Core Tor/sbws sbws: unspecified 3 months ago teor teor pastly, juga, teor #27689 3 months ago
#28463 Measure exits as non-exits with 50% probability Core Tor/sbws sbws: unspecified 3 months ago teor pastly, juga 3 months ago
#28545 Use an 11 second download in sbws Core Tor/sbws sbws: unspecified 3 months ago teor pastly, juga #22453 3 months ago
#28667 Obtain the new descriptors and network status documents by signals instead of every 5min Core Tor/sbws sbws: unspecified 3 months ago juga juga, teor 3 months ago
#28684 Make sbws easy to understand and maintain Core Tor/sbws sbws: unspecified 6 weeks ago juga juga, teor, irl 6 weeks ago
#28706 Maybe implement resolving destination domain using Tor's RESOLVE and ADDRMAP events Core Tor/sbws sbws: unspecified 3 months ago juga 3 months ago
#28718 Simplify configuration options Core Tor/sbws sbws: unspecified 3 months ago juga juga, teor #28684 3 months ago
#28758 Add requirements files Core Tor/sbws sbws: unspecified 3 months ago anadahz juga, teor 3 months ago
#28759 Upload sbws to PyPI Core Tor/sbws sbws: unspecified 3 months ago anadahz juga, teor 3 months ago
#28990 Move all or part of the documentation about Tor and the bandwidth scanner Core Tor/sbws sbws: unspecified 7 weeks ago juga juga 7 weeks ago
#29033 Some tor configuration options are sensitive to the order Core Tor/sbws sbws: unspecified 6 weeks ago juga juga 6 weeks ago

Tickets kit keyword bwauth or scanner or bandwidth description, not SoP parent not sbws component:

Ticket Resolution Summary Component Milestone Modified Owner Reporter Cc Parent ID Modified
#24045 Measure and map overloaded or over-weighted relays Metrics/Relay Search 11 months ago metrics-team teor gk, brade, mcs, arthuredelstein 11 months ago
#24210 Avoid division by zero and excessive ranks in pytorctl's ScanHandler.rank_to_percent Core Tor/pytorctl 16 months ago teor tom #17064 16 months ago
#24499 Bandwidth determination is flawed Core Tor/Torflow 9 months ago tom Hassprediger juga 9 months ago
#24506 Move some bandwidth authority servers to a CDN Core Tor/Torflow 8 months ago tom teor starlight@…, juga #24674 8 months ago
#24674 Bandwidth authorities should use geographically distributed bandwidth servers Core Tor/Torflow 8 months ago tom teor ln5, juga #24499 8 months ago
#24730 Clarify the bandwidth authority spec to include client and server/service paths Core Tor/Torflow 8 months ago tom teor ln5 #24674 8 months ago
#24834 Map consensus weight vs bandwidth for each bandwidth authority's votes Metrics/Relay Search 8 months ago metrics-team teor ln5, tom, juga #24674 8 months ago
#26804 Create a valid bwfile using a chutney script Core Tor/Chutney 7 months ago teor teor #26803 7 months ago
#26902 Download and parse bwauth files Core Tor/Stem 2 weeks ago atagar atagar juga 2 weeks ago
#28807 Ask authority operators to set `MaxAdvertisedBandwidth 0` in their torrcs Core Tor/DirAuth 2 months ago wagon arma 2 months ago
#29539 version key should be in the second possition in the Bandwidth File Core Tor/Stem 2 days ago atagar juga 2 days ago
#4079 Comb bw auth logs and fix or demote frequent ERRORs and WARNs Core Tor/Torflow 15 months ago mikeperry #4445 15 months ago
#4445 Gather WARN and ERROR log lines from bwauth children in aggregate.py Core Tor/Torflow 15 months ago mikeperry 15 months ago
#16843 Add all bwauth measurements (from votes) Metrics/Onionoo 14 months ago metrics-team cypherpunks tyseom #24834 14 months ago
#21378 Archive bwauth bandwidth files Metrics/CollecTor 11 days ago irl tom teor, metrics-team, starlight@… 11 days ago
#21377 DirAuths should expose bwauth bandwidth files Core Tor/Tor Tor: 0.4.1.x-final 17 hours ago juga tom starlight@… #25925 17 hours ago
#26694 dir-spec: DirAuths should expose bwauth bandwidth files Core Tor/Tor Tor: 0.4.1.x-final 3 weeks ago juga juga #21377 3 weeks ago
#29132 Clean up dirserv_read_measured_bandwidths after #26223 Core Tor/Tor Tor: 0.4.1.x-final 8 days ago teor juga 8 days ago
#29137 Standardize 'Bandwidth File' name Core Tor/Tor Tor: 0.4.1.x-final 5 weeks ago atagar 5 weeks ago
#29354 Update bandwidth-file-spec.txt with the country keyword Core Tor/Tor Tor: 0.4.0.x-final 39 hours ago juga juga #29299 39 hours ago
#28547 Monitor relays that are not measured by each sbws instance Core Tor/sbws sbws: 1.1.x-final 3 months ago teor pastly, juga #25925 3 months ago
#28563 Work out how sbws can report excluded relays in the bandwidth file Core Tor/sbws sbws: 1.1.x-final 5 weeks ago teor pastly, juga #28547 5 weeks ago
#28565 Report excluded results in a relay's bandwidth line Core Tor/sbws sbws: 1.1.x-final 2 weeks ago juga teor juga #28547 2 weeks ago
#28566 Report relays that sbws sees in the consensus, but never chooses to test Core Tor/sbws sbws: 1.1.x-final 4 days ago juga teor #28547 4 days ago
#28567 Report relays that sbws wants to test, but the test doesn't work Core Tor/sbws sbws: 1.1.x-final 4 days ago juga teor #28547 4 days ago
#16559 bwauth code needs to be smarter about failed circuits Core Tor/sbws sbws: unspecified 3 months ago juga TvdW s7r@…, starlight.2015q2@…, juga, teor 3 months ago
#9062 Authorities should describe their bwauth version in their votes Core Tor/Tor Tor: unspecified 11 months ago nickm karsten, pastly 11 months ago
#10922 tor connected to bwauth produces lots of pathbias_count_use_attempt BUG messages Core Tor/Tor Tor: unspecified 21 months ago Sebastian mikeperry 21 months ago
#16696 BWauth no-consensus fallback logic may need revision Core Tor/Tor Tor: unspecified 20 months ago starlight aagbsn, karsten 20 months ago
#19009 bandwidth testing circuits should be allowed to use our guards Core Tor/Tor Tor: unspecified 3 months ago juga arma #22453 3 months ago
#22453 Relays should regularly do a larger bandwidth self-test Core Tor/Tor Tor: unspecified 3 months ago juga arma 3 months ago
#24110 document control protocol router status format surprises when using microdescriptors Core Tor/Tor Tor: unspecified 2 months ago teor atagar, catalyst #7646 2 months ago
#25687 over-report of observed / self-measure bandwidth on fast hardware -- important to torflow / peerflow Core Tor/Tor Tor: unspecified 5 months ago starlight juga 5 months ago
#26478 Unify bandwidth related terms in dir-spec and Tor code. Core Tor/Tor Tor: unspecified 39 hours ago juga 39 hours ago
#26565 Create unit tests for reading measured bandwidths with non-NULL routerstatuses Core Tor/Tor Tor: unspecified 8 months ago juga teor 8 months ago
#26567 Replace ArgumentCharValue with ValueChar in dir-spec and bandwidth-file-spec Core Tor/Tor Tor: unspecified 8 months ago teor 8 months ago
#26797 DirAuths should only read the V3BandwidthsFile once per vote Core Tor/Tor Tor: unspecified 4 months ago teor #27047 4 months ago
#26798 Write unit tests for reading measured bandwidths with non-NULL routerststuses Core Tor/Tor Tor: unspecified 7 months ago teor juga 7 months ago
#26803 Add a bwfile test to tor's make test-network-all Core Tor/Tor Tor: unspecified 7 months ago teor 7 months ago
#26805 Work out why Tor networks with all-zero bandwidths are unstable Core Tor/Tor Tor: unspecified 7 months ago teor neel@… #26803 7 months ago
#26904 Work out if we need to round scanner measured bandwidths to protect individual client usage Core Tor/Tor Tor: unspecified 6 weeks ago teor teor #22898 6 weeks ago
#26905 Work out if we need to round observed relay bandwidths to protect individual client usage Core Tor/Tor Tor: unspecified 6 weeks ago teor #22898 6 weeks ago
#27047 Authorities should keep recent consensuses, votes, and bandwidth files Core Tor/Tor Tor: unspecified 7 months ago teor irl, metrics-team #25925 7 months ago
#28509 Limit relay bandwidth self-tests based on RelayBandwidthRate, not BandwidthRate Core Tor/Tor Tor: unspecified 5 weeks ago teor #22453 5 weeks ago
#28510 When relays reset bandwidth tests, the test waits until the next directory document is received Core Tor/Tor Tor: unspecified 3 months ago teor #22453 3 months ago
#28511 Limit the number of open testing circuits, and the total number of testing circuits Core Tor/Tor Tor: unspecified 5 weeks ago teor #22453 5 weeks ago
#28512 Increase NUM_PARALLEL_TESTING_CIRCS to avoid circuit windows Core Tor/Tor Tor: unspecified 3 months ago teor #22453 3 months ago
#28514 Open NUM_PARALLEL_TESTING_CIRCS when a bandwidth test is initiated Core Tor/Tor Tor: unspecified 3 months ago teor #22453 3 months ago
#28714 "No circuits are opened" on controllers that DisablePredictedCircuits and build 2-hop circuits Core Tor/Tor Tor: unspecified 39 hours ago teor 39 hours ago
#28815 Refactor similar compression buffer code in dircache.c Core Tor/Tor Tor: unspecified 2 months ago teor #21377 2 months ago
#29133 Refactor dirserv_read_measured_bandwidths Core Tor/Tor Tor: unspecified 8 days ago teor juga 8 days ago

{{{#! last two queries do not work with change time, but changetime should be added to see tickets only during sop period ,changetime=May+14%2C+2018..Aug+31%2C+2018 }}}

PF

confirmed:

  • [ ] compared total consensus weights across bandwidth authorities (#25459)
  • [ ] understand how accurate bandwidth authority measurements are (#7177)
  • [ ] find over-weighted relays (#24045, #10791)
  • [ ] map consensus weight vs bandwidth in votes (#24834)
  • [ ] provide bwauth votes via the directory protocol (#21377)
  • [ ] make authorities without bandwidth scanners vote flags based on measured bandwidths (#11327)
  • [ ] reset uptimes / bandwidths when a relay changes IP address (#22308 / sbws #154)
  • [ ] analyse and fix a bug in authority time known (#19162)
  • [ ] some bandwidth weight cases not matching the spec (#20284)
  • [ ] fix DirAuth weights for other relays, or stop them using other relays (#13297)

Related pages

Last modified 4 months ago Last modified on Nov 8, 2018, 3:10:36 PM