Opened 5 years ago

Last modified 18 months ago

#13630 new project

Bandwidth Authority Renovation

Reported by: aagbsn Owned by:
Priority: Medium Milestone:
Component: Core Tor/Torflow Version:
Severity: Normal Keywords:
Cc: Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

This ticket describes the Bandwidth Authority Refactoring Project.

Goals:

Replace pyTorCtl with stem[1] or txtorcon
Address Critical Issues
Address Scalability Issues
Implement New Feedback Mechanisms


Critical:

Bandwith Authorities are leaving 10 percent of the network unmeasured
https://trac.torproject.org/projects/tor/ticket/13450

Understand how accurate the bandwidth authority estimates are
https://trac.torproject.org/projects/tor/ticket/7177

The slowest fraction of the network takes 18 hours to complete
https://trac.torproject.org/projects/tor/ticket/9795

BwAuth takes a long time to scan the network
https://trac.torproject.org/projects/tor/ticket/13449

Make a torflow option that starts every testing circuit at a local Tor relay first
https://trac.torproject.org/projects/tor/ticket/9762

Decentralize:

Decentralized measurement for network load balancing
https://trac.torproject.org/projects/tor/ticket/5464

Parallelise:

Split BwAuth children more equally
https://trac.torproject.org/projects/tor/ticket/3440

Increase the number of parallel scanners in a BandwidthAuthority from 4 to 8
https://trac.torproject.org/projects/tor/ticket/9795

Feedback Mechanisms:

Implement bwauth cap for latency
https://trac.torproject.org/projects/tor/ticket/4708

Implement bwauth cap for TCP socket exhaustion
https://trac.torproject.org/projects/tor/ticket/4709

Bwauth: Use only "slow" measurements for positive feedback
https://trac.torproject.org/projects/tor/ticket/4730

Bandwidth auths should publish average and weighted onionskin failure rates
https://trac.torproject.org/projects/tor/ticket/7281

Detect overtuned exit relays
https://trac.torproject.org/projects/tor/ticket/10791

untracked issues?:

Scalability - how are the BwAuths scaling?
How long does this take?
How much longer will it take with 10000 relays?
Network stability?
Circuit failure rates?
Partition detection?

papers:

EigenSpeed: Secure Peer-to-peer Bandwidth Evaluation
https://www.usenix.org/legacy/event/iptps09/tech/full_papers/snader/snader.pdf
http://www.cs.ucla.edu/~nrl/CapProbe/

Requirements and Software Design for a Better Tor Performance Measurement Tool
https://research.torproject.org/techreports/torperf2-2013-10-30.pdf


[1] stem is "is a synchronous (threaded) Python library". There is/was? a plan to integrate it with txtorcon[2].

[2] https://www.torproject.org/getinvolved/volunteer.html.en#txtorcon-stemIntegration

Child Tickets

TicketStatusOwnerSummaryComponent
#2550newmikeperrybwauth should reschedule quicker bandwidth test when bandwidthrate changes?Core Tor/Torflow
#2742closedmikeperryImproved flushing of bw auth scanner log message output?Core Tor/Torflow
#2888newmikeperryTesting framework/dataset for bw authsCore Tor/Torflow
#3679closedTorCtl Event Parsing RewriteTorctl
#3701closedPyTorCtl shouldn't os.kill() itselfCore Tor/Torflow
#3798closedBwAuthority does not exit cleanly; Traceback in _eventLoop at exitCore Tor/Torflow
#4015closedaagbsnBwAuthority postgres scanner freezes with changes from #3679 TorCtl Event Parsing RewriteCore Tor/Torflow
#4270newmikeperryProvide optimized postgres config for bw scannersCore Tor/Torflow
#4271newPerform some integrity checking in bw auth fetchesCore Tor/Torflow
#4359newmikeperryMinimize time between new relay appearing and having some bw vote for itCore Tor/Torflow
#4708newImplement bwauth cap for latencyCore Tor/Torflow
#4709newImplement bwauth cap for TCP socket exhaustionCore Tor/Torflow
#4909newbwscan never cleans up log filesCore Tor/Torflow
#4910closedaagbsnbwauthpid has confusing parameter namesCore Tor/Torflow
#9788closedaagbsnTorflow BandwidthAuthority url lists are baked into sourceCore Tor/Torflow
#9789newTorFlow BandwidthAuthority sleep scheduleCore Tor/Torflow
#10791newDetect overtuned exit relaysCore Tor/Torflow
#13449closedaagbsnbwauth takes a long time to scan the networkCore Tor/Torflow
#14177closedaagbsnbwauth spends some 40 minutes "updating rank history". Every hour.Core Tor/Torflow
#15238newImprove or replace TorFlowCore Tor/Torflow
#23412closedDo bandwidth authorities reset history when the relay fingerprint changes?Core Tor/Torflow

Change History (5)

comment:1 Changed 5 years ago by atagar

[1] stem is "is a synchronous (threaded) Python library". There is/was? a plan to integrate it with txtorcon[2].

Yup. This is something both meejah and I would love for somebody to take on. We've directed folks to it for GSoC but no nibbles...

https://www.torproject.org/getinvolved/volunteer.html.en#txtorcon-stemIntegration

comment:2 Changed 5 years ago by aagbsn

I can't edit the original ticket, so here are additional related tickets:

Publish and use circuit success rates in extrainfo descriptors
https://trac.torproject.org/projects/tor/ticket/7509

Bandwidth auths should publish average and weighted onionskin failure rates
https://trac.torproject.org/projects/tor/ticket/7281

comment:3 Changed 19 months ago by teor

Severity: Normal

Priorities and Severities in torflow are meaningless, setting them all to Medium/Normal.

comment:4 Changed 19 months ago by teor

Owner: aagbsn deleted
Status: newassigned

aagbsn was the default owner, unassigning

comment:5 Changed 18 months ago by teor

Status: assignednew

Mark all tickets that are assigned to nobody as "new".

Note: See TracTickets for help on using tickets.