Opened 12 months ago

Closed 12 months ago

Last modified 11 months ago

#28042 closed defect (fixed)

Restrictions on the number of minimum measurements to include a relay in the bandwidth file give very few relays

Reported by: juga Owned by:
Priority: Medium Milestone: sbws: 1.0.x-final
Component: Core Tor/sbws Version:
Severity: Normal Keywords: sbws-1.0-nice-closed-moved-20181128
Cc: pastly, juga, teor, micah Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

In #27338##comment:6, teor suggested:

If any of these things are true, do not put the relay in the bandwidth file:

  • there are less than 2 sbws measured bandwidths
  • all the sbws measured bandwidths are within 24 hours of each other

With these restrictions and the current data, the number of relays to be included in the bandwidth file is only 242.
Removing 1st restriction (ie, min num measurements: 1), there're 5209 relays
Incrementing the number of hours between each other to 3d, doesn't give any relays
Decrementing the number of hours between each other to 12h, gives 302 relays

So:

  1. we should check prioritization, in case we're not doing it correctly, probably should open a new ticket for that
  2. do we want to publish bandwidth files with such a small number of relays?
  3. should we eliminate the restriction of having at least 2 measured bandwidths?

Child Tickets

TicketStatusOwnerSummaryComponent
#28061closedCheck prioritisation, it should make 2 measurements that are 24 appartCore Tor/sbws
#28062closedjugaPublish bandwidth files only when they contain only the 60% of relaysCore Tor/sbws
#28155closedjugaWarn when the minimum percent was reached beforeCore Tor/sbws
#28159closedjugaConsider disabling rtt measurementsCore Tor/sbws

Change History (7)

comment:1 Changed 12 months ago by juga

Cc: micah added

Add micah to CC, he asked whether they should give to the dirauth such small files

comment:2 in reply to:  description ; Changed 12 months ago by teor

Replying to juga:

In #27338##comment:6, teor suggested:

If any of these things are true, do not put the relay in the bandwidth file:

  • there are less than 2 sbws measured bandwidths
  • all the sbws measured bandwidths are within 24 hours of each other

With these restrictions and the current data, the number of relays to be included in the bandwidth file is only 242.
Removing 1st restriction (ie, min num measurements: 1), there're 5209 relays
Incrementing the number of hours between each other to 3d, doesn't give any relays
Decrementing the number of hours between each other to 12h, gives 302 relays

So:

  1. we should check prioritization, in case we're not doing it correctly, probably should open a new ticket for that

If sbws' current prioritisation rule says "stop when there is 1 measurement", we need to change it to "stop when there are 2 measurements that are at least 24 hours apart".

  1. do we want to publish bandwidth files with such a small number of relays?

Torflow only publishes files when they contain 60% of the relays in the network. We should open a ticket for sbws to do the same thing. (If we want to change from 60%, we can do it in sbws 1.1.)

https://gitweb.torproject.org/torflow.git/tree/NetworkScanners/BwAuthority/aggregate.py#n46

  1. should we eliminate the restriction of having at least 2 measured bandwidths?

No, we should keep measuring until there are enough bandwidths. See 1.

comment:3 in reply to:  2 Changed 12 months ago by pastly

Replying to teor:

Replying to juga:

In #27338##comment:6, teor suggested:

If any of these things are true, do not put the relay in the bandwidth file:

  • there are less than 2 sbws measured bandwidths
  • all the sbws measured bandwidths are within 24 hours of each other

With these restrictions and the current data, the number of relays to be included in the bandwidth file is only 242.
Removing 1st restriction (ie, min num measurements: 1), there're 5209 relays
Incrementing the number of hours between each other to 3d, doesn't give any relays
Decrementing the number of hours between each other to 12h, gives 302 relays

So:

  1. we should check prioritization, in case we're not doing it correctly, probably should open a new ticket for that

If sbws' current prioritisation rule says "stop when there is 1 measurement", we need to change it to "stop when there are 2 measurements that are at least 24 hours apart".

It doesn't. It is min-priority queue where priority is based on how recently a relay's measurements were made. Read more here. (edit: or here)

sbws never "stops."

It would be fair to say sbws prioritizes getting 1 measurement for each relay, then 2 measurements, then 3, then 4...

For further explaination about the prioritizer, see my comment on #28042

Last edited 12 months ago by pastly (previous) (diff)

comment:4 Changed 12 months ago by juga

  1. When there's not enough percentage or relays measured to create a bandwidth file, should sbws
    • 4.1 remove the link to latest.v3bw? A dirauth running sbws locally, won't find the file and won't include it A dirauth running sbws remotely, will remain with the older one locally, unless that's detected and remove it
    • 4.2 do not do anything? A dirauth running sbws locally, will serve the old file A dirauth running sbws remotely, will remain with the older one locally, unless that's detected and remove it

So 4.1 seems slightly better. Opinions?

comment:5 in reply to:  4 Changed 12 months ago by teor

Replying to juga:

  1. When there's not enough percentage or relays measured to create a bandwidth file, should sbws
    • 4.1 remove the link to latest.v3bw? A dirauth running sbws locally, won't find the file and won't include it A dirauth running sbws remotely, will remain with the older one locally, unless that's detected and remove it
    • 4.2 do not do anything? A dirauth running sbws locally, will serve the old file A dirauth running sbws remotely, will remain with the older one locally, unless that's detected and remove it

So 4.1 seems slightly better. Opinions?

Option 3: create a file that only contains the headers, without any relays. See #28076.

comment:6 Changed 12 months ago by juga

Resolution: fixed
Status: newclosed

Closing since all children are closed.

comment:7 Changed 11 months ago by teor

Keywords: sbws-1.0-nice-closed-moved-20181128 added
Milestone: sbws 1.0 (MVP nice)sbws: 1.0.x-final

Move all closed sbws 1.0 nice tickets to sbws 1.0.x-final

Note: See TracTickets for help on using tickets.