Opened 3 years ago

Last modified 7 months ago

#16386 needs_information defect

Chutney generates network with no bandwidth weights

Reported by: asn Owned by:
Priority: Medium Milestone:
Component: Core Tor/Chutney Version:
Severity: Normal Keywords: SponsorS
Cc: ln5, juga Actual Points:
Parent ID: #25925 Points:
Reviewer: Sponsor:

Description

Hello,

when chutney makes a Tor network, it sets the following options in the dirauth torrc:

TestingDirAuthVoteExit *
TestingDirAuthVoteGuard *

which results in a network where all nodes are guards and exits.

This results in totally unbalanced bandwidths (0 guard bandwidth, 0 middle bandwidth, full guard+exit bandwidth) and authorities will not generate bandwidth weights:

[warn] Consensus with empty bandwidth: G=0 M=0 E=0 D=65770 T=65770

This is quite different from the real Tor network.

It would be great if chutney could assign Exit/Guard flag in a smarter way to only a few nodes, so that bandwidth can be allocated more widely.

Child Tickets

Change History (18)

comment:1 Changed 3 years ago by teor

Hi asn,

Setting these flags on every relays was a part of the rapid bootstrap changes that I made in #13163 and related issues.

You're right, it would be great to be able to select parts of the network for these flags.
I can imagine this feature being implemented like this:

TestingDirAuthVoteExit m-n
TestingDirAuthVoteGuard p%-q%

This would vote relays m to n (by fingerprint) as Exits, and relays p% to q% as Guards. We'd probably want to make a similar change to the corresponding TestingDirAuthVoteHSDir flag as well, even if we didn't use it.

We could implement this feature in chutney or in tor:

  • In chutney, we could perhaps do this as part of the templating system, or maybe it could wait for chutney 2?
  • In tor, we could allow the TestingDirAuthVote... parameters to take a m-n or p%-q% argument, by modifying routerset_contains_routerstatus. I'd want to ensure this was only active in test networks, of course.

I am in favour of making this change in tor, where it might assist a wider range of users. And it would work in chutney 2 for free. But I'd want to get nickm's advice.

What do you think?

comment:2 in reply to:  1 Changed 3 years ago by asn

Replying to teor:

We could implement this feature in chutney or in tor:

  • In chutney, we could perhaps do this as part of the templating system, or maybe it could wait for chutney 2?
  • In tor, we could allow the TestingDirAuthVote... parameters to take a m-n or p%-q% argument, by modifying routerset_contains_routerstatus. I'd want to ensure this was only active in test networks, of course.

I am in favour of making this change in tor, where it might assist a wider range of users. And it would work in chutney 2 for free. But I'd want to get nickm's advice.

Hehe. My initial intuition would be to build this feature in chutney.

I prefer our intrusions to the Tor codebase for the purposes of testing networks to be minimal, if it can be helped. Also, chutney is in Python and I imagine that editing the templating logic of chutney shouldn't be too hard for this purpose.

However, if there is indeed a wider range of users, then we should think about it more. Who is doing testing networks with something other than chutney and why? Maybe Shadow needs this feature, but I'm not sure.

comment:3 Changed 3 years ago by teor

I agree - I'd prefer testing infrastructure to be external as much as possible. Makes it easier to deploy and change.

However, this feature could be useful for Shadow, and various people on tor-dev and tor-relays who seem to enjoy setting up test networks without using TestingTorNetwork or chutney. It's almost a requirement to use TestingDirAuthVote... if you want a Tor network to bootstrap in a few seconds rather than minutes.

I'd hate to exclude guardfraction and similar features from testing, just because we don;t have sufficient support in the tor codebase to make them work in test networks. (Does that make sense?)

I'd be happy to prototype it in chutney and see if that's sufficient, or ask on tor-dev - I don't know, what do you think?

comment:4 Changed 3 years ago by asn

I think the second issue here, is that by default chutney relays will publish zero bandwidth. That's because there are no bwauths, and they also don't see any traffic so their bandwidth self-evaluation fails.

My dirty fix is to make each relay publish a random value as its self-reported bandwidth, but it's a bad fix.

comment:5 Changed 3 years ago by teor

Parent ID: #13976

Have you tried using CHUTNEY_DATA_BYTES=104857600 chutney verify?
(That command sends 100MB through each client and HS. It was merged to chutney in #14175.)

Although, I wonder how often the self-reported bandwidths are reported.
Maybe we need to add them to #13976 to get fixed up in test networks.

This ticket may also be related to #14034 (which is about making test networks behave more like the real thing).

comment:6 Changed 3 years ago by teor

Keywords: SponsorS added
Status: newneeds_information

Still trying to work out the best approach to this.

comment:7 Changed 3 years ago by teor

I think we need to reduce CHECK_DESCRIPTOR_INTERVAL and perhaps BANDWIDTH_RECHECK_INTERVAL in test networks. They should probably get a Testing* option and be defaulted to their current values (1 minute, 12 hours); but be ~4 seconds and ~16 seconds by default (and configurable) in test networks.

This will allow us to:

  • Use TestingDirAuthVoteExit, TestingDirAuthVoteGuard, and TestingDirAuthVoteGuardIsStrict to set up the network (#14882, git 359faf5e4b4a88663201c2b42dd89a6f77569856)
  • Run CHUTNEY_DATA_BYTES=104857600 ./chutney verify to add some data to the network (this step might be able to be skipped if self-testing adds enough data by itself)
  • Wait a few seconds for the values to be included in the consensus

comment:8 Changed 3 years ago by ln5

Cc: ln5 added

comment:9 Changed 3 years ago by teor

This change has a chutney component and a tor component:

  • chutney: automatically assign 25% guards, exits, guards + exits, middles (asn, are there any other categories we need here?) - this task
  • tor: report bandwidth more frequently - split into #17036

comment:10 in reply to:  9 Changed 3 years ago by asn

Replying to teor:

This change has a chutney component and a tor component:

  • chutney: automatically assign 25% guards, exits, guards + exits, middles (asn, are there any other categories we need here?) - this task
  • tor: report bandwidth more frequently - split into #17036

Sounds reasonable!

comment:11 Changed 2 years ago by nickm

Owner: nickm deleted
Status: needs_informationassigned

Remove myself as owner from old chutney tickets; that's not what we use owner for.

comment:12 Changed 2 years ago by nickm

Status: assignedneeds_information

comment:13 Changed 21 months ago by teor

Severity: Normal

This was partially fixed in 0.3.0 with a new consensus method that initialises weights to 1 (see #14881).

comment:14 Changed 12 months ago by teor

We should only implement early bandwidth reporting for test networks, or networks with no -default authority sets. Implementing in the real network can expose client guards (#23856), but if we only do fast reporting in the first 24 hours, it might be ok (#24104).

comment:15 Changed 8 months ago by juga

Cc: juga added

comment:16 Changed 7 months ago by teor

Parent ID: #13976#24250

#24250 will fix this issue by making relays report 1 as their minimum bandwidth.

comment:17 Changed 7 months ago by juga

Parent ID: #2425025925

comment:18 Changed 7 months ago by juga

Parent ID: 25925#25925
Note: See TracTickets for help on using tickets.