Opened 4 years ago

Closed 4 years ago

Last modified 4 years ago

#15866 closed defect (fixed)

BridgeDB has less bridges because the BridgeAuthority appears to be giving it incorrect networkstatuses

Reported by: isis Owned by: isis
Priority: Very High Milestone:
Component: Circumvention/BridgeDB Version:
Severity: Keywords: bridgedb-parsers, bridgedb-dist, bridgeauth, tor-auth, bridgedb-0.3.2
Cc: isis, yawning, Lucky, nickm, arma Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

tl;dr: We really need to redesign and rewrite the BridgeAuthority. For now, BridgeDB is going to ignore the BridgeAuthority's networkstatus documents.

There appears to be something quite wrong with the way the BridgeAuthority produces its networkstatus-bridges documents.

As explained on #9380, BridgeDB started verifying signatures and matching digests for the full chain of bridge descriptors from networkstatusserver-descriptorextrainfo. Thus, if a bridge is missing from the BridgeAuthority's networkstatus-bridges document, then it doesn't exist as far as BridgeDB is concerned. This afternoon, users were complaining that BridgeDB was only giving one bridge at a time (which is normal behaviour when BridgeDB doesn't have enough bridges).

To get to the point, Bridgedb doesn't have very many bridges because the networkstatus-bridges document is completely whack — it's missing 83.41% of the total bridges. It's not that the file is empty. It's just missing most of the bridges that it should have, and instead it has strange networkstatus documents in it, like for bridges which don't exist anymore and documents which reference seemingly non-existent server-descriptors.

This is what part of a second of descriptor parsing looks like (sanitised):

03:33:24 WARNING  L144:Main.load()              Received server descriptor for bridge '6722DAAEADE603C9626975ED8C8CF545236C44A7' which wasn't in the networkstatus!
03:33:24 WARNING  L144:Main.load()              Received server descriptor for bridge 'F151AC2EE601361D125D5E5963178038E606B440' which wasn't in the networkstatus!
03:33:24 WARNING  L144:Main.load()              Received server descriptor for bridge '63E42362C38B0D482B9BED7CA3B6D8F513B85AC1' which wasn't in the networkstatus!
03:33:24 WARNING  L144:Main.load()              Received server descriptor for bridge '8F0A9018A4313D0CFCBA79004F9DE5FE66E73368' which wasn't in the networkstatus!
03:33:24 WARNING  L144:Main.load()              Received server descriptor for bridge 'FC80E087A8728AAD0A8FE946C5C4EEE2F937487D' which wasn't in the networkstatus!
03:33:24 WARNING  L144:Main.load()              Received server descriptor for bridge '97255849FB90EAEDE3DDC9CDA088A1ECCF71FDC2' which wasn't in the networkstatus!
03:33:24 WARNING  L149:Main.load()              The server-descriptor digest for bridge '2A624DD84370EDAC58BD73D427B1BBFF53C72315' doesn't match the digest reported by the BridgeAuthority in the networkstatus doc
ument:
Digest reported in networkstatus: D47CC3D7FEACF75ABB780B0F63044CEB4D7101F4
Actual descriptor digest:         39C622B8C7C0CB90BFDE273149E57B6CAF06AAD7

03:33:24 WARNING  L144:Main.load()              Received server descriptor for bridge 'E3C750F06B9043B2DAD4275613FBF355EAB161D2' which wasn't in the networkstatus!
03:33:24 WARNING  L144:Main.load()              Received server descriptor for bridge '95374284A3A6B0C289DD8ED49B49A32DF769A677' which wasn't in the networkstatus!
03:33:24 WARNING  L144:Main.load()              Received server descriptor for bridge 'A699637AAF2BB6DD2FDC338647BF5DBE668A79AC' which wasn't in the networkstatus!
03:33:24 WARNING  L144:Main.load()              Received server descriptor for bridge 'ABD206AA7A2C607EAA641D8567A307E031968DBA' which wasn't in the networkstatus!
03:33:24 WARNING  L144:Main.load()              Received server descriptor for bridge '265AD3890E6FE46E84EE2756815E7101976E4E76' which wasn't in the networkstatus!
03:33:24 WARNING  L144:Main.load()              Received server descriptor for bridge '2038634774046BB0D58780AB4718462427E1A372' which wasn't in the networkstatus!
03:33:24 WARNING  L144:Main.load()              Received server descriptor for bridge '6E2AD7E1D9A912058A895193FB94EB0AE2B91B7E' which wasn't in the networkstatus!
03:33:24 WARNING  L144:Main.load()              Received server descriptor for bridge 'FFBD398A3BF169A9FD60620AE2C2C1CC1C9493DE' which wasn't in the networkstatus!
03:33:24 WARNING  L144:Main.load()              Received server descriptor for bridge 'E83EB92BB3DE7FFA9AC188313A63E023809EAD44' which wasn't in the networkstatus!
03:33:24 WARNING  L144:Main.load()              Received server descriptor for bridge '810AF92A276DC364969F16B4A27C8529E0D771B7' which wasn't in the networkstatus!
03:33:24 WARNING  L144:Main.load()              Received server descriptor for bridge '3D73330F11479E32A0E88AAF4E7E2984A7F743BA' which wasn't in the networkstatus!
03:33:24 WARNING  L144:Main.load()              Received server descriptor for bridge 'C2549EB8853561C8BB798B2661697E80579974AD' which wasn't in the networkstatus!
03:33:24 WARNING  L144:Main.load()              Received server descriptor for bridge '7313AD77ED8AF12E4D91835CFB21BBCCDC900A13' which wasn't in the networkstatus!
03:33:24 WARNING  L144:Main.load()              Received server descriptor for bridge '8FD5261825BC50EA557EBCFF92FABEE6749855B5' which wasn't in the networkstatus!
03:33:24 WARNING  L144:Main.load()              Received server descriptor for bridge 'D096A70EFD67C1198DA0DBA06CDC1B55075FB326' which wasn't in the networkstatus!
03:33:24 WARNING  L144:Main.load()              Received server descriptor for bridge '02327187D5A3F89F864200D3A697CA4B8C8246CC' which wasn't in the networkstatus!
03:33:24 WARNING  L144:Main.load()              Received server descriptor for bridge 'C9D611438E7B127DD06D1CA49BCF39634C1E92EA' which wasn't in the networkstatus!
03:33:24 WARNING  L144:Main.load()              Received server descriptor for bridge '2C398670D16EC6C311AE3B5B035D6154D1B871E2' which wasn't in the networkstatus!
03:33:24 WARNING  L144:Main.load()              Received server descriptor for bridge '0BD5EEC61594FC25BF565C5DCB5B9C0F9F99B5F0' which wasn't in the networkstatus!
03:33:24 WARNING  L144:Main.load()              Received server descriptor for bridge 'FB23D1A30043ABDD0C6DA9EAD428DF49BC65F7F0' which wasn't in the networkstatus!
03:33:24 WARNING  L144:Main.load()              Received server descriptor for bridge '4B0A85A4FE8AB67F0F769FD1EC25C27B057271C5' which wasn't in the networkstatus!
03:33:24 WARNING  L144:Main.load()              Received server descriptor for bridge '575A7C152ECDE01756564E89F74727F8C259FBA9' which wasn't in the networkstatus!
03:33:24 WARNING  L144:Main.load()              Received server descriptor for bridge 'BE9182355E2A10303D7F69BCECD14EF89A568518' which wasn't in the networkstatus!
03:33:24 WARNING  L144:Main.load()              Received server descriptor for bridge '0549DCE8B5FAE293BA94D5BEB81782C54AA37C3D' which wasn't in the networkstatus!
03:33:24 WARNING  L144:Main.load()              Received server descriptor for bridge '9DFA242252B2D85C9889C7270D5B6C562E9AC711' which wasn't in the networkstatus!
03:33:24 WARNING  L144:Main.load()              Received server descriptor for bridge '627BDCE8D86F4E4406D41A8B3081509CF9A99EA0' which wasn't in the networkstatus!
03:33:24 WARNING  L144:Main.load()              Received server descriptor for bridge 'F7198BBF43EDBB32DFF7C7923A8799884471FFE1' which wasn't in the networkstatus!
03:33:24 WARNING  L144:Main.load()              Received server descriptor for bridge '3DB7D81C77A164DA0EE5B1DB915C78047EDBB4B5' which wasn't in the networkstatus!
03:33:24 WARNING  L144:Main.load()              Received server descriptor for bridge 'FA1670376088B544AF3C54D117E3325EF6977B50' which wasn't in the networkstatus!
03:33:24 WARNING  L144:Main.load()              Received server descriptor for bridge '418AE2105849C379EBD8F416B5EF670793A4E719' which wasn't in the networkstatus!
03:33:24 WARNING  L144:Main.load()              Received server descriptor for bridge 'EC17838F9B34A9009CD2CA8296B50AA4124EC963' which wasn't in the networkstatus!
03:33:24 WARNING  L144:Main.load()              Received server descriptor for bridge '0C82FDAFFB41B5CC3C209C6DC50B33B03FA1C316' which wasn't in the networkstatus!
03:33:24 WARNING  L144:Main.load()              Received server descriptor for bridge '20273A6DC581B92F6D30330D7BD81DFDE45A9A92' which wasn't in the networkstatus!
03:33:24 WARNING  L144:Main.load()              Received server descriptor for bridge 'F8855C2CEB6FE2D5256795FFAFC072904790F334' which wasn't in the networkstatus!
03:33:24 WARNING  L144:Main.load()              Received server descriptor for bridge '5426A87A1914A4414031390C48561AC6B80A502F' which wasn't in the networkstatus!
03:33:24 WARNING  L144:Main.load()              Received server descriptor for bridge '3BFFE8B3AB2BEF7BB8D848687899739AF7676E6E' which wasn't in the networkstatus!
03:33:24 WARNING  L144:Main.load()              Received server descriptor for bridge '32F8F2DA49B414374D22525A43783A3A757F1333' which wasn't in the networkstatus!
03:33:24 WARNING  L144:Main.load()              Received server descriptor for bridge 'DCFECBFB14C241487E48117B82FC8D40B9C89FB5' which wasn't in the networkstatus!
03:33:24 WARNING  L144:Main.load()              Received server descriptor for bridge 'B45D16748A0A458AAF1E1CF12F6A0E1470221AC1' which wasn't in the networkstatus!
03:33:24 WARNING  L144:Main.load()              Received server descriptor for bridge '0C56BC8C6FA39D3D6B474B311412545B656FFDCB' which wasn't in the networkstatus!
03:33:24 WARNING  L144:Main.load()              Received server descriptor for bridge '0D0870C71AAFDE28298748A7D6C1C7BADE3E648D' which wasn't in the networkstatus!
03:33:24 WARNING  L144:Main.load()              Received server descriptor for bridge '22CA5908E13A94FFD9E3A549D3B5D297EC4C491A' which wasn't in the networkstatus!
03:33:24 WARNING  L144:Main.load()              Received server descriptor for bridge '7DBC81F21827C3A08128D3E0E79772C78DCDC223' which wasn't in the networkstatus!
03:33:24 WARNING  L144:Main.load()              Received server descriptor for bridge 'C96261D3C370A1CD0CEB47985B0130B1EF25D04E' which wasn't in the networkstatus!
03:33:24 WARNING  L144:Main.load()              Received server descriptor for bridge '0D1B368FBB152B18348BBE0930DD3C891B208E9F' which wasn't in the networkstatus!
03:33:24 WARNING  L144:Main.load()              Received server descriptor for bridge 'A938247AC831B1F9BE4F8AF24291A7D3402FB3E8' which wasn't in the networkstatus!
03:33:24 WARNING  L144:Main.load()              Received server descriptor for bridge '9758F5954682E7677CFC6389AD95F7B60BB8A7C5' which wasn't in the networkstatus!


Because of this, BridgeDB has only 901 bridges right now, when in reality, there are 5429 bridges.

My proposed solution is put a THE_BRIDGE_AUTH_IS_A_BROKEN_PIECE_OF_SHIT = True option in BridgeDB's config file, and ignore the BridgeAuthority altogether¹. Combined with other problems like #11216 and #15707, the BridgeAuthority now serves essentially no purpose beyond bridge ORPort reachability tests and being a wastebasket for whatever descriptors anyone wants to throw at it.

¹ BridgeDB will still parse networkstatuses for the Bridge flags. That's it.

Child Tickets

Change History (9)

comment:1 Changed 4 years ago by yawning

Cc: yawning added; Yawning removed

comment:2 Changed 4 years ago by nickm

Can we get a ticket or two for this under the Tor component too?

comment:3 Changed 4 years ago by isis

Keywords: tor-auth added
Status: newneeds_review

The proposed solution is implemented in my fix/15866-broken-bridgeauth branch.

comment:4 in reply to:  2 Changed 4 years ago by isis

Replying to nickm:

Can we get a ticket or two for this under the Tor component too?


Will #15883 do? Or would you like a more specific ticket for investigating whether or not there is a problem in the BridgeAuthority networkstatus document creation code? Or a ticket for investigating if Tonga is misconfigured?

comment:5 in reply to:  3 ; Changed 4 years ago by isis

Replying to isis:

The proposed solution is implemented in my fix/15866-broken-bridgeauth branch.

There are now unittests for the changes as well, in the same branch.

comment:6 in reply to:  5 Changed 4 years ago by isis

Keywords: bridgedb-0.3.2 added

Replying to isis:

Replying to isis:

The proposed solution is implemented in my fix/15866-broken-bridgeauth branch.

There are now unittests for the changes as well, in the same branch.

Unless someone speaks up or has a better idea, this is merged for bridgedb-0.3.2. I'm getting yelled at by a bunch of people/users, and that number is likely to increase the longer this drags on. :(

comment:7 Changed 4 years ago by isis

Resolution: fixed
Status: needs_reviewclosed

comment:8 Changed 4 years ago by isis

This worked quite well, with one minor glitch:

From BridgeDB's log files:

07:32:28 INFO     L200:Main.load()              Done inserting 4643 bridges into splitter.
07:32:28 DEBUG    L204:persistent.save()        Saving state to:        '/srv/bridges.torproject.org/run/bridgedb.state'
07:32:28 DEBUG    L166:persistent.load()        Retrieving state from:  '/srv/bridges.torproject.org/run/bridgedb.state'
07:32:28 INFO     L428:Main.reload()            Bridges loaded: 3444

[…]

07:32:29 INFO     L670:Bridges.addRing()        Bridges inserted into Email-IP4 subring: 990
07:32:29 INFO     L670:Bridges.addRing()        Bridges inserted into Email-IP6 subring: 50
07:32:29 INFO     L438:Main.reload()            Bridges allotted for Email distribution: 990

[…]

07:32:30 INFO     L447:Main.reload()            Bridges allotted for HTTPS distribution: 2054
07:32:30 INFO     L448:Main.reload()                    Num bridges:    Filter set:
07:32:30 INFO     L456:Main.reload()                    64 bridges      filterAssignBridgesToRing filterBridgesByIP6
07:32:30 INFO     L456:Main.reload()                    1052 bridges    filterAssignBridgesToRing filterBridgesByIP4
07:32:30 INFO     L456:Main.reload()                    45 bridges      filterAssignBridgesToRing filterBridgesByIP6
07:32:30 INFO     L456:Main.reload()                    1002 bridges    filterAssignBridgesToRing
07:32:30 INFO     L456:Main.reload()                    1002 bridges    filterAssignBridgesToRing filterBridgesByIP4
07:32:30 INFO     L456:Main.reload()                    1052 bridges    filterAssignBridgesToRing
07:32:30 INFO     L459:Main.reload()            Total subrings for HTTPS: 6


It's probably just an issue with one of the old hashring classes in bridgedb.Bridges having a stupid/buggy __len__() method that double-counts bridges which are in a hashring and also in one of that hashring's sub-hashrings.

comment:9 Changed 4 years ago by arma

Ok, I just explored what I think is the cause of this bug. I believe Tor is behaving as expected.

Specifically, Tonga is writing a networkstatus file each hour, with a stanza for each relay that is a) sufficiently recent (within the past 24 hours) and b) it has known about long enough to test.

Isis just gave me a list of descriptors that were "left out" of the networkstatus, and they were all descriptors that fail to meet 'a' and 'b' above.

So I think the networkstatus is correct, and we should continue to ignore old descriptors that aren't mentioned in the networkstatus.

Note: See TracTickets for help on using tickets.