Changes between Initial Version and Version 14 of Ticket #21378

Nov 5, 2018, 9:51:35 AM (10 months ago)

I just went through the long discussion above and tried to identify next steps. irl's list of needed changes looks pretty good. I'll add some thoughts to these steps below that we need to discuss when implementing this.

Replying to irl:

It looks like the changes that are needed are: 1) Teach RelayDescriptorDownloader to download the new URL (in the downloadDescriptors function)

  • We can either attempt to fetch this file from each authority every time, or we can have a config option which authorities should have them. In the future, we can switch to fetching only those files that are referenced from votes, unless for some reason we want to have non-referenced files, too.
  • The relaydescs module runs twice per hour, so it's going to download the file twice every hour. Again, if we only fetch referenced files, we wouldn't download the same file more than once. But it sounds like the initial version will be rather simple in this regard. Which is fine.
  • I assume there are no plans that authorities serve bandwidth files of other authorities? That's different for votes which are cached by other authorities. Should be fine, but something to consider for the future.

2) Teach RelayDescriptorParser to recognise the file

  • While we're waiting for #21377, can we have a sample file to start writing some parsing code?

3) Teach ArchiveWriter where it should put the files in CollecTor's hierachy

  • Let's discuss what should go into the file name. Timestamp, fingerprint, and digest? Maybe something similar to the vote file name format (with some parts shortened): 2018-11-05-09-00-00-vote-EFCBE720[...]-0D97EDB6[...]?
  • As part of this step, we might have to teach metrics-lib to recognize the new descriptor type. I believe that CollecTor will store it anyway, but it's going to complain loudly. Just in case it acts up, we can teach metrics-lib to just recognize the descriptor type without providing getters for descriptor contents.


  • Ticket #21378

    • Property Keywords tor-bwauth tor-dirauth added
    • Property Cc teor metrics-team added
    • Property Priority changed from Low to Medium
    • Property Summary changed from Collector should archive bwauth votes to Archive bwauth bandwidth files
  • Ticket #21378 – Description

    initial v14  
    11The raw bwauth votes (sample: contain information such as last measured time, circuit failures and (eventually) scanner information. This can be used for debugging purposes.
    3 Blocked by #21377
     3Blocked by #21377, possible next steps in [#comment:14 comment 14].