Opened 19 months ago

Last modified 3 months ago

#25625 assigned enhancement

Make CollecTor's file structure description part of Metrics-Web's CollecTor docs

Reported by: iwakeh Owned by: metrics-team
Priority: Medium Milestone:
Component: Metrics/Website Version:
Severity: Normal Keywords:
Cc: metrics-team Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description (last modified by iwakeh)

Transfer PROTOCOL text file to some useful place and format in Metrics-Web.

Child Tickets

Attachments (1)

task-25625-draft.pdf (190.2 KB) - added by karsten 19 months ago.

Download all attachments as: .zip

Change History (13)

comment:1 Changed 19 months ago by iwakeh

Description: modified (diff)

comment:2 Changed 19 months ago by iwakeh

This ticket should be tackled by someone familiar with metrics-web jsps and web design.

comment:3 Changed 19 months ago by karsten

Owner: changed from metrics-team to karsten
Status: newaccepted

Changed 19 months ago by karsten

Attachment: task-25625-draft.pdf added

comment:4 Changed 19 months ago by karsten

Status: acceptedneeds_review

Please find the attached PDF or alternatively commit 512c5f0 in my task-25625 branch with a possible start. The idea is that the most important parts of this protocol are the file names of files in recent/ and archive/ as well as paths contained in file names.

If this looks reasonable, I'll include similar paragraphs for all descriptor types, and then we can see if something else remains from the protocol that we need to include, too.

comment:5 Changed 16 months ago by karsten

irl, can you take over the review part from iwakeh here?

comment:6 Changed 16 months ago by irl

Reviewer: irl

comment:7 Changed 16 months ago by irl

Status: needs_reviewneeds_revision

with YYYY-MM again being year and month of the descriptor publication time, X and Y being the first and second character of the hex-encoded, lower-case SHA-1 descriptor digest, and DIGEST being that descriptor digest in full.

Y is year but is also part of the fingerprint. Can we use YYYY-MM/X/Z/DIGEST (or some other letters) instead?

I feel this chunk may get repeated a lot. Let's pick some letters we like and add this as a paragraph under the top "Data Formats" heading. We also need to explain what HH, MM (which means both month and minutes at the moment) and SS mean. This could even be a table, e.g.:

<table class="table table-striped">
  <thead>
    <tr>
      <th>Marker</th>
      <th>Description</th>
    </tr>
   </thead>
   <tbody>
    <tr>
      <td>YYYY</td>
      <td>Year</td>
    </tr>
    <tr>
      <td>MM</td>
      <td>Month (2-digits; with leading zero)</td>
    </tr>
    ....
  </tbody>
</table>

comment:8 Changed 15 months ago by karsten

Hmm, is this something that you can write a patch for, and where I can then go ahead and fill in all the details? It's something that I keep postponing, because I'm not yet sure how to best present things in order for others to make sense of. Of course, if you're already too busy with other things, let me know, and I'll give it another try.

comment:9 Changed 11 months ago by irl

I'd forgotten we were doing this and I've been referencing section numbers for the original specification in both the modern CollecTor technical report and in the documentation for the prototype. I had also added the protocol to https://spec.torproject.org/collector-protocol.

I think for now I'd prefer to set this to wontfix, and instead start thinking about what version 2 of this will look like. There are some backwards-incompatible changes that I think we need to make. If we're doing that anyway, there are more backwards-incompatible changes that I think we really should make.

comment:10 Changed 11 months ago by karsten

Reviewer: irl
Status: needs_revisionneeds_information

Or let's move this to needs_information and circle back in a couple weeks or months.

comment:11 Changed 11 months ago by irl

Agreed.

comment:12 Changed 3 months ago by karsten

Owner: changed from karsten to metrics-team
Status: needs_informationassigned

I'm not working on this ticket at the moment. Re-assigning to metrics-team.

Note: See TracTickets for help on using tickets.