Opened 10 months ago

Last modified 2 months ago

#28664 assigned defect

Describe consensus digest calculation

Reported by: atagar Owned by:
Priority: Medium Milestone: Tor: unspecified
Component: Core Tor/Tor Version:
Severity: Normal Keywords: fast-fix, tor-spec, doc, postfreeze-ok, 040-deferred-20190220, teor-backlog
Cc: Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

Hi lovely network team folks. No doubt I'm being blind but I'm having difficulty figuring out how to calculate network status document digests.

During the voting period (minutes 55-60 of the hour) I fetched the detached signatures and upcoming consensus. The detached signatures cite the digest...

% curl http://128.31.0.39:9131/tor/status-vote/next/consensus-signatures > sigs
% curl http://128.31.0.39:9131/tor/status-vote/next/consensus > next_consensus
% grep consensus-digest sigs 
consensus-digest 296BA01987256A1C8EFB20E17667152DCFA50755

But in trying hex encoded sha1s of various ranges of the consensus I'm having difficulty getting a value that matches that. No doubt I'm missing something but the spec is unhelpfully vague saying simply 'this is the digest' without citing a section describing how it's calculated...

https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt#n3309

It's probably buried in there somewhere but I've skimmed through the spec a few times and it's not jumping out at me. Mind clarifying in the spec how to calculate this?

Thanks!

Child Tickets

Change History (11)

comment:1 Changed 10 months ago by atagar

Ah ha! Through more experimentation got it. It's the sha1/hex up though the first 'directory-signature ' (inclusive of the space). That space is what was buggering me up.

Still I'd appreciate if the spec was clarified.

Thanks!

comment:2 Changed 10 months ago by teor

Keywords: tor-spec doc added
Milestone: Tor: 0.4.0.x-final

You're right, it's ambiguous, but not quite in the way you think.

The signing documents section applies to digests, and digests can be on the same line as their keyword:
https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt#n276

Here's what we can do to fix this spec issue:

  1. In section 1.3 Signing documents, clarify that the digest includes all characters before the digest or signature starts. If the digest or signature is on the same line as the keyword, the final character is a space. If it's on a separate line, the final character is a newline.
  2. Point to section 1.3 from elsewhere in the document.

comment:3 Changed 10 months ago by arma

sounds smart to me!

comment:4 Changed 10 months ago by atagar

Thanks teor, sounds good! For me the operative bit I needed was the digested range. Section 3.1 tells us quite a bit but my eyes glazed over the 'final "Signature Item"' bit (and in re-reading that it seems to say the last signature, not the first). I'm delighted for us to make the fixes you mention but how do you feel about adjusting the detached signature description to something like the following?

From:

    "consensus-digest" SP Digest NL

        [At start, at most once.]

        The digest of the consensus being signed.

To:

    "consensus-digest" SP Digest NL

        [At start, at most once.]

        The digest of the consensus being signed. This is the uppercase
        hex encoded sha1 digest of the next hour's consensus document 
        from it beginning up through its first 'directory-signature '
        line (space included).

        For more information see Section 3.1.

comment:5 Changed 10 months ago by teor

Status: newneeds_revision

It turns out that this information is already in directory-signature at the end of 3.4.1:
https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt#n2441

So I would like to to make these changes:

  • update consensus-digest, additional-digest, and vote-digest to point to 3.4.1, with a short note
  • update extra-info-digest to point to 2.1.2 (which points to 1.3)
  • update 1.3 to point to 3.4.1 for digests

Alternately, we could move the info about digests from 3.4.1 to 1.3, with a short note in 3.4.1.
Then we'd have to update the consensus diff format to point to 1.3 instead of 3.4.1:
https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt#n3484

Do you think we should say "network-status-version" instead of "the beginning of the document"?

comment:6 Changed 10 months ago by atagar

It turns out that this information is already in directory-signature at the end of 3.4.1

Good eye!

Alternately, we could move the info about digests from 3.4.1 to 1.3

Personally I'm a more inclined for that because section 3.4.1 is huge and as such not a great spot for other fields to reference.

Do you think we should say "network-status-version" instead of "the beginning of the document"?

Up to you. It's synonymous in this case because 'network-status-version' is defined as being the start of the document, so that cannot change without violating backward compatibility.

comment:7 Changed 10 months ago by teor

Owner: set to teor
Status: needs_revisionassigned

I will do this at some point.

comment:8 Changed 8 months ago by nickm

Keywords: postfreeze-ok added

Mark some tickets as postfreeze-ok, to indicate that I think they are okay to accept in 0.4.0 post-freeze. Does not indicate that they are all necessary to do postfreeze.

comment:9 Changed 7 months ago by nickm

Keywords: 040-deferred-20190220 added
Milestone: Tor: 0.4.0.x-finalTor: unspecified

Deferring 51 tickets from 0.4.0.x-final. Tagging them with 040-deferred-20190220 for visibility. These are the tickets that did not get 040-must, 040-can, or tor-ci.

comment:10 Changed 7 months ago by teor

Keywords: fast-fix added

comment:11 Changed 2 months ago by teor

Keywords: teor-backlog added
Owner: teor deleted

These items are not on our roadmap, and they do not have a sponsor.
But I might do them some day.

Note: See TracTickets for help on using tickets.