disparate duplicate subproto handling in protover

[cyberpunks]

protover.c treats Link=1 Link=1 and Link=1 identically, allowing duplicate entries without complaint, though it does explicitly check for duplicates to avoid double-counting it as two votes for the same version.

protover.c also treats Link=1 Link=2 and Link=1-2 the same, while the rust implementation of protover treats Link=1 Link=2 as if it were Link=2.

[cyberpunks]

The C code already has to check for duplicates to prevent double-voting. Could both C and Rust be changed to consider it an error to have duplicate subprotocol entries?

[teor]

No, duplicate subprotocol entries are allowed by the spec:
​https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt#n780

The rust code is wrong, and should be fixed.

[cyberpunks]

Replying to teor:

No, duplicate subprotocol entries are allowed by the spec:

That spec seems ambiguous and could be clarified in either direction.

If it's clarified to forbid duplicates--it already says the entries should be sorted alphabetically--it would become simpler to also verify that there aren't any overlapping ranges, which the spec says shouldn't happen but the C implementation doesn't check for right now.

Also, make test-network-all passes on branch protodupes1, at least.

[teor]

Replying to cyberpunks:

Replying to teor:

No, duplicate subprotocol entries are allowed by the spec:

That spec seems ambiguous and could be clarified in either direction.

If it's clarified to forbid duplicates--it already says the entries should be sorted alphabetically--it would become simpler to also verify that there aren't any overlapping ranges, which the spec says shouldn't happen but the C implementation doesn't check for right now.

"must" is an absolute requirement, but "should" is a recommendation:

SHOULD This word, or the adjective "RECOMMENDED", mean that there
may exist valid reasons in particular circumstances to ignore a
particular item, but the full implications must be understood and
carefully weighed before choosing a different course.

​https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt#n19
​https://tools.ietf.org/html/rfc2119

Therefore, Tor's parser can tolerate duplicates, unsorted ranges, and overlapping ranges. I think that it's worth accepting them, to support a broad range of relay implementations. As a trivial example, it's worth accepting both "Foo=1,2" and "Foo=1-2".

But all directory authority implementations MUST generate identical consensuses:

​https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt#n1614

Therefore:

• all directory authority parser implementations MUST interpret edge cases as the same protocol versions, and
• all directory authority serialisation implementations MUST generate exactly the same text for the same protocol versions.

So I think we should fix the Rust to match the C.

We can also open another ticket to clarify the spec, but I'm not sure exactly what changes to make. We don't try to specify the exact format of a consensus in the spec - we only provide enough detail to parse a consensus.

Also, make test-network-all passes on branch protodupes1, at least.

Unfortunately, those tests don't test all the edge cases. And apparently neither do our unit tests. So we should add more unit tests.

[teor]

Since my last comment raises some spec issues, I'm cc'ing nickm.

[nickm]

Marking a number of 0.3.5 tickets as possible, maybe even a good idea, for later. Possibly backportable, some of them. But not currently things to do as part of 0.3.5 stabilization.

[teor]

These open, non-merge_ready tickets can not get backported to 0.3.3, because 0.3.3 is now unsupported.

[teor]

Hmm, I guess they should still get 033-backport-unreached