Opened 4 weeks ago

Closed 4 weeks ago

#25046 closed defect (fixed)

reveal_count incorrect in stem

Reported by: tom Owned by: atagar
Priority: Medium Milestone:
Component: Core Tor/Stem Version:
Severity: Normal Keywords:
Cc: Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

I am stem master, commit b2c8810dc94b0e2e62c0e482158de3d33504a709 Sat Jan 20 17:53:41 2018 -0800

» python
Python 2.7.10 (default, Jul 15 2017, 17:16:57)
[GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.31)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import stem.descriptor.remote
>>>
>>> downloader = stem.descriptor.remote.DescriptorDownloader(
...     timeout = 60,
...     fall_back_to_authority = False,
...     document_handler = stem.descriptor.DocumentHandler.DOCUMENT,
... )
>>> query = downloader.query('/tor/status-vote/current/consensus', default_params = False)
>>> d = query.run()
>>> d
[<stem.descriptor.networkstatus.NetworkStatusDocumentV3 object at 0x10aa8ea10>]
>>> c = d[0]
>>> c.shared_randomness_current_reveal_count
>>> c.shared_randomness_previous_reveal_count
>>> c.shared_randomness_current_value
u'vLbCqI2m02UXzOUYtSMPWxC/QUSXSw0mrOBNyBCGtcE='
>>> c.shared_randomness_previous_value
u'2xFM9tuS7c84CfE2OIURwJjh/XYop/tQoQD70DBBJNA='

Previously these values... had values. But now it appears they're not being populated? The documentation is a little fuzzy, but can you confirm they are intentionally not being populated anymore, even though they appear in the consensus?

shared-rand-previous-value 9 2xFM9tuS7c84CfE2OIURwJjh/XYop/tQoQD70DBBJNA=
shared-rand-current-value 9 vLbCqI2m02UXzOUYtSMPWxC/QUSXSw0mrOBNyBCGtcE=

Child Tickets

Change History (12)

comment:1 Changed 4 weeks ago by atagar

Hi Tom, agreed it's a bit confusing. Is this the bit of the docs you were looking for?

.. versionchanged:: 1.6.0
   The shared randomness attributes were misdocumented in the tor spec and as
   such never set. They're now an attribute of **directory_authorities**.

comment:2 Changed 4 weeks ago by tom

I had seen that, but in that section (DirectoryAuthorities) it says it's part of the vote. So the data isn't available anywhere in the Consensus object?

comment:3 Changed 4 weeks ago by tom

The plot thickens. *Somehow* consensus-health is getting a reveal count from the consensus, because it's outputting it on the page. But I can't reproduce it using the command line!

comment:4 Changed 4 weeks ago by atagar

So the data isn't available anywhere in the Consensus object?

Lets step back a bit. Please curl the url you're having stem fetch and provide here both the url and the lines of the document it gave you that you want but cannot find in the consensus object you're getting from stem.

Stem is just a parser for the dir-spec. This means there can be four things going on here...

  1. The data you want isn't in the consensus, in which case this should be an ask for the field or clarification from the core tor folks.
  1. The data is present in the consensus and Stem has it. If you provide the above I'll be able to look.
  1. The data is present in the consensus but Stem doesn't have it and the spec says it shouldn't be there. In this case there's a spec bug.
  1. The data is present in the consensus but Stem doesn't have it despite the spec saying it should be there. In this case there's a stem bug.
Last edited 4 weeks ago by atagar (previous) (diff)

comment:5 Changed 4 weeks ago by atagar

Oops. Actually, on reflection the original post had everything I need. Fetching it from moria1 and taking a peek.

comment:6 Changed 4 weeks ago by atagar

% curl 128.31.0.39:9131/tor/status-vote/current/consensus > moria_consensus

Within the conesus header it had the lines you wanted...

shared-rand-previous-value 9 2xFM9tuS7c84CfE2OIURwJjh/XYop/tQoQD70DBBJNA=
shared-rand-current-value 9 vLbCqI2m02UXzOUYtSMPWxC/QUSXSw0mrOBNyBCGtcE=

... then when I read it with Stem...

% python
>>> import stem.descriptor
>>> import stem.descriptor.remote
>>> consensus = stem.descriptor.remote.get_consensus(document_handler = stem.descriptor.DocumentHandler.DOCUMENT).run()[0]
>>> consensus
<stem.descriptor.networkstatus.NetworkStatusDocumentV3 object at 0x8d0b64c>
>>> consensus.shared_randomness_current_value
u'vLbCqI2m02UXzOUYtSMPWxC/QUSXSw0mrOBNyBCGtcE='
>>> consensus.shared_randomness_current_reveal_count
9

Turns out that a year ago we had a similar discussion about this (#21102, which resulted in commit d713b29). :P

One bug I *am* spotting is that the 'reveal_count' attributes are undocumented. Also, if you fetch those *before* the 'current_value' then you get None. This is because when lazily loading stem is missing the 'reveal_count' handler. Fixing...

Last edited 4 weeks ago by atagar (previous) (diff)

comment:7 Changed 4 weeks ago by atagar

Hi Tom, pushed a fix. How does this look to you?

https://gitweb.torproject.org/stem.git/commit/?id=76e8eb5

comment:8 Changed 4 weeks ago by tom

Looks good! Thanks!

comment:9 Changed 4 weeks ago by atagar

Resolution: fixed
Status: newclosed

Great! Thanks for the catch. :P

comment:10 Changed 4 weeks ago by teor

Resolution: fixed
Status: closedreopened

Something weird happened with the pydoc recently, and I'm now seeing:

HEADER_PARSER_FOR_LINE = {'valid-after': <function _parse at 0x7fba0426e488>, 'server-versions': <function _parse at 0x7fba0426e668>, 'shared-rand-previous-value': <function _parse_shared_rand_previous_value at 0x7fba0426e398>, 'network-status-version': <function _parse_header_network_status_version_line at 0x7fba0426fc08>, 'consensus-method': <function _parse_header_consensus_method_line at 0x7fba0426fed8>, 'client-versions': <function _parse at 0x7fba0426e5f0>, 'known-flags': <function _parse at 0x7fba0426e758>, 'voting-delay': <function _parse_header_voting_delay_line at 0x7fba0426ff50>, 'shared-rand-current-value': <function _parse_shared_rand_current_value at 0x7fba0426e410>, 'package': <function _parse_package_line at 0x7fba0426e2a8>, 'params': <function _parse_header_parameters_line at 0x7fba0426e140>, 'recommended-relay-protocols': <function _parse at 0x7fba0426e9b0>, 'fresh-until': <function _parse at 0x7fba0426e500>, 'valid-until': <function _parse at 0x7fba0426e578>, 'published': <function _parse at 0x7fba0426fa28>, 'required-client-protocols': <function _parse at 0x7fba0426ea28>, 'vote-status': <function _parse_header_vote_status_line at 0x7fba0426fde8>, 'required-relay-protocols': <function _parse at 0x7fba0426eaa0>, 'flag-thresholds': <function _parse_header_flag_thresholds_line at 0x7fba0426e0c8>, 'consensus-methods': <function _parse_header_consensus_methods_line at 0x7fba0426fe60>, 'recommended-client-protocols': <function _parse at 0x7fba0426e938>}

FOOTER_PARSER_FOR_LINE = {'directory-signature': <function _parse_footer_directory_signature_line at 0x7fba0426e230>, 'bandwidth-weights': <function _parse at 0x7fba0426e848>, 'directory-footer': <function _parse_directory_footer_line at 0x7fba0426e1b8>}

classmethod content(attr=None, exclude=(), sign=False, authorities=None, routers=None)[source]

classmethod create(attr=None, exclude=(), validate=True, sign=False, authorities=None, routers=None)

On https://stem.torproject.org/api/descriptor/networkstatus.html#stem.descriptor.networkstatus.NetworkStatusDocumentV3.content

comment:11 Changed 4 weeks ago by atagar

Thanks Tim, nice catch. I expect that's a long standing documentation bug (not spotting a way that it could be related to this particular fix). Good thing to address though none the less.

Moved to #25075.

comment:12 Changed 4 weeks ago by atagar

Resolution: fixed
Status: reopenedclosed
Note: See TracTickets for help on using tickets.