Opened 5 years ago

Closed 4 years ago

#7953 closed enhancement (fixed)

Control port method to get v3 directory informaion

Reported by: atagar Owned by:
Priority: High Milestone: Tor: 0.2.5.x-final
Component: Core Tor/Tor Version:
Severity: Keywords: tor-client controller needs-proposal
Cc: Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

Hi Nick. Do we have v3 counterparts for the 'GETINFO ns/*' methods? From some quick experimentation...

  • 'GETINFO dir/status-vote/current/consensus' seems to provide the v3 network status document (same as cached-consensus). Is this the suggested method for reading the full v3 document?
  • Based on their names I suspect that 'GETINFO dir/status/all' and 'GETINFO dir/status/fp/*' should also provide network status information but they give me empty responses. I'm just guessing at their purpose since the control-spec doesn't actually say what any of the 'dir/*' options do.

Is there a method to query the v3 router status entry for a particular relay (like there is for v2)?

Thanks! -Damian

20:38 < atagar> armadev: Speaking of nice things to have, are there v3 counterparts for the 
                'GETINFO ns/*' methods? It looks like 'GETINFO dir/status-vote/current/consensus' 
                works though I'm not finding anything that lets you query for a single relay.
20:39 < atagar> I suspect that 'dir/status/fp/*' and 'dir/status/all' are for this purpose, but 
                they both give an empty response.
20:41 < armadev> atagar: i think the trouble is that each of these things is designed to query for 
                 a particular dir protocol version, and they all look like synonyms
20:41 < armadev> it's only going to get worse if we keep up the trend
20:43 < atagar> Agreed. There's "GETINFO network-status" which gives v1 network status information, 
                "GETINFO ns/*" for v2 status information, and "GETINFO 
                dir/status-vote/current/consensus" for v3 (?). Definitely confusing. :)
20:44 < atagar> Though at this point I'm just trying to figure out if individual relays can be 
                queried with v3 at all.
20:45 < armadev> atagar: it's very possible nobody implemented that yet.
20:46 < atagar> gotcha, thanks
20:46 < armadev> i suggest a new master way to query, with a defined algorithm for how it will 
                 always answer what you expected.
20:46 < armadev> (though the trouble is that 'what you expected' may well change in dir v4. 
                 whatever that will be.)
20:48 < atagar> The method of querying hasn't changed for a long while (it's always either 'all', 
                'by fignerprint', or 'by nickname'). Personally I think we should mark the present 
                GETINFO network status options as being deprecated and go with a scheme with the 
                version in it.
20:48 < armadev> that sounds good too
20:48 < atagar> Maybe something like 'GETINFO dir/ns/v3/fp/*'

Child Tickets

Change History (18)

comment:1 Changed 5 years ago by nickm

Keywords: tor-client controller added
Milestone: Tor: 0.2.5.x-final
Type: defectenhancement

The dir/status stuff is v2 only; a new interface to get v3 stuff would be welcome. I'm not going to get to it before the small-features deadline for 0.2.4 though.

Another wrinkle is that the ns/* stuff is still in v2 format, because we don't want to break compatibility. Maybe that needs a new interface too? Or is there enough info there to work with?

comment:2 Changed 5 years ago by atagar

Another wrinkle is that the ns/* stuff is still in v2 format, because we don't want to break compatibility. Maybe that needs a new interface too? Or is there enough info there to work with?

That's why I suggested in the irc backlog deprecating the current ns/* options in favor of a new set that includes the version.

'GETINFO dir/status-vote/current/consensus' presently seems to work. Is that the suggested method for getting the v3 consensus? Are the 'GETINFO dir/status/*' options intended to give v3 information but simply broken?

comment:3 in reply to:  2 Changed 5 years ago by nickm

Replying to atagar:

Another wrinkle is that the ns/* stuff is still in v2 format, because we don't want to break compatibility. Maybe that needs a new interface too? Or is there enough info there to work with?

That's why I suggested in the irc backlog deprecating the current ns/* options in favor of a new set that includes the version.

'GETINFO dir/status-vote/current/consensus' presently seems to work. Is that the suggested method for getting the v3 consensus?

Yes.

Are the 'GETINFO dir/status/*' options intended to give v3 information but simply broken?

The original intent (IIRC) was that they should give "the status information as present in the directory." There was no v3 contemplated at the time.

I'm not sure it's right to have dir/status/* return v3 information, in case there's anything out there that expects the v2 format. But it's not right to have it return v3 information in the v2 format, since what it was supposed to do was to include the information as present in the directory.

comment:4 Changed 5 years ago by atagar

Priority: normalmajor

'GETINFO dir/status-vote/current/consensus' presently seems to work. Is that the suggested method for getting the v3 consensus?

Yes.

Now this doesn't seem to work...

GETINFO version
version=0.2.4.10-alpha-dev (git-8be6058d8f31e578)
OK

GETINFO dir/status-vote/current/consensus
Unrecognized key "dir/status-vote/current/consensus"

This and #8323 top my 'stem wishlist' at present since between them controllers can't get modern descriptor information (microdescriptors can only be fetched individually and v3 directory information is simply unavailable).

comment:5 Changed 5 years ago by nickm

Milestone: Tor: 0.2.5.x-finalTor: 0.2.4.x-final

This part feels like a bug, if in fact the microdesc isn't exposed at all.

comment:6 Changed 5 years ago by atagar

if in fact the microdesc isn't exposed at all.

Maybe I was unclear. v3 directory information is unavailable, but micriodescriptors *are* available on an individual bases (there's just no method for querying them in bulk).

comment:7 Changed 5 years ago by nickm

Argh, I meant to type, "This part feels like a bug, if in fact the microdesc consensus isn't exposed at all"

comment:8 Changed 5 years ago by nickm

Hm. It looks like right now dir/status-vote/current/consensus will only return a "ns" (non-microdesc) consensus. That's changeable, I guess, though I wonder if it's more correct to break clients that would hate a microdesc consensus, or more correct to break clients that would hate getting no answer for dir/status-vote/current/consensus.

By the logic of the other dir/status* stuff, the right GETINFO should probably be something more like the URL for fetching micordescriptor consensus, I guess?

comment:9 Changed 5 years ago by atagar

By the logic of the other dir/status* stuff, the right GETINFO should probably be something more like the URL for fetching micordescriptor consensus, I guess?

Guess so, though honestly the 'dir/*' options still puzzle me. They're over seven years old and have a cryptic enough description that I'm not sure what they're even *supposed* to do. From my read of the commits that added them they're for raw descriptor access? Maybe an addition for some long-forgotten research project?

My vote would be to mark the 'dir/*' options as being deprecated (does anyone use them?) and instead have this ticket be for v3 counterparts of the 'ns/*' options. I suppose we'll need to distinguish microdescriptor flavored documents so maybe something like...

  • GETINFO ns/v3/id/<OR identity>
  • GETINFO ns/v3-md/id/<OR identity>
  • ... etc...

comment:10 Changed 5 years ago by nickm

If what you really want is something else, shall we close this ticket and open a new one for 0.2.5?

And before we do that, is there something more bugfixish than featurey that would still be useful to do in 0.2.4?

comment:11 Changed 5 years ago by atagar

If what you really want is something else, shall we close this ticket and open a new one for 0.2.5?

Why close this ticket? This has always been about having a method to query v3 directory information. Feel free though to mark it as being for 0.2.5.

comment:12 Changed 5 years ago by nickm

Keywords: needs-proposal added
Milestone: Tor: 0.2.4.x-finalTor: 0.2.5.x-final

Okay. Sounds like there isn't a non-feature-like fix here.

comment:13 Changed 4 years ago by atagar

09:15 < atagar> I'm probably missing something obvious but the 'GETINFO ns/*' options are documented as providing v2 style directory data. However, it certainly looks to provide v3 
                router status entries instead (note the 'p' and 'w' lines): http://pastebin.com/raw.php?i=rrhXZ4ZF
09:16 < atagar> Not that I'm complaining - v2 entries aren't nearly as useful as their v3 counterparts but I am a little puzzled. :)
09:17 < atagar> btw, this might mean that https://trac.torproject.org/7953 can be 'fixed' with a simple spec change
09:17 < atagar> though it would be nice if it said what tor versions provide what kind of network status data

comment:14 Changed 4 years ago by atagar

God I'm awful at making sense of C. Ok, think I understand what's up...

  • Commit 3ad6dc0e switched us from v2 to v3 network status documents.
  • The getinfo_helper_networkstatus in networkstatus.c operates on routerstatus_t. That struct (in or.h) is version agnostic. It's used for both v2 and v3 network status documents.
  • Prior to commit 3ad6dc0e getinfo_helper_networkstatus operated on routerstatus_list, which were v2 router status entries.
  • Commit 3ad6dc0e removed routerstatus_list in favour of current_consensus->routerstatus_list. The current_consensus is a networkstatus_vote_t, which is a v3 network status document.

TL;DR. When we switch tor to v3 network status documents in 2007 the control protocol switched as well. No one noticed because v3 router status entries are a superset of v2. We should probably simply pretend that GETINFO always provided v3 responses and change the spec.

comment:15 Changed 4 years ago by nickm

Thanks for the code archeology!

The reason I thought that we were still on v2 here is that routerstatus_format_entry provides a NS_CONTROL_PORT format, which I thought was locked onto v2. That controls the format of what we output. The stuff you're describing here is about _where we get_ the routerstatus_t entries, which is indeed from the v3 consensus.

Did we really update the NS_CONTROL_PORT format to output the v3 networkstatus format at some point? I guess we must have, if that's what you're seeing...

comment:16 Changed 4 years ago by atagar

Hi Nick. Thanks for the pointer!

  • NS_CONTROL_PORT didn't exist back in the dark ages of directory v2 (3ad6dc0e). Nowadays it's used in routerstatus_format_entry() of dirserv.c.
  • In 3ad6dc0e routerstatus_format_entry() did not support multiple formats. It always wrote 'r', 's', and 'v' lines. Those are what a v2 router status entry consists of.
  • In master for the NS_CONTROL_PORT format it outputs 'r', 'a', 's', 'v', 'w', and 'p'. This matches with the definition of a v3 router status entry minus 'm' entries (which only appear in votes).
  • These attributes have been added over time as the v3 format evolved. My guess is that no one knew that 'NS_CONTROL_PORT' was meant to be the v2 format. If we wanted that to be the case then we probably should have included the version in the enumeration (like NS_V2).

Cheers! -Damian

comment:17 Changed 4 years ago by atagar

Status: newneeds_review

Hi Nick. This is probably at a point where we can move forward with updating the spec. Pushed a tweak to the 7953 of my repo...

https://gitweb.torproject.org/user/atagar/torspec.git/commitdiff/d2b7ebb098c5cae9c40aaaf2bbc50efe67e1c91f

I'll switch stem over to providing v3 router status entries afterwards.

Cheers! -Damian

comment:18 Changed 4 years ago by nickm

Resolution: fixed
Status: needs_reviewclosed

Works for me. Merged. Thanks!

Note: See TracTickets for help on using tickets.