Opened 7 years ago

Closed 2 years ago

#7059 closed defect (fixed)

TorControl: Unrecognized (node) key errors

Reported by: mr-4 Owned by: aagbsn
Priority: Medium Milestone: Tor: unspecified
Component: Core Tor/Tor Version: Tor: 0.2.4.3-alpha
Severity: Normal Keywords: tor-client, tor-control
Cc: Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

OK, I am not 100% sure whether this is a new bug, though I thought to report it anyway, just in case.

Some background: Alongside tor, sometimes I use event handler, which reports all tor events I am interested in (mainly to create and close streams and also to display the full path of those streams, together with their IP address and country code).

All this is java based (I will try to attach the java code - 3 files in total - at the end of this report).

Up until I upgraded to 0.2.4.3, everything was OK, but since the upgrade I started getting a lot of errors like these:

Error from Tor process: dash.four.tor.torctl.TorControlError: Error reply: Unrecognized key "desc/id/$97B867190D2A0387F3259EA65664A90884A8BDC4" [null]
Error from Tor process: dash.four.tor.torctl.TorControlError: Error reply: Unrecognized key "desc/id/$A1130635A0CDA6F60C276FBF6994EFBD4ECADAB1" [null]
Error from Tor process: dash.four.tor.torctl.TorControlError: Error reply: Unrecognized key "desc/id/$8894CAA82AA1DC8B72B679ACB54FFED561737B68" [null]
Error from Tor process: dash.four.tor.torctl.TorControlError: Error reply: Unrecognized key "desc/id/$F4EAE475A7117DD53EF631F031F000966972A2F5" [null]
Error from Tor process: dash.four.tor.torctl.TorControlError: Error reply: Unrecognized key "desc/id/$6330CCF8FEED2EF9B12FCF6688E2577C65522BA4" [null]
Exception in thread "Thread-0" dash.four.tor.torctl.TorControlError: Error reply: Unrecognized key "desc/id/$B749E078AD23AFA33D5BD2FF1AD15EE6409E3D04"

These are caused by the DebuggingEventHandler, and con.getInfo("desc/id/" + node) in particular.

Am I missing something or has the API changed in 0.2.4.3? the same code used to work, with very small exceptions in 0.2.3.x.

Let me know if you need any more info from me.

Child Tickets

Attachments (1)

torctl.tar.gz (3.2 KB) - added by mr-4 7 years ago.
My torctl event handler (java) code

Download all attachments as: .zip

Change History (32)

Changed 7 years ago by mr-4

Attachment: torctl.tar.gz added

My torctl event handler (java) code

comment:1 Changed 7 years ago by mr-4

Component: TorctlTor

comment:2 Changed 7 years ago by nickm

Hm. Could it be that you're using the microdescriptor code, and you aren't downloading descriptors at all?

comment:3 Changed 7 years ago by mr-4

Possibly, though looking at my /var/lib/tor directory I do have cached-descriptors and cached-descriptors.new, though they are very small and do not contain any information on these nodes.

Is there a way to map these nodes in my code as, evidently, the "old" method (con.getInfo) doesn't seem to work with microdescriptors?

comment:4 Changed 7 years ago by mr-4

Apologies, forgot to add - it may not seem obvious, but all methods/classes/definitions in dash.four.tor.torctl.* are in fact a layer above the "standard" APIs defined in net.freehaven.tor.control as originally created by you, Nick, and Roger Dingledine way back in 2007, I think. I just enhanced/modified their functionality a bit to suit my particular environment, using control-spec.txt as guidance.

comment:5 Changed 7 years ago by nickm

Keywords: tor-client controller added
Milestone: Tor: unspecified

getInfo is probably calling the GETINFO command of the controller API. (I haven't looked at this code in a few years, so I could be wrong.) If that's right , you should be able to use the GETINFO targets starting with "md/" to get microdescriptor information. See control-spec.txt for more information on that.

comment:6 in reply to:  5 Changed 7 years ago by mr-4

Replying to nickm:

getInfo is probably calling the GETINFO command of the controller API. (I haven't looked at this code in a few years, so I could be wrong.) If that's right , you should be able to use the GETINFO targets starting with "md/" to get microdescriptor information. See control-spec.txt for more information on that.

Even if I get microdescriptor's information (yes, I am able to, thanks Nick!), that is no good as most of the time when I execute GETINFO (md/id/node) I get this from the microdescriptors:

onion-key

-----BEGIN RSA PUBLIC KEY-----

<RSA_public_key_encoded_in_64base>

-----END RSA PUBLIC KEY-----

That "information" isn't enough as it doesn't give me the IP address or the OR port used by that node. It is very rare that microdescriptor information contains the following lines, in addition to the RSA public key:
a <ip>:<port>
family <node_hash>*

The above is not guaranteed, however (indeed, it is the exception, rather than the rule), so using md/id/node isn't good. Any other ideas?

comment:7 Changed 7 years ago by mr-4

Just to add: tried "GETINFO ns/id/<node_id>", but get "551 Data not decodeable as hex".

Don't know what that means! If it is related to the node id, it is the "standard" node ID format used everywhere - '$' followed by 40 hex characters. I also tried without the '$' character, but no joy there either.

comment:8 Changed 7 years ago by nickm

Status: newneeds_information

It's expected that microdescriptors have less information that server descriptors. That's the point. They omit the stuff that's redundant with the consensus networkstatus document. You need to look at the ns/id/* field too.

I just tried a couple of example GETINFO queries, and found them to work okay:

GETINFO ns/id/ff8845046db449da989b025aea1d10e19e709166
250+ns/id/ff8845046db449da989b025aea1d10e19e709166=
r godzilla /4hFBG20SdqYmwJa6h0Q4Z5wkWY 6Ua1mnT2Fwm1d0RaKbFQPeycP+s 2012-11-02 01:39:04 83.227.52.129 9001 0
s Fast Named Running Valid
w Bandwidth=2
.
250 OK
GETINFO md/id/ff8845046db449da989b025aea1d10e19e709166
250+md/id/ff8845046db449da989b025aea1d10e19e709166=
onion-key
-----BEGIN RSA PUBLIC KEY-----
MIGJAoGBALeYJbHGyPSJ1vMEdEs5ygA4YykBxq66BdvodLNWp9vXBSuCx211eGNj
bUeJxAHm8b3XugJXSwuc9IwQwrctIB/0qotrG4BM9mdd8d6UPr1CycgNuThwtozg
f5L1LNMQQkcuITXANkEp3TWDPeucrsevpSJ0hQQaXqxEt+kdt3CJAgMBAAE=
-----END RSA PUBLIC KEY-----
family $84CA95A4D9E824C6F9662711B887433525760F72
p accept 443,706,993,995
.

Can you be more specific about what isn't working for you?

comment:9 in reply to:  8 ; Changed 7 years ago by mr-4

Replying to nickm:

Can you be more specific about what isn't working for you?

Thanks Nick. I figured that I had to use ns/id/*, but when I try "GETINFO ns/id/<node_id>" (from the java code, admittedly), I get "551 Data not decodeable as hex" error as I already pointed out in my previous response.

I tried both $XXX (where 'X' is the hash id - without spaces - of the node in question), as well as XXX, but didn't get much luck. Am I doing something wrong?

Would you like me to get you more information (I am able to test the entire sequence of commands/responses received as I am debugging this remotely from my Eclipse Java GUI)? If so, let me know and I'll do my best.

comment:10 in reply to:  9 Changed 7 years ago by nickm

Replying to mr-4:

Replying to nickm:

Can you be more specific about what isn't working for you?

Thanks Nick. I figured that I had to use ns/id/*, but when I try "GETINFO ns/id/<node_id>" (from the java code, admittedly), I get "551 Data not decodeable as hex" error as I already pointed out in my previous response.

I tried both $XXX (where 'X' is the hash id - without spaces - of the node in question), as well as XXX, but didn't get much luck. Am I doing something wrong?

What I need here is the actual bytes getting sent to the Tor control port for which the "Data not decodeable as hex" response is returned. As you can see above, "GETINFO ns/id/ff8845046db449da989b025aea1d10e19e709166" worked fine for me, so I need to see what your code is sending instead to understand what is going on.

comment:11 Changed 7 years ago by mr-4

OK, I finally got this working. There are a few, what I'd consider, inconsistencies with node/function definitions as described below though:

  1. desc/id/node seems to only accept the "$XXX" syntax as node ID, where "X" is the hash id of the node in question - in *uppercase*.
  1. ns/id/node seems to *only* accept the "xxx" syntax as node ID, where "x" is the hash id of the node in question - in *lowercase*. Please note that the "$" is not accepted, hence the mysterious "Data not decodeable as hex" error I was getting previously if I include "$".
  1. ns/id/node is *not* universal! In other words, it doesn't seem to recognise nodes which do not use microdescriptors. I am getting "Unrecognised Key" errors when I use this command for nodes, which are defined in my "cached-descriptors{.new}" files.

Apart from the above inconsistencies, when the right format is applied for the right commands as described above, everything seems to work.

comment:12 Changed 7 years ago by mr-4

Sorry, forgot to add something which should be pretty obvious:

  1. desc/id/node only works on nodes which do not have microdescriptors (exactly the opposite of ns/id/node), so what I am currently doing is trying "desc/id/node" first, and if that fails, I then revert to "ns/id/node" - using the "appropriate" syntax for both commands.

comment:13 Changed 7 years ago by nickm

Milestone: Tor: unspecifiedTor: 0.2.4.x-final
Status: needs_informationnew

Ugh; the $ should never be mandatory, and case should never get checked. That's probably a bug.

ns/id/node should cover everything that appears *in the consensus directory*, I think. There should probably be alternate forms of it that match the complete set of nodes more completely though.

comment:14 Changed 7 years ago by nickm

Status: newneeds_review

Also, your description above is possibly inaccurate? I just tried both upper and lowercase for ns/id/* and md/id/*, and both worked. The only thing I can see that doesn't work is providing the $ to ns/id/* . If you find that there really is a difference between upper and lowercase, could you please attach a command log so I can try to reproduce it?

There's a fix for the $ thing in branch "bug7059" of my public repository; please test and review if possible?

comment:15 in reply to:  14 Changed 7 years ago by mr-4

Replying to nickm:

Ugh; the $ should never be mandatory, and case should never get checked. That's probably a bug.

Would you keep this report as a reference or would you like me to submit a separate one?

ns/id/node should cover everything that appears *in the consensus directory*, I think. There should probably be alternate forms of it that match the complete set of nodes more completely though.

The most logical thing, at least to me anyway, would be if ns/id/node is made "universal" so that it would pick and display information about *any* node, regardless of whether it uses microdescriptors or not.

Same goes for desc/id/node as well - it should probably be made to work with all nodes, not just the ones which do not have microdescriptors.

Replying to nickm:

Also, your description above is possibly inaccurate? I just tried both upper and lowercase for ns/id/* and md/id/*, and both worked. The only thing I can see that doesn't work is providing the $ to ns/id/* . If you find that there really is a difference between upper and lowercase, could you please attach a command log so I can try to reproduce it?

Apologies, you are quite right - there is no difference between lower and upper case - just tested this again.

There's a fix for the $ thing in branch "bug7059" of my public repository; please test and review if possible?

What's the url for that? I am willing to give it a go.

comment:17 Changed 7 years ago by mr-4

Apologies for this late reply (I was very busy last week), but I just had the chance to test your patch against the newly-released .6-alpha and it is sound - I don't have to cut the '$' from the node anymore.

Can't wait for the rest of the issues I reported above - if you need further help with the testing just let me know Nick.

comment:18 Changed 7 years ago by nickm

Merged the fix for this one.

Of the issues you reported above, several turned out to be false alarms. What do you think are the remaining issues?

comment:19 in reply to:  18 ; Changed 7 years ago by mr-4

Replying to nickm:

What do you think are the remaining issues?

Issues 3 and 4 as reported above: "ns/id/node" seems to only be working for "microdescriptor" nodes whereas desc/id/node is exactly the opposite. Should these be "universal" and work on all nodes regardless, particularly "ns/id/node"?

comment:20 in reply to:  19 ; Changed 7 years ago by nickm

Status: needs_reviewnew

Replying to mr-4:

Replying to nickm:

What do you think are the remaining issues?

Issues 3 and 4 as reported above: "ns/id/node" seems to only be working for "microdescriptor" nodes whereas desc/id/node is exactly the opposite. Should these be "universal" and work on all nodes regardless, particularly "ns/id/node"?

As specified, no. "ns/*" is for stuff in the consensus; "desc/" is for stuff for which you have router descriptors; "md" is for microdescriptors in particular.

It might be neat to have a "node" item that merged all the info we knew about a router and returned it in a reasonable format, but that feels like another ticket.

comment:21 in reply to:  20 Changed 7 years ago by mr-4

Replying to nickm:

As specified, no. "ns/*" is for stuff in the consensus; "desc/" is for stuff for which you have router descriptors; "md" is for microdescriptors in particular.

I can understand the "md" and "desc" part of the argument - it makes sense, but not the "ns" bit - the spec in particular doesn't make any distinction on whether microdescriptor information should only be included.

If anything, it refers to the v2 format, which includes "router" descriptor info (which in itself includes IP address, OR port etc), so I think this should be properly enforced. From the control-spec.txt document:

"...the latest router status info (v2 directory style) for a given OR. Router status info is as given in dir-spec.txt, and reflects the current beliefs of this Tor about the router in question."

It might be neat to have a "node" item that merged all the info we knew about a router and returned it in a reasonable format, but that feels like another ticket.

To me, that's the function of "ns", but if that is not the case I can submit another ticket for this - no problem.

comment:22 Changed 7 years ago by mr-4

Update: I've just got the following error message: Unrecognized key "ns/id/7F900F192A264136E41FCA4092CDA4ED9E75009F".

This node has been returned as not recognised by both desc/id/... as well as ns/id/...

As far as I can see, this node is not new - it has been in existence for quite a while (7F900F192A264136E41FCA4092CDA4ED9E75009F is TheBlackKnight and is SE based), so I don't know why I am getting this error. Let me know if you need anything from me,

comment:23 Changed 7 years ago by nickm

Milestone: Tor: 0.2.4.x-finalTor: 0.2.5.x-final

Fixing up all the ns/* stuff is doing to take design and implementation work -- so it's 0.2.5 or later unless a patch shows up in the next 4 days :/

comment:24 Changed 7 years ago by mr-4

Just for reference - I've submitted another report a while ago just for this issue - see bug #7646.

comment:25 Changed 6 years ago by nickm

Milestone: Tor: 0.2.5.x-finalTor: 0.2.???

comment:26 Changed 3 years ago by teor

Milestone: Tor: 0.2.???Tor: 0.3.???

Milestone renamed

comment:27 Changed 3 years ago by nickm

Keywords: tor-03-unspecified-201612 added
Milestone: Tor: 0.3.???Tor: unspecified

Finally admitting that 0.3.??? was a euphemism for Tor: unspecified all along.

comment:28 Changed 2 years ago by nickm

Keywords: tor-03-unspecified-201612 removed

Remove an old triaging keyword.

comment:29 Changed 2 years ago by dgoulet

Unify controller keyword to "tor-control".

comment:30 Changed 2 years ago by dgoulet

Keywords: tor-control added; controller removed

Unify "controller" keyword to "tor-control".

comment:31 Changed 2 years ago by nickm

Resolution: fixed
Severity: Normal
Status: newclosed

Calling this fixed because of the number of things already fixed in it. ns/id/* is only working for the things in the networkstatus, and that's as designed fwict. If I missed something, please open a new ticket with a reference to this one?

Note: See TracTickets for help on using tickets.