Opened 7 months ago

Closed 5 months ago

Last modified 5 months ago

#33675 closed enhancement (fixed)

Search microdescriptor files for relay ed25519 keys

Reported by: teor Owned by: anuradha1904
Priority: Medium Milestone:
Component: Core Tor/Chutney Version:
Severity: Normal Keywords: ipv6, prop311, outreachy-ipv6
Cc: teor, anuradha1904, nickm, ahf Actual Points: 0.2
Parent ID: #33050 Points: 0.5
Reviewer: teor Sponsor: Sponsor55-can

Description (last modified by teor)

Your code in #33428 needs to pass your local "make test-network-all", before you start this ticket.

We need to enable searching for ed25519 keys in relay microdescriptor files.

There are instructions and a draft search pattern here:
https://github.com/torproject/chutney/blob/master/lib/chutney/TorNet.py#L1325

Please open a new pull request for this ticket. Your branch should be based on the final version of #33428.

Before you push new changes to your pull request, your chutney code should pass:

  • "make test-network-all" on tor master
  • "make test-network-all" on tor maint-0.3.5

You can build a tor branch using these commands:

cd tor
git checkout -b <branch>
make

Where <branch> is master or maint-0.3.5

Child Tickets

Change History (37)

comment:1 Changed 7 months ago by teor

Description: modified (diff)

comment:2 Changed 7 months ago by anuradha1904

hey teor, should I do all the changes on the same branch by making commits for each issue solved or rather make different PRs, since all the issue assigned to me are in the same file.

comment:3 Changed 7 months ago by anuradha1904

Last edited 7 months ago by anuradha1904 (previous) (diff)

comment:4 Changed 7 months ago by nickm

Generally it's best to have one issue per PR; that way they can get reviewed and merged independently. (Reviewing two little PRs is usally easier than reviewing one big one.)

That said, if branches have to edit the same part of the file, then there might be merge conflicts. If that's the case, it's probably better to have one branch.

comment:5 Changed 7 months ago by anuradha1904

Thank you nickm. Also, Can you explain me what does "nickname" mean in function getNodeDirInfoStatusPattern? Thank you.

comment:6 Changed 7 months ago by nickm

It's a short string that identifies the node, typically something like "000a" or "006c"

comment:7 Changed 7 months ago by nickm

Err, sorry. That should have said 'something like "test000a" or "test006c"'.

comment:8 Changed 7 months ago by anuradha1904

Great, so for "md", all the code that I have to add is:-

elif md:
         return r'^id ed25519 ' + " "


Am I following the right way, because I checked in the cached-microdescs.new file, and the ed25519 key was just after the string "id ed25519" and there was no nickname like for desc and cons in the file cached-microdesc-consensus and cached-microdescs.new. teor, nickm , can you confirm ? If not, where am I wrong? Kindly guide me and Thank you.

comment:9 in reply to:  8 Changed 7 months ago by teor

Replying to anuradha1904:

Great, so for "md", all the code that I have to add is:-

elif md:
         return r'^id ed25519 ' + " "

Can you explain why you wrote this code?
Why do you think you need to use the nickname?

The code in the ticket description is:

        elif md:
            # Not yet implemented, see #33428
            # r'^id ed25519 " + ed25519_identity (end of line)
            # needs ed25519-identity from #30642
            # or the existing keys/ed25519_master_id_public_key

At https://github.com/torproject/chutney/blob/master/lib/chutney/TorNet.py#L1325

Am I following the right way, because I checked in the cached-microdescs.new file, and the ed25519 key was just after the string "id ed25519" and there was no nickname like for desc and cons in the file cached-microdesc-consensus and cached-microdescs.new. teor, nickm , can you confirm ? If not, where am I wrong? Kindly guide me and Thank you.

Regular expressions are patterns that are matched against each line. You can read a short introduction here:
https://docs.python.org/3/howto/regex.html

Remember how we just loaded ed25519-id in #33428 ?
https://github.com/torproject/chutney/pull/61/files#diff-5fe4ecd0da6560d24d20e8ce90281abfR792

Maybe it could be useful here.

comment:10 Changed 7 months ago by anuradha1904

Hey teor, I mentioned I don't think I need to use nickname because nickname returns value "test000a" or "test006c" which does not follow 'id ed25519' , However I did read the articles you mentioned, and found this code :-

elif md:
        return r'^id ed25519 ' + ed25519-id 

where ed25519-id is the environment variable I set up in my function? Am I following the correct way. Thank you.

comment:11 Changed 7 months ago by teor

That looks great!

But base64 contains "+" characters:
https://docs.python.org/3.5/library/base64.html#base64.b64encode

And "+" is a special character in regular expressions:
https://docs.python.org/3/library/re.html#regular-expression-syntax

So you'll need to use re.escape() on ed25519-id:
https://docs.python.org/3/library/re.html#re.escape

Please write some code, and test it using "make test-network-all".
Send us a pull request when you're done.

If you get stuck, send us:

  • a pull request with your draft code
  • the detailed logs from the ".log" files

And tell us what you've tried already.

comment:12 Changed 7 months ago by anuradha1904

Hey teor, I have added the PR , can you kindly check? Thank you very much. https://github.com/torproject/chutney/pull/62

comment:13 Changed 7 months ago by teor

Status: assignedneeds_revision

I have asked for some changes on the PR. And some more information about how you are running the tests.

comment:14 Changed 7 months ago by anuradha1904

teor: I have followed the steps provided by you. Kindly check if I following the wrong steps and suggest me steps that I should take further for improvement. Thank you very much for your help.

comment:15 Changed 7 months ago by teor

Hi, just letting you know that I've seen these changes, and I'll try to review them over the next few days.

comment:16 Changed 7 months ago by anuradha1904

Hey teor, Thank you very much for reviewing the changes.

Last edited 7 months ago by anuradha1904 (previous) (diff)

comment:17 Changed 7 months ago by teor

Status: needs_revisionneeds_review

comment:18 Changed 7 months ago by teor

Reviewer: teorahf

I have spoken with Alex, and he will review this ticket.

comment:19 Changed 7 months ago by anuradha1904

Hey teor, can I ask alex the question regarding this ticket then? Thank you very much.

comment:20 in reply to:  19 Changed 7 months ago by teor

Replying to anuradha1904:

Hey teor, can I ask alex the question regarding this ticket then? Thank you very much.

Please ask your questions on the ticket, and someone will answer them :-)

comment:21 Changed 7 months ago by anuradha1904

hey ahf, can you please review this Pull Request https://github.com/torproject/chutney/pull/62 . Thank you very much.

comment:22 Changed 7 months ago by teor

Reviewer: ahfteor

comment:23 Changed 7 months ago by teor

Actual Points: 0.10.2
Status: needs_reviewneeds_revision

Hi, I have reviewed this PR, and suggested some changes.

comment:24 Changed 7 months ago by teor

I have spoken to anuradha1904 on IRC, and they sent me these chutney logs:

Bootstrap failed: 120 seconds
Node status:
test000a     :  100, done                     , Done
test001a     :  100, done                     , Done
test002r     :  100, done                     , Done
test003r     :  100, done                     , Done
test004r     :  100, done                     , Done
test005c     :  100, done                     , Done
test006c     :  100, done                     , Done
test007h     :  100, done                     , Done
test008h     :  100, done                     , Done
Published dir info:
test000a     :    0, all nodes                , md                            , Not in dir file
test001a     :    0, all nodes                , md                            , Not in dir file
test002r     :    0, all nodes                , md                            , Not in dir file
test003r     :    0, all nodes                , md                            , Not in dir file
test004r     :    0, all nodes                , md                            , Not in dir file

It looks like chutney is using the new md pattern, but it doesn't match any of the files.

comment:25 Changed 7 months ago by teor

Here are the things I need to do when I merge:

  • squash the commits
  • document what "None" means in getNodeDirInfoStatusPattern()

Here are some things that either of us can do:

  • check TorNet.py for any other comments about microdescs not being implemented
  • if md is True, print the values of nickname and ed25519_key in getNodeDirInfoStatusPattern()
  • if we are not getting the right value for ed25519_id, move _setEd25519Id() into LocalNodeController, so we can use it during wait_for_bootstrap()

comment:26 Changed 7 months ago by anuradha1904

Provided below is the log after running the test after making the changes in the code.

test-network.sh: $TOR_DIR not set, trying $PWD
test-network.sh: $CHUTNEY_PATH not valid, trying $TOR_DIR/../chutney
test-network.sh: Calling newer chutney script /home/anuradha/tor/../chutney/tools/test-network.sh
test-network.sh: using CHUTNEY_DNS_CONF '/dev/null'
test-network.sh: $TOR_DIR not set, trying $PWD
test-network.sh: $TOR_DIR is a Tor 0.3.5 or later build directory
test-network.sh: $CHUTNEY_PATH not valid, trying $TOR_DIR/../chutney
test-network.sh: Setting $CHUTNEY_TOR and $CHUTNEY_TOR_GENCERT based on TOR_DIR: '/home/anuradha/tor'
test-network.sh: Using $CHUTNEY_TOR: '/home/anuradha/tor/src/app/tor' and $CHUTNEY_TOR_GENCERT: '/home/anuradha/tor/src/tools/tor-gencert'
==== Running tests: bootstrap attempt 1/1

Launching chutney using Python 3.7.5
Sending SIGINT to nodes
Waiting for nodes to finish.

Launching chutney using Python 3.7.5
bootstrap-network.sh: bootstrapping network: single-onion-v23-ipv6-md

Launching chutney using Python 3.7.5
NOTE: creating '/home/anuradha/tor/../chutney/net/nodes.1586426206', linking to '/home/anuradha/tor/../chutney/net/nodes'
Creating identity key for test000a     with /home/anuradha/tor/src/tools/tor-gencert
Creating identity key for test001a     with /home/anuradha/tor/src/tools/tor-gencert

Launching chutney using Python 3.7.5
Starting nodes

Launching chutney using Python 3.7.5
test000a     is running with PID 23143: Tor 0.4.4.0-alpha-dev (git-00ce25a7206d0914)
test001a     is running with PID 23146: Tor 0.4.4.0-alpha-dev (git-00ce25a7206d0914)
test002r     is running with PID 23149: Tor 0.4.4.0-alpha-dev (git-00ce25a7206d0914)
test003r     is running with PID 23152: Tor 0.4.4.0-alpha-dev (git-00ce25a7206d0914)
test004r     is running with PID 23155: Tor 0.4.4.0-alpha-dev (git-00ce25a7206d0914)
test005c     is running with PID 23158: Tor 0.4.4.0-alpha-dev (git-00ce25a7206d0914)
test006c     is running with PID 23161: Tor 0.4.4.0-alpha-dev (git-00ce25a7206d0914)
test007h     is running with PID 23164: Tor 0.4.4.0-alpha-dev (git-00ce25a7206d0914)
test008h     is running with PID 23167: Tor 0.4.4.0-alpha-dev (git-00ce25a7206d0914)
9/9 nodes are running
Waiting up to 120 seconds for a consensus containing relays to be generated...

Launching chutney using Python 3.7.5
Waiting for nodes to bootstrap...

test000a
None
test000a
None
test001a
None
test001a
None
test002r
None
test002r
None
test003r
None
test003r
None
test004r
None
test004r
None
test000a
None
test000a
None
test001a
None
test001a
None
test002r
None
test002r
None
test003r
None
test003r
None
test004r
None
test004r
None
test000a
None
test000a
None
test001a
None
test001a
None
test002r
None
test002r
None
test003r
None
test003r
None
test004r
None
test004r
None
test000a
None
test000a
None
test001a
None
test001a
None
test002r
None
test002r
None
test003r
None
test003r
None
test004r
None
test004r
None
test000a
None
test000a
None
test001a
None
test001a
None
test002r
None
test002r
None
test003r
None
test003r
None
test004r
None
test004r
None
test000a
None
test000a
None
test001a
None
test001a
None
test002r
None
test002r
None
test003r
None
test003r
None
test004r
None
test004r
None
test000a
None
test000a
None
test001a
None
test001a
None
test002r
None
test002r
None
test003r
None
test003r
None
test004r
None
test004r
None
test000a
None
test000a
None
test001a
None
test001a
None
test002r
None
test002r
None
test003r
None
test003r
None
test004r
None
test004r
None
test000a
None
test000a
None
test001a
None
test001a
None
test002r
None
test002r
None
test003r
None
test003r
None
test004r
None
test004r
None
test000a
None
test000a
None
test001a
None
test001a
None
test002r
None
test002r
None
test003r
None
test003r
None
test004r
None
test004r
None
Bootstrap in progress: 10 seconds
Node status:
test000a     :  100, done                     , Done
test001a     :  100, done                     , Done
test002r     :    0, starting                 , Starting
test003r     :    0, starting                 , Starting
test004r     :    0, starting                 , Starting
test005c     :   50, loading_descriptors      , Loading relay descriptors
test006c     :   50, loading_descriptors      , Loading relay descriptors
test007h     :   50, loading_descriptors      , Loading relay descriptors
test008h     :   30, loading_status           , Loading networkstatus consensus
Published dir info:
test000a     : -400, 002r 003r 004r 008h      , all formats                   , No dir file
test001a     : -400, 002r 003r 004r 008h      , all formats                   , No dir file
test002r     : -400, 002r 003r 004r 008h      , all formats                   , No dir file
test003r     : -400, 002r 003r 004r 008h      , all formats                   , No dir file
test004r     : -400, 002r 003r 004r 008h      , all formats                   , No dir file

test000a
None
test000a
None
test000a
None
test001a
None
test001a
None
test001a
None
test002r
None
test002r
None
test002r
None
test003r
None
test003r
None
test003r
None
test004r
None
test004r
None
test004r
None
test000a
None
test000a
None
test000a
None
test001a
None
test001a
None
test001a
None
test002r
None
test002r
None
test002r
None
test003r
None
test003r
None
test003r
None
test004r
None
test004r
None
test004r
None
test000a
None
test000a
None
test000a
None
test000a
None
test001a
None
test001a
None
test001a
None
test001a
None
test002r
None
test002r
None
test002r
None
test002r
None
test003r
None
test003r
None
test003r
None
test003r
None
test004r
None
test004r
None
test004r
None
test004r
None
test000a
None
test000a
None
test000a
None
test000a
None
test001a
None
test001a
None
test001a
None
test001a
None
test002r
None
test002r
None
test002r
None
test002r
None
test003r
None
test003r
None
test003r
None
test003r
None
test004r
None
test004r
None
test004r
None
test004r
None
test000a
None
test000a
None
test000a
None
test000a
None
test001a
None
test001a
None
test001a
None
test001a
None
test002r
None
test002r
None
test002r
None
test002r
None
test003r
None
test003r
None
test003r
None
test003r
None
test004r
None
test004r
None
test004r
None
test004r
None
test000a
None
test000a
None
test000a
None
test000a
None
test001a
None
test001a
None
test001a
None
test001a
None
test002r
None
test002r
None
test002r
None
test002r
None
test003r
None
test003r
None
test003r
None
test003r
None
test004r
None
test004r
None
test004r
None
test004r
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
Bootstrap in progress: 20 seconds
Node status:
test000a     :  100, done                     , Done
test001a     :  100, done                     , Done
test002r     :  100, done                     , Done
test003r     :  100, done                     , Done
test004r     :  100, done                     , Done
test005c     :  100, done                     , Done
test006c     :  100, done                     , Done
test007h     :  100, done                     , Done
test008h     :  100, done                     , Done
Published dir info:
test000a     : -400, 004r                     , desc ns_cons                  , No dir file
test001a     : -400, 004r                     , desc ns_cons                  , No dir file
test002r     : -400, 004r                     , desc ns_cons                  , No dir file
test003r     : -400, 004r                     , desc ns_cons                  , No dir file
test004r     : -400, 004r                     , desc ns_cons                  , No dir file

test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
Bootstrap in progress: 30 seconds
Node status:
test000a     :  100, done                     , Done
test001a     :  100, done                     , Done
test002r     :  100, done                     , Done
test003r     :  100, done                     , Done
test004r     :  100, done                     , Done
test005c     :  100, done                     , Done
test006c     :  100, done                     , Done
test007h     :  100, done                     , Done
test008h     :  100, done                     , Done
Published dir info:
test000a     : -400, 004r                     , desc ns_cons                  , No dir file
test001a     : -400, 004r                     , desc ns_cons                  , No dir file
test002r     : -400, 004r                     , desc ns_cons                  , No dir file
test003r     : -400, 004r                     , desc ns_cons                  , No dir file
test004r     : -400, 004r                     , desc ns_cons                  , No dir file

test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
Bootstrap in progress: 41 seconds
Node status:
test000a     :  100, done                     , Done
test001a     :  100, done                     , Done
test002r     :  100, done                     , Done
test003r     :  100, done                     , Done
test004r     :  100, done                     , Done
test005c     :  100, done                     , Done
test006c     :  100, done                     , Done
test007h     :  100, done                     , Done
test008h     :  100, done                     , Done
Published dir info:
test000a     : -400, 004r                     , desc ns_cons                  , No dir file
test001a     : -400, 004r                     , desc ns_cons                  , No dir file
test002r     : -400, 004r                     , desc ns_cons                  , No dir file
test003r     : -400, 004r                     , desc ns_cons                  , No dir file
test004r     : -400, 004r                     , desc ns_cons                  , No dir file

test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test000a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test001a
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test002r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test003r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
test004r
None
Everything bootstrapped after 49 sec
Bootstrap finished: 49 seconds
Node status:
test000a     :  100, done                     , Done
test001a     :  100, done                     , Done
test002r     :  100, done                     , Done
test003r     :  100, done                     , Done
test004r     :  100, done                     , Done
test005c     :  100, done                     , Done
test006c     :  100, done                     , Done
test007h     :  100, done                     , Done
test008h     :  100, done                     , Done
Published dir info:
test000a     :  100, all nodes                , desc md_cons ns_cons          , Dir info cached
test001a     :  100, all nodes                , desc md_cons ns_cons          , Dir info cached
test002r     :  100, all nodes                , desc md_cons ns_cons          , Dir info cached
test003r     :  100, all nodes                , desc md_cons ns_cons          , Dir info cached
test004r     :  100, all nodes                , desc md_cons ns_cons          , Dir info cached

Waiting 30 seconds for other dir info to sync...

Running 1 verify rounds for this bootstrap...

Launching chutney using Python 3.7.5
Verifying data transmission: (retrying for up to 60 seconds)
Connecting:
  HS to xnjnkfr4hgyzkknj.onion:5858 (127.0.0.1:4747) via client localhost:9005
  HS to xnjnkfr4hgyzkknj.onion:5858 (127.0.0.1:4747) via client localhost:9006
  HS to ixy7xwk3a22wk7uupwyhfxmqixykb7dyxvlcnxzfname65akllha4mad.onion:5858 (127.0.0.1:4747) via client localhost:9005
  HS to ixy7xwk3a22wk7uupwyhfxmqixykb7dyxvlcnxzfname65akllha4mad.onion:5858 (127.0.0.1:4747) via client localhost:9006
Transmitting Data:

Transmission: Success
NOTE: Registering send-data-1
NOTE: Registering check-2
NOTE: Registering send-data-3
NOTE: Registering check-4
NOTE: Registering send-data-5
NOTE: Registering check-6
NOTE: Registering send-data-7
NOTE: Registering check-8
NOTE: Success for send-data-3
NOTE: Success for send-data-1
NOTE: Success for check-4
NOTE: Success for check-2
NOTE: Success for send-data-5
NOTE: Success for check-6
NOTE: Success for send-data-7
NOTE: Success for check-8
NOTE: Status:
{'send-data-1': 'Flushed', 'send-data-3': 'Flushed', 'send-data-5': 'Flushed', 'send-data-7': 'Flushed', 'check-4': 'successful verification', 'check-2': 'successful verification', 'check-6': 'successful verification', 'check-8': 'successful verification'}
Completed verify round 1/1 in this bootstrap

Launching chutney using Python 3.7.5
Sending SIGINT to nodes
Waiting for nodes to finish.
..
==== Chutney succeeded after 1 attempt(s).
PASS single-onion-v23-ipv6-md (exit status: 0)

comment:27 in reply to:  25 Changed 7 months ago by teor

Replying to teor:

Here are the things I need to do when I merge:

  • squash the commits
  • document what "None" means in getNodeDirInfoStatusPattern()

Here are some things that either of us can do:

  • check TorNet.py for any other comments about microdescs not being implemented
  • if md is True, print the values of nickname and ed25519_key in getNodeDirInfoStatusPattern()
  • if we are not getting the right value for ed25519_id, move _setEd25519Id() into LocalNodeController, so we can use it during wait_for_bootstrap()

Are these the right values for ed25519_id?

test000a
None
...
test001a
None
...
test002r
None
...
test003r
None
...
test004r
None
...
test000a
None
...

comment:28 Changed 7 months ago by anuradha1904

Hey teor, no,they are not. Okay, will move _setEd25519Id() into LocalNodeController. I had a doubt though, do we wanted the key to be shown instant of "None"?

Last edited 7 months ago by anuradha1904 (previous) (diff)

comment:29 in reply to:  28 Changed 7 months ago by anuradha1904

Thank you got the issue now. We have mentioned in the function None as an expection, and we don't necessarily want that. I did print the values to understand that. Thank you.
Replying to anuradha1904:

Hey teor, no,they are not. Okay, will move _setEd25519Id() into LocalNodeController. I had a doubt though, do we wanted the key to be shown instant of "None"?

comment:30 Changed 7 months ago by teor

Hi, you've told me that the changes are working in this PR:
https://github.com/torproject/chutney/pull/65

You also asked what the problem with the old code was:

  • _setEd25519Id() was only called when the keys were created, during chutney configure
  • so _env['ed25519_id'] was only set during chutney configure
  • but getNodeDirInfoStatusPattern() is only called during chutney wait_for_bootstrap, which runs in a different python process
  • so when getNodeDirInfoStatusPattern() looked for _env['ed25519_id'], it was not set

The solution was to move _setEd25519Id() to LocalNodeController, so we could call it during chutney wait_for_bootstrap.

I will leave a detailed review on the pull request.

Please use this ticket to communicate - we don't seem to be on IRC at the same time.

comment:31 Changed 6 months ago by anuradha1904

Hi teor, can you kindly look at the failed travis job in

the PR : https://github.com/torproject/chutney/pull/65 I have provided the link. I am sorry for informing you bit late, got caught up in something very urgent.

comment:32 Changed 6 months ago by teor

I see this error:

  File "/Users/travis/build/torproject/chutney/lib/chutney/TorNet.py", line 843, in _setEd25519Id
    CURRENT_FILE_SIZE = os.stat(key_file).st_size
OSError: [Errno 2] No such file or directory: '/var/folders/nz/vv4_9tw56nv9k3tkvyszvwg80000gn/T/tmp.lzZm6Ev0/nodes/005c/keys/ed25519_master_id_public_key'

https://travis-ci.org/github/torproject/chutney/jobs/675703309#L912

What do you think this error means?

Remember that we said that the code needs to work, even if the ed25519_master_id_public_key file does not exist?

What do we need to do to fix this code?

How do we catch errors in python?

comment:33 Changed 6 months ago by teor

Cc: nickm ahf added

You also seem to be struggling with editing python files, and resolving git conflicts.

What's the best way for us to help you with that?
What editor are you using?
Maybe you can do some online python and git tutorials?

comment:34 Changed 6 months ago by anuradha1904

Sorry with the editing, I am using VS code, the git conflict issue was something, that got ignored by me mistakenly. I am very sorry for that. I am comfortable with python, but I think i need to polish it more, and I am working hard on that by practising data structures and algorithms in python.Answering your further questions:-

  1. This means that the file is not found.
  2. Add conditions in a try catch block to tackle situation like this.
  3. We use try catch block.
Last edited 6 months ago by anuradha1904 (previous) (diff)

comment:35 in reply to:  34 Changed 5 months ago by teor

Replying to anuradha1904:

Sorry with the editing, I am using VS code, the git conflict issue was something, that got ignored by me mistakenly. I am very sorry for that. I am comfortable with python, but I think i need to polish it more, and I am working hard on that by practising data structures and algorithms in python.Answering your further questions:-

  1. This means that the file is not found.
  2. Add conditions in a try catch block to tackle situation like this.
  3. We use try catch block.

Hi, I'm sorry I didn't see your response. Trac doesn't send out emails for comment edits.

comment:36 Changed 5 months ago by teor

Resolution: fixed
Status: needs_revisionclosed

Fixed in #33428.

comment:37 Changed 5 months ago by teor

Parent ID: #33428#33050
Note: See TracTickets for help on using tickets.