Opened 3 years ago

Closed 3 years ago

#22366 closed defect (fixed)

run_tests.py can't find tor via relative pathname if cwd not the top of the stem tree

Reported by: catalyst Owned by: atagar
Priority: Medium Milestone:
Component: Archived/Stem Version:
Severity: Normal Keywords:
Cc: Actual Points:
Parent ID: #22301 Points:
Reviewer: Sponsor:

Description

When running run_tests.py with a current working directory that's not the top of the stem source tree, it can't find a tor binary via a relative pathname. This prevents make test-stem from working in the tor source tree. (run_tests.py seems to mostly work if given a relative pathname to tor when run from the top of the stem source tree.)

tlyu@arcadia:~/src/tor$ ../stem/run_tests.py --tor ./src/or/tor --all --log notice --target RUN_ALL

produces many errors such as

Starting ./src/or/tor...

  failed to start tor: './src/or/tor' doesn't exist

A workaround is to provide an absolute pathname for the tor binary, like

tlyu@arcadia:~/src/tor$ ../stem/run_tests.py --tor `pwd`/src/or/tor --all --log notice --target RUN_ALL

but it would be nice if the relative pathname also worked. (Adapting the make test-stem rule in tor is also a possibility.)

Child Tickets

Change History (10)

comment:1 Changed 3 years ago by atagar

Hmmm, I'm having trouble reproing this one...

% env -i ../stem/run_tests.py --tor ./tor/src/or/tor --all --log notice

...

Starting ./tor/src/or/tor...

  May 24 12:29:31.192 [notice] Tor 0.3.1.0-alpha-dev (git-44102714460aafe5) running on Linux with Libevent 2.0.16-stable, OpenSSL 1.0.1, Zlib 1.2.3.4, Liblzma N/A, and Libzstd N/A.
  May 24 12:29:31.192 [notice] Tor can't help you if you use it wrong! Learn how to be safe at https://www.torproject.org/download/download#warning
  May 24 12:29:31.192 [notice] This version is not a stable Tor release. Expect more bugs than usual.
  May 24 12:29:31.192 [notice] Read configuration file "/home/atagar/Desktop/stem/test/data/torrc".
  May 24 12:29:31.197 [warn] ControlPort is open, but no authentication method has been configured.  This means that any program on your computer can reconfigure your Tor.  That's bad!  You should upgrade your Tor controller as soon as possible.
  May 24 12:29:31.198 [notice] Opening Socks listener on 127.0.0.1:1112
  May 24 12:29:31.198 [notice] Opening Control listener on 127.0.0.1:1111
  May 24 12:29:31.000 [notice] Bootstrapped 0%: Starting
  May 24 12:29:32.000 [notice] Starting with guard context "default"
  May 24 12:29:32.000 [notice] Bootstrapped 80%: Connecting to the Tor network
  done (1 seconds)

Running tests...

... everything passes...

Per chance any chroots or other interesting twists?

comment:2 Changed 3 years ago by catalyst

Oh! After experimenting a bit more, this bug also seems dependent on the lack of a system tor binary in $PATH for reasons I don't yet understand.

tlyu@arcadia:~/src/tor$ PATH=/tmp/tor-stem:$PATH ../stem/run_tests.py --tor ./src/or/tor --all --log notice

successfully finds ./src/or/tor and runs it.

Starting ./src/or/tor...

  May 24 15:36:09.928 [notice] Tor 0.3.1.1-alpha (git-af98b862a51c001e) running on Linux with Libevent 2.0.21-stable, OpenSSL 1.0.2g, Zlib 1.2.8, Liblzma N/A, and Libzstd N/A.

/tmp/tor-stem/tor is a shell script that just runs kill -ABRT $$, so it's clearly not producing the quoted output. Of course, now test.integ.version fails because of the unusual execution properties of the "system" tor.

comment:3 Changed 3 years ago by atagar

This is strange. I've been able to repro this twice but every time I get a successful repro the subsequent invocation passes. Maybe some caching I don't yet grok. Still digging.

comment:4 Changed 3 years ago by atagar

Still little puzzled but I did manage to get a repro *if* the RELATIVE target is invoked. I'll go ahead and fix that and we'll see if it addresses the issue for you.

% PATH="/usr/bin" ../stem/run_tests.py --tor ./tor/src/or/tor --all --log notice --target RELATIVE
======================================================================
                             INITIALISING                             
======================================================================

  checking stem version...                             1.5.4-dev
  checking tor version...                              0.3.1.0-alpha-dev
  checking python version...                           2.7.3
  checking cryptography version...                     1.7.2
  checking pynacl version...                           1.0.1
  checking mock version...                             1.3.0
  checking pyflakes version...                         1.0.0
  checking pycodestyle version...                      2.3.1
  checking for orphaned .pyc files...                  done
  checking for unused tests...                         done
  running pyflakes...                                  done (1.8s)
  running pycodestyle...                               done (6.4s)

======================================================================
                              UNIT TESTS                              
======================================================================

...

======================================================================
                          INTEGRATION TESTS                           
======================================================================

Setting up a test instance...
  making test directory (/home/atagar/Desktop/stem/test/data)... skipped
  configuring logger (/home/atagar/Desktop/stem/test/data/log)... done
  writing torrc (/home/atagar/Desktop/stem/test/data/torrc)... done
    # configuration for stem integration tests
    DataDirectory ./data
    SocksPort 1112
    DownloadExtraInfo 1
    Log notice stdout
    Log notice file ./data/tor_log
    ControlPort 1111

Starting ./tor/src/or/tor...

  failed to start tor: './tor/src/or/tor' doesn't exist


Shutting down tor... done

TESTING FAILED (10 seconds)

You can re-run just these tests with:

comment:5 Changed 3 years ago by atagar

Pushed a fix for the scenario I was able to repro. Mind giving it a whirl?

https://gitweb.torproject.org/stem.git/commit/?id=0d6212e

comment:6 Changed 3 years ago by atagar

Oh! On thinking about this some more I think I know why I got the inconsistent failures earlier. To improve test performance we prepare for the installation test in parallel. That preparation calls chdir so if we start tor while in the middle of that context we'd fail this way.

Ick. Gonna look into a fix for that.

comment:7 Changed 3 years ago by catalyst

tlyu@arcadia:~/src/tor$ ../stem/run_tests.py --tor ./src/or/tor --all --log notice --target RUN_ALL

now produces several instances of

Starting /home/tlyu/src/stem/src/or/tor...

  failed to start tor: '/home/tlyu/src/stem/src/or/tor' doesn't exist

which seems to be making some progress.

Given your most recent comment I tried limiting to only test.integ.process, which succeeded, so there is some evidence to support your hypothesis.

tlyu@arcadia:~/src/tor$ ../stem/run_tests.py --tor ./src/or/tor --all --log notice --target RUN_ALL --test test.integ.process
50 TESTS WERE SKIPPED
TESTING PASSED (37 seconds)

comment:8 Changed 3 years ago by atagar

Sorry about all the back-and-forth. Change pushed. Mind giving it a whirl?

https://gitweb.torproject.org/stem.git/commit/?id=dfcdc18

comment:9 Changed 3 years ago by catalyst

Looks like it works. Thanks!

tlyu@arcadia:~/src/tor$ STEM_SOURCE_DIR=../stem make test-stem
613 TESTS WERE SKIPPED
TESTING PASSED (56 seconds)

make test-stem-full seems to have some problems but I can open separate tickets about those.

comment:10 Changed 3 years ago by atagar

Resolution: fixed
Status: newclosed

Wonderful! Thanks for pointing these out catalyst - all great catches. :P

Note: See TracTickets for help on using tickets.