Skip to content
Snippets Groups Projects
New look: Off

run_tests.py can't find tor via relative pathname if cwd not the top of the stem tree

    • Closed Issue created by Taylor Yu @catalyst

    When running run_tests.py with a current working directory that's not the top of the stem source tree, it can't find a tor binary via a relative pathname. This prevents make test-stem from working in the tor source tree. (run_tests.py seems to mostly work if given a relative pathname to tor when run from the top of the stem source tree.)

    tlyu@arcadia:~/src/tor$ ../stem/run_tests.py --tor ./src/or/tor --all --log notice --target RUN_ALL

    produces many errors such as

    Starting ./src/or/tor...

  failed to start tor: './src/or/tor' doesn't exist

    A workaround is to provide an absolute pathname for the tor binary, like

    tlyu@arcadia:~/src/tor$ ../stem/run_tests.py --tor `pwd`/src/or/tor --all --log notice --target RUN_ALL

    but it would be nice if the relative pathname also worked. (Adapting the make test-stem rule in tor is also a possibility.)

    To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information

    Child items 0

    • No child items are currently assigned. Use child items to break down this issue into smaller parts.

    Activity

    • Taylor Yu added component::archived/stem owner::atagar parent::22301 priority::medium resolution::fixed severity::normal status::closed type::defect labels

      added component::archived/stem owner::atagar parent::22301 priority::medium resolution::fixed severity::normal status::closed type::defect labels

    • A
      Damian Johnson @atagar ·

      Hmmm, I'm having trouble reproing this one...

      % env -i ../stem/run_tests.py --tor ./tor/src/or/tor --all --log notice

...

Starting ./tor/src/or/tor...

  May 24 12:29:31.192 [notice] Tor 0.3.1.0-alpha-dev (git-44102714460aafe5) running on Linux with Libevent 2.0.16-stable, OpenSSL 1.0.1, Zlib 1.2.3.4, Liblzma N/A, and Libzstd N/A.
  May 24 12:29:31.192 [notice] Tor can't help you if you use it wrong! Learn how to be safe at https://www.torproject.org/download/download#warning
  May 24 12:29:31.192 [notice] This version is not a stable Tor release. Expect more bugs than usual.
  May 24 12:29:31.192 [notice] Read configuration file "/home/atagar/Desktop/stem/test/data/torrc".
  May 24 12:29:31.197 [warn] ControlPort is open, but no authentication method has been configured.  This means that any program on your computer can reconfigure your Tor.  That's bad!  You should upgrade your Tor controller as soon as possible.
  May 24 12:29:31.198 [notice] Opening Socks listener on 127.0.0.1:1112
  May 24 12:29:31.198 [notice] Opening Control listener on 127.0.0.1:1111
  May 24 12:29:31.000 [notice] Bootstrapped 0%: Starting
  May 24 12:29:32.000 [notice] Starting with guard context "default"
  May 24 12:29:32.000 [notice] Bootstrapped 80%: Connecting to the Tor network
  done (1 seconds)

Running tests...

... everything passes...

      Per chance any chroots or other interesting twists?

    • C
      Taylor Yu @catalyst ·
      Author

      Oh! After experimenting a bit more, this bug also seems dependent on the lack of a system tor binary in $PATH for reasons I don't yet understand.

      tlyu@arcadia:~/src/tor$ PATH=/tmp/tor-stem:$PATH ../stem/run_tests.py --tor ./src/or/tor --all --log notice

      successfully finds ./src/or/tor and runs it.

      Starting ./src/or/tor...

  May 24 15:36:09.928 [notice] Tor 0.3.1.1-alpha (git-af98b862a51c001e) running on Linux with Libevent 2.0.21-stable, OpenSSL 1.0.2g, Zlib 1.2.8, Liblzma N/A, and Libzstd N/A.

      /tmp/tor-stem/tor is a shell script that just runs kill -ABRT $$, so it's clearly not producing the quoted output. Of course, now test.integ.version fails because of the unusual execution properties of the "system" tor.

    • A
      Damian Johnson @atagar ·

      This is strange. I've been able to repro this twice but every time I get a successful repro the subsequent invocation passes. Maybe some caching I don't yet grok. Still digging.

    • A
      Damian Johnson @atagar ·

      Still little puzzled but I did manage to get a repro if the RELATIVE target is invoked. I'll go ahead and fix that and we'll see if it addresses the issue for you.

      % PATH="/usr/bin" ../stem/run_tests.py --tor ./tor/src/or/tor --all --log notice --target RELATIVE
======================================================================
                             INITIALISING                             
======================================================================

  checking stem version...                             1.5.4-dev
  checking tor version...                              0.3.1.0-alpha-dev
  checking python version...                           2.7.3
  checking cryptography version...                     1.7.2
  checking pynacl version...                           1.0.1
  checking mock version...                             1.3.0
  checking pyflakes version...                         1.0.0
  checking pycodestyle version...                      2.3.1
  checking for orphaned .pyc files...                  done
  checking for unused tests...                         done
  running pyflakes...                                  done (1.8s)
  running pycodestyle...                               done (6.4s)

======================================================================
                              UNIT TESTS                              
======================================================================

...

======================================================================
                          INTEGRATION TESTS                           
======================================================================

Setting up a test instance...
  making test directory (/home/atagar/Desktop/stem/test/data)... skipped
  configuring logger (/home/atagar/Desktop/stem/test/data/log)... done
  writing torrc (/home/atagar/Desktop/stem/test/data/torrc)... done
    # configuration for stem integration tests
    DataDirectory ./data
    SocksPort 1112
    DownloadExtraInfo 1
    Log notice stdout
    Log notice file ./data/tor_log
    ControlPort 1111

Starting ./tor/src/or/tor...

  failed to start tor: './tor/src/or/tor' doesn't exist


Shutting down tor... done

TESTING FAILED (10 seconds)

You can re-run just these tests with:
    • A
      Damian Johnson @atagar ·

      Pushed a fix for the scenario I was able to repro. Mind giving it a whirl?

      https://gitweb.torproject.org/stem.git/commit/?id=0d6212e

    • A
      Damian Johnson @atagar ·

      Oh! On thinking about this some more I think I know why I got the inconsistent failures earlier. To improve test performance we prepare for the installation test in parallel. That preparation calls chdir so if we start tor while in the middle of that context we'd fail this way.

      Ick. Gonna look into a fix for that.

    • C
      Taylor Yu @catalyst ·
      Author 
      tlyu@arcadia:~/src/tor$ ../stem/run_tests.py --tor ./src/or/tor --all --log notice --target RUN_ALL

      now produces several instances of

      Starting /home/tlyu/src/stem/src/or/tor...

  failed to start tor: '/home/tlyu/src/stem/src/or/tor' doesn't exist

      which seems to be making some progress.

      Given your most recent comment I tried limiting to only test.integ.process, which succeeded, so there is some evidence to support your hypothesis.

      tlyu@arcadia:~/src/tor$ ../stem/run_tests.py --tor ./src/or/tor --all --log notice --target RUN_ALL --test test.integ.process
      50 TESTS WERE SKIPPED
TESTING PASSED (37 seconds)
    • A
      Damian Johnson @atagar ·

      Sorry about all the back-and-forth. Change pushed. Mind giving it a whirl?

      https://gitweb.torproject.org/stem.git/commit/?id=dfcdc18

    • C
      Taylor Yu @catalyst ·
      Author

      Looks like it works. Thanks!

      tlyu@arcadia:~/src/tor$ STEM_SOURCE_DIR=../stem make test-stem
      613 TESTS WERE SKIPPED
TESTING PASSED (56 seconds)

      make test-stem-full seems to have some problems but I can open separate tickets about those.

    • A
      Damian Johnson @atagar ·

      Wonderful! Thanks for pointing these out catalyst - all great catches. :P

      Trac:
      Status: new to closed
      Resolution: N/A to fixed

    • Trac closed

      closed

    • Taylor Yu mentioned in issue #22894 (closed)

      mentioned in issue #22894 (closed)

    Please register or sign in to reply