test_practracker.sh python issues

It looks like we have a failure on Jenkins: https://jenkins.torproject.org/job/tor-ci-linux-master/ARCHITECTURE=amd64,SUITE=bullseye/4402/consoleFull

It seems like when we check-practracker-unit-tests, we try to run it even when we don't have python installed. Additionally, we don't set the PYTHON environment variable when we run check-pracktracker-unit-tests, so the script can look for python with the wrong name.

Both of these problems are solved if we remove check-practracker-unit-tests, I believe, since test_practracker.sh is already listed in TESTSCRIPTS, so we don't acutally need to have it in check-local as we thought in #32609 (moved)

changed milestone to %Tor: 0.4.3.x-final

Trac:
Child Ticket(s): #31796 (moved)

added 043-should actualpoints::0.5 component::core tor/tor jenkins milestone::Tor: 0.4.3.x-final network-team-roadmap-2020Q1 owner::teor points::0.1 priority::medium python resolution::fixed reviewer::dgoulet severity::normal status::closed tor-ci-fixed-fail type::defect labels

Trac:
Keywords: N/A deleted, tor-ci jenkins python added

Trac:
Owner: N/A to teor
Keywords: tor-ci jenkins python deleted, tor-ci-fail jenkins python added
Status: new to assigned

So I get this when running scripts/maint/practracker/test_practracker.sh.

ex0:
--- /tmp/pracktracker.test.y7DQvj/ex0-received.txt	2019-12-10 10:04:32.845275565 -0500
+++ scripts/maint/practracker/testdata/ex0-expected.txt	2019-12-10 09:52:26.505312737 -0500
@@ -0,0 +1,11 @@
+problem file-size a.c 41
+problem include-count a.c 6
+problem function-size a.c:i_am_a_function() 9
+problem function-size a.c:another_function() 12
+problem dependency-violation a.c 4
+problem file-size b.c 15
+problem function-size b.c:foo() 4
+problem function-size b.c:bar() 5
+problem file-size header.h 8
+problem include-count header.h 4
+problem dependency-violation header.h 3
FAILED

Nick asked me to describe my python situation:

python --version
Python 2.7.17

I have python 2.7, 3.7 and 3.8 installed.

I think this might be a separate bug?

Is there any stderr output? test_practracker.sh doesn't capture stderr in ex0-received.txt. But it should, so we get better diagnostics.

Replying to nickm:

Both of these problems are solved if we remove check-practracker-unit-tests, I believe, since test_practracker.sh is already listed in TESTSCRIPTS, so we don't acutally need to have it in check-local as we thought in #32609 (moved)

I pushed a commit to master, which removes this redundant, broken test. It should fix this CI issue.

Replying to dgoulet:

So I get this when running scripts/maint/practracker/test_practracker.sh.

{{{ ex0: --- /tmp/pracktracker.test.y7DQvj/ex0-received.txt 2019-12-10 10:04:32.845275565 -0500 +++ scripts/maint/practracker/testdata/ex0-expected.txt 2019-12-10 09:52:26.505312737 -0500 @@ -0,0 +1,11 @@ +problem file-size a.c 41 +problem include-count a.c 6 +problem function-size a.c:i_am_a_function() 9 +problem function-size a.c:another_function() 12 +problem dependency-violation a.c 4 +problem file-size b.c 15 +problem function-size b.c:foo() 4 +problem function-size b.c:bar() 5 +problem file-size header.h 8 +problem include-count header.h 4 +problem dependency-violation header.h 3 FAILED }}}

Nick asked me to describe my python situation:

{{{ python --version Python 2.7.17 }}}

I have python 2.7, 3.7 and 3.8 installed.

Maybe practracker is broken with one of your python versions? Or maybe it produces different output?

Can you run scripts/maint/practracker/test_practracker.sh with this PR:

• master: https://github.com/torproject/tor/pull/1599

Trac:
Summary: check-local tries to run test_practracker.sh even if we have no python to test_practracker.sh python issues

Trac:
Status: assigned to needs_review

Trac:
Keywords: tor-ci-fail jenkins python deleted, tor-ci-fixed-fail jenkins python added
Reviewer: N/A to dgoulet
Actualpoints: N/A to 0.2

Replying to teor:

Can you run scripts/maint/practracker/test_practracker.sh with this PR:

• master: https://github.com/torproject/tor/pull/1599

Same result... :S

Trac:
Status: needs_review to needs_information

That's really strange. Either the OS or the Python process should produce some kind of output. But you're getting an empty file.

Can you please run:

• "$PYTHON" --version
• "${PYTHON:-python}" "scripts/maint/practracker/practracker.py"

I also have to revise this patch, because it failed on Windows: https://ci.appveyor.com/project/teor2345/tor/builds/29486238

And I want to fix the script so we print something if run_practracker fails. (Or we should set -e in the script.)

Trac:
Status: needs_information to needs_revision

Trac:
Keywords: tor-ci-fixed-fail jenkins python deleted, tor-ci-fixed-fail jenkins python network-team-roadmap-2020Q1 added

Trac:
Keywords: tor-ci-fixed-fail jenkins python network-team-roadmap-2020Q1 deleted, tor-ci-fixed-fail jenkins python network-team-roadmap-2020Q1 043-should added

Replying to teor:

That's really strange. Either the OS or the Python process should produce some kind of output. But you're getting an empty file.

Can you please run:

I assume that by $PYTHON you mean "my python" because it is just not set.

• "$PYTHON" --version

Python 2.7.17

• "${PYTHON:-python}" "scripts/maint/practracker/practracker.py"

Does nothing. No output no nothing.

TOR_DISABLE_PRACTRACKER=1 was the culprit....

Without it, I get a stacktrace with python2 and works with python3. But I assume python3 is what is used anyway so seems all good on my side.

Here is a small patch to never set TOR_DISABLE_PRACTRACKER in the test:

Branch: no_skip_test_practracker_042 PR: https://github.com/torproject/tor/pull/1658

Trac:
Status: needs_revision to needs_review

Replying to nickm:

Here is a small patch to never set TOR_DISABLE_PRACTRACKER in the test:

Branch: no_skip_test_practracker_042 PR: https://github.com/torproject/tor/pull/1658 Looks reasonable at first glance. Maybe we should also make practracker.py write a notice or warning to stderr when it exits early due to TOR_DISABLE_PRACTRACKER, to make this sort of situation easier to troubleshoot in the future?

Replying to catalyst:

Replying to nickm:

Here is a small patch to never set TOR_DISABLE_PRACTRACKER in the test:

Branch: no_skip_test_practracker_042 PR: https://github.com/torproject/tor/pull/1658 Looks reasonable at first glance. Maybe we should also make practracker.py write a notice or warning to stderr when it exits early due to TOR_DISABLE_PRACTRACKER, to make this sort of situation easier to troubleshoot in the future?

I merged nickm's PR 1658 to 0.4.2 and later, and added catalyst's suggested stderr notice.

I also fixed up my PR for master, which improves the practracker tests:

• master: https://github.com/torproject/tor/pull/1599

My PR makes the tests:

• capture stderr output (including exceptions)
• canonicalise paths, so that logged paths are the same, even in out-of-tree builds
• fail the test if any process exits unexpectedly
• check the practracker exit status as part of the tests

I think dgoulet is right, and we shouldn't worry about Python 2 failures. (Unless someone really wants to fix them.) I updated our supported platforms policy to include Python 3: https://trac.torproject.org/projects/tor/wiki/org/teams/NetworkTeam/SupportedPlatforms#Python

Edit: Supported Python 3 versions only, canonical implementation

Trac:
Actualpoints: 0.2 to 0.5