Opened 2 years ago

Closed 2 years ago

#14113 closed enhancement (implemented)

It would be cool if it were easier to figure out how to re-run a single test.

Reported by: nickm Owned by: atagar
Priority: Low Milestone:
Component: Core Tor/Stem Version:
Severity: Keywords:
Cc: Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

In #14111, I had a set of tests fail. Re-running them one at a time wasn't too easy. At first, I thought that the --test option to run_tests.py would help, but it turned out to do something different.

It would be cool if, after seeing a string like ERROR: test_pid_by_name_ps_bsd or test_pid_by_name_ps_bsd [FAILURE], I could just pass test_pid_by_name_ps_bsd directly back into the run_tests.py script and repeat the single failing test.

Child Tickets

Change History (4)

comment:1 Changed 2 years ago by atagar

I suspect --test does what you want but its usage is a little confusing. It takes a test module but not individual tests. I tried to illustrate how it's used in the examples...

  Examples:

   run_tests.py --integ --test test.integ.util
     Only run integration tests for the util modules.

... but maybe there's another tweak for the --help output that would make it clearer. Mind a suggestion on what would've helped?

comment:2 Changed 2 years ago by nickm

I think my problem here is that looking at the output of "run_test.py" didn't seem to tell me what module I was in. So to figure out how to re-run test_pid_by_name_ps_bsd, I need to figure out what module that's actually in. Is there something in the output that's supposed to tell me that?

comment:3 Changed 2 years ago by atagar

Yup. The test run output shows modules, then only includes individual test names if there was a failure. So the thing for the --test output was the first line that said failure (rather than the lower line that had the individual test).

Not sure how best for the test output to communicate it though. One option is that at the very end we could give the run_tests.py --test foo commands to re-run failures,..

comment:4 Changed 2 years ago by atagar

  • Resolution set to implemented
  • Status changed from new to closed

Thanks Nick! This was my favourite suggestion. Stem now includes the commands for re-running individual test failures...

https://gitweb.torproject.org/stem.git/commit/?id=6c15247fbfd991090f91d0c4f28f36d0e31b4b43

Note: See TracTickets for help on using tickets.