Test and support python 3.x

Stem's setup.py does a python 2to3 conversion of our codebase, but we don't yet have a method for running our tests on the built output. We should do that and investigate testing failures in the 2to3 conversion.

added component::archived/stem owner::atagar priority::medium resolution::implemented status::closed type::enhancement labels

I should have mentioned that the goal for this from a testing perspective is to have a '--python3' argument for the tests that...

• does a 2to3 export of our codebase via setup.py (including the tests)
• runs the tests on the 2to3 conversion of our codebase
• optionally cleans up the build output

Damian,

Some comments:

1. remember to allow for specifying the python3 binary location
2. run_tests.py also needs to be run through 2to3
3. there are several cross-version changes I believe can be had now

For 2 (above), maybe python3 testing should be a separate command? For example, a shell script which does:

mkdir python3-tests
2to3 --write --no-diffs --nobackups --write-unchanged-files --output-dir=python3-tests stem test run_tests.py
( cd python3-tests; $PYTHON3 ./run_tests.py )

For 3 (above): are you interested in some low-hanging Py2->Py3 fruit? For example:

- import Queue
+ import Queue as queue

and change Queue.Queue initializations to queue.Queue. Then only one line (the import statement) is changed by 2to3 (because the Queue module was renamed to queue in Py3).

Trac:
Username: robinson

For 2 (above), maybe python3 testing should be a separate command? For example, a shell script which does:

I like that idea but lets do it through run_tests.py. I admit it's weird to have a python 2 instance of run_tests.py invoking a python 3 version, but it's probably a little nicer from an interface perspective.

For 3 (above): are you interested in some low-hanging Py2->Py3 fruit? For example:

If you're sure that it'll be a problem for python 3 and this maintains python 2.5 compatibility then go for it (dropping 2.5 might be ok if 2to3 forces us to, but that would be something to think about a bit).

Replying to atagar:

For 2 (above), maybe python3 testing should be a separate command? For example, a shell script which does:

I like that idea but lets do it through run_tests.py. I admit it's weird to have a python 2 instance of run_tests.py invoking a python 3 version, but it's probably a little nicer from an interface perspective.

So... um... you mean, run a Py2 script which converts the code base, including itself, to Py3, then execs Py3 on the newly-converted version of the running script?

For 3 (above): are you interested in some low-hanging Py2->Py3 fruit? For example:

If you're sure that it'll be a problem for python 3 and this maintains python 2.5 compatibility then go for it (dropping 2.5 might be ok if 2to3 forces us to, but that would be something to think about a bit).

I'll put together some examples and send a public branch address to read.

Trac:
Username: robinson

So... um... you mean, run a Py2 script which converts the code base, including itself, to Py3, then execs Py3 on the newly-converted version of the running script?

Somehow I get the sense that you're not quite thrilled with this idea. :P

It would largely be the same as the shell script you mentioned. The only difference is that it lives in test_runner.py, in essence...

python3_destination = os.path.join(CONFIG["integ.test_directory"], 'python3')
os.makedirs(python3_destination)
system.call("2to3 --write --no-diffs --nobackups --write-unchanged-files --output-dir=%s stem test run_tests.py" % python3_destination)
system.call("python3 %s/run_tests.py %s" % (python3_destination, " ".join(sys.argv[1:]))
shutil.rmtree(python3_destination, ignore_errors = True)
sys.exit(0) # TODO: use the exit status of the above call instead

The advantages of this are...

• '--python3' can be a testing argument just like any other.
• No need to have yet another little shell script floating around (fewer of those mean less clutter, which I like).

Feel free to talk me down from the ledge. ;)

Replying to atagar:

So... um... you mean, run a Py2 script which converts the code base, including itself, to Py3, then execs Py3 on the newly-converted version of the running script?

Somehow I get the sense that you're not quite thrilled with this idea. :P

[snipped]

Feel free to talk me down from the ledge. ;)

Ok. That's not as horrible as I was imagining. I can support your concept for this method of testing in Py3.

Trac:
Username: robinson

I've started working on this a bit. The tests in the following branch now support a '--python3' argument and can at least get through a unit test run. Still lots of testing failures/errors though...

https://gitweb.torproject.org/user/atagar/stem.git/shortlog/refs/heads/python3

We now have python 3.x support. Yay!

https://gitweb.torproject.org/stem.git/commitdiff/3930f1f1eda99bcf22fe574b95665f8ef47ffdc4?hp=7921a46ba8655baa7c4b81d900a9854444675564

Trac:
Status: new to closed
Resolution: N/A to implemented

closed

mentioned in issue #7906 (closed)