#23281 closed defect (fixed)

py-stem test failures on FreeBSD

Reported by: yurivict271 Owned by: atagar
Priority: Medium Milestone:
Component: Core Tor/Stem Version:
Severity: Normal Keywords: testing, freebsd
Cc: Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

Tests for py-stem-1.5.4 on FreeBSD 11.1:

--- failures with python27 ---
TESTING FAILED (53 seconds)

[UNIT TEST] test_stats (test.unit.util.proc.TestProc) ... FAIL
[UNIT TEST] test_tail (test.unit.util.system.TestSystem) ... FAIL
[UNIT TEST] test_load_processed_files_permissions (test.unit.descriptor.reader.TestDescriptorReader) ... FAIL
[UNIT TEST] test_skip_listener_read_failure (test.unit.descriptor.reader.TestDescriptorReader) ... FAIL
[UNIT TEST] test_parsing_with_example (test.unit.manual.TestManual) ... ERROR
[UNIT TEST] test_parsing_with_unknown_options (test.unit.manual.TestManual) ... ERROR
[UNIT TEST] test_saving_manual (test.unit.manual.TestManual) ... ERROR
[RUN_OPEN] test_expand_path (test.integ.util.system.TestSystem) ... FAIL
[RUN_OPEN] test_pids_by_user (test.integ.util.system.TestSystem) ... ERROR
[RUN_OPEN] test_sdist_matches_git (test.integ.installation.TestInstallation) ... ERROR
[RUN_OPEN] test_hash_password (test.integ.process.TestProcess) ... FAIL
[RUN_OPEN] test_hidden_services_conf (test.integ.control.controller.TestController) ... ERROR

--- failures with python36 ---
TESTING FAILED (53 seconds)

[UNIT TEST] test_stats (test.unit.util.proc.TestProc) ... FAIL
[UNIT TEST] test_tail (test.unit.util.system.TestSystem) ... FAIL
[UNIT TEST] test_load_processed_files_permissions (test.unit.descriptor.reader.TestDescriptorReader) ... FAIL
[UNIT TEST] test_skip_listener_read_failure (test.unit.descriptor.reader.TestDescriptorReader) ... FAIL
[UNIT TEST] test_parsing_with_example (test.unit.manual.TestManual) ... ERROR
[UNIT TEST] test_parsing_with_unknown_options (test.unit.manual.TestManual) ... ERROR
[UNIT TEST] test_saving_manual (test.unit.manual.TestManual) ... ERROR
[RUN_OPEN] test_expand_path (test.integ.util.system.TestSystem) ... FAIL
[RUN_OPEN] test_pids_by_user (test.integ.util.system.TestSystem) ... ERROR
[RUN_OPEN] test_set_process_name (test.integ.util.system.TestSystem) ... ERROR
[RUN_OPEN] test_sdist_matches_git (test.integ.installation.TestInstallation) ... ERROR
[RUN_OPEN] test_hash_password (test.integ.process.TestProcess) ... FAIL
[RUN_OPEN] test_hidden_services_conf (test.integ.control.controller.TestController) ... ERROR

Tested the port with this patch: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=221637

Child Tickets

Attachments (3)

py-stem-test.txt (966.8 KB) - added by yurivict271 23 months ago.
full test log
test.integ.control.controller.txt (100.2 KB) - added by yurivict271 22 months ago.
tests-log-python-36.txt (23.6 KB) - added by yurivict271 22 months ago.

Download all attachments as: .zip

Change History (16)

comment:1 Changed 23 months ago by atagar

Component: - Select a componentCore Tor/Stem
Owner: set to atagar

comment:2 Changed 23 months ago by atagar

Status: newneeds_information

Hi yurivict271, what is the full test output? A simple listing of which tests doesn't tell me much.

Changed 23 months ago by yurivict271

Attachment: py-stem-test.txt added

full test log

comment:3 Changed 23 months ago by yurivict271

The git-related failure is fine because this isn't a git repository.
Other failures look genuine.

comment:4 Changed 22 months ago by atagar

Very helpful - thanks yurivict271!


======================================================================
FAIL: test_stats
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/mock/mock.py", line 1305, in patched
    return func(*args, **keywargs)
  File "/usr/ports/security/py-stem/work/stem-1.5.4/test/unit/util/proc.py", line 138, in test_stats
    self.assertEqual(response, proc.stats(24062, *args))
AssertionError: Tuples differ: ('0.13',) != ('0.1015625',)

First differing element 0:
'0.13'
'0.1015625'

- ('0.13',)
+ ('0.1015625',)

----------------------------------------------------------------------

Think I see what's up. Proc results take a CLOCK_TICK constant into account. It probably differs between our systems. Pushed a fix that should get this test to pass...

https://gitweb.torproject.org/stem.git/commit/?id=3fb2faa


======================================================================
FAIL: test_tail
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/ports/security/py-stem/work/stem-1.5.4/test/unit/util/system.py", line 399, in test_tail
    self.assertRaises(IOError, list, system.tail(temp_path))
AssertionError: IOError not raised

----------------------------------------------------------------------

======================================================================
FAIL: test_load_processed_files_permissions
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/ports/security/py-stem/work/stem-1.5.4/test/unit/descriptor/reader.py", line 206, in test_load_processed_files_permissions
    self.assertRaises(IOError, stem.descriptor.reader.load_processed_files, test_listing_path)
AssertionError: IOError not raised

======================================================================
FAIL: test_skip_listener_read_failure
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/ports/security/py-stem/work/stem-1.5.4/test/unit/descriptor/reader.py", line 584, in test_skip_listener_read_failure
    self.assertTrue(isinstance(skip_exception, stem.descriptor.reader.ReadFailed))
AssertionError: False is not true

----------------------------------------------------------------------

Interesting. This tests are creating files we lack read access on, then asserting we cannot read them. We can add a 'if freebsd skip this test' but I'm kinda curious - can you not revoke your own read access with freebsd? What do you get when you run the following?

% echo "hello world" > /tmp/test_file
% chmod -r /tmp/test_file 
% python
Python 2.7.3 (default, Oct 26 2016, 21:04:23) 
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> open('/tmp/test_file', 'rb').read()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IOError: [Errno 13] Permission denied: '/tmp/test_file'

======================================================================
ERROR: test_parsing_with_example
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/ports/security/py-stem/work/stem-1.5.4/test/unit/manual.py", line 157, in test_parsing_with_example
    manual = stem.manual.Manual.from_man(EXAMPLE_MAN_PATH)
  File "/usr/ports/security/py-stem/work/stem-1.5.4/stem/manual.py", line 378, in from_man
    raise IOError("Unable to run '%s': %s" % (man_cmd, exc))
IOError: Unable to run 'man --encoding=ascii -P cat /usr/ports/security/py-stem/work/stem-1.5.4/test/unit/tor_man_example': man --encoding=ascii -P cat /usr/ports/security/py-stem/work/stem-1.5.4/test/unit/tor_man_example returned exit status 1

======================================================================
ERROR: test_parsing_with_unknown_options
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/ports/security/py-stem/work/stem-1.5.4/test/unit/manual.py", line 180, in test_parsing_with_unknown_options
    manual = stem.manual.Manual.from_man(UNKNOWN_OPTIONS_MAN_PATH)
  File "/usr/ports/security/py-stem/work/stem-1.5.4/stem/manual.py", line 378, in from_man
    raise IOError("Unable to run '%s': %s" % (man_cmd, exc))
IOError: Unable to run 'man --encoding=ascii -P cat /usr/ports/security/py-stem/work/stem-1.5.4/test/unit/tor_man_with_unknown': man --encoding=ascii -P cat /usr/ports/security/py-stem/work/stem-1.5.4/test/unit/tor_man_with_unknown returned exit status 1

======================================================================
ERROR: test_saving_manual
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/ports/security/py-stem/work/stem-1.5.4/test/unit/manual.py", line 207, in test_saving_manual
    manual = stem.manual.Manual.from_man(EXAMPLE_MAN_PATH)
  File "/usr/ports/security/py-stem/work/stem-1.5.4/stem/manual.py", line 378, in from_man
    raise IOError("Unable to run '%s': %s" % (man_cmd, exc))
IOError: Unable to run 'man --encoding=ascii -P cat /usr/ports/security/py-stem/work/stem-1.5.4/test/unit/tor_man_example': man --encoding=ascii -P cat /usr/ports/security/py-stem/work/stem-1.5.4/test/unit/tor_man_example returned exit status 1

----------------------------------------------------------------------

I'd like to fix this if we can. What do you get when you run that command?

man --encoding=ascii -P cat /usr/ports/security/py-stem/work/stem-1.5.4/test/unit/tor_man_with_unknown

Is there something else we can run to get a man page's content without the formatting markup? Here's what I get...

% man --encoding=ascii -P cat test/unit/tor_man_with_unknown
TOR(1)                                                                                            Tor Manual                                                                                            TOR(1)



NAME
       tor - The second-generation onion router

GENERAL OPTIONS
       BandwidthRate N bytes|KBytes|MBytes|GBytes|KBits|MBits|GBits
           A token bucket limits the average incoming bandwidth usage on this node to the specified number of bytes per second, and the average outgoing bandwidth usage to that same value. If you want to
           run a relay in the public network, this needs to be at the very least 30 KBytes (that is, 30720 bytes). (Default: 1 GByte)


           With this option, and in other options that take arguments in bytes, KBytes, and so on, other formats are also supported. Notably, "KBytes" can also be written as "kilobytes" or "kb"; "MBytes"
           can be written as "megabytes" or "MB"; "kbits" can be written as "kilobits"; and so forth. Tor also accepts "byte" and "bit" in the singular. The prefixes "tera" and "T" are also recognized. If
           no units are given, we default to bytes. To avoid confusion, we recommend writing "bytes" or "bits" explicitly, since it's easy to forget that "B" means bytes, not bits.

NEW OPTIONS
       SpiffyNewOption transport exec path-to-binary [options]
           Description of this new option.

Tor                                                                                               10/03/2015                                                                                            TOR(1)

======================================================================
ERROR: test_pids_by_user
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/ports/security/py-stem/work/stem-1.5.4/test/integ/util/system.py", line 377, in test_pids_by_user
    self.assertTrue(os.getpid() in pids)
TypeError: argument of type 'NoneType' is not iterable

======================================================================

Is there a way with freebsd to get the pids a user is running? This test is running 'ps -o pid -u username' and seems that doesn't work. If not then not the end of the world - we can note this isn't supported on freebsd.


======================================================================
FAIL: test_expand_path
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/ports/security/py-stem/work/stem-1.5.4/test/integ/util/system.py", line 563, in test_expand_path
    self.assertEqual(home_dir, stem.util.system.expand_path('~%s' % username))
AssertionError: '/root' != '/home/yuri'

----------------------------------------------------------------------

Interesting. Does the output of these two commands differ for you?

% python
Python 2.7.3 (default, Oct 26 2016, 21:04:23) 
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.path.expanduser('~')
'/home/atagar'
>>> os.path.expanduser('~%s' % os.environ['USER'])
'/home/atagar'

======================================================================
FAIL: test_hash_password
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/ports/security/py-stem/work/stem-1.5.4/test/integ/process.py", line 104, in test_hash_password
    self.assertTrue(re.match('^16:[0-9A-F]{58}\n$', output))
AssertionError: None is not true

----------------------------------------------------------------------

Interesting. What do you get from the following? Note that 'my_password' here can be anything - just interested in the format of the output.

tor --hush --hash-password my_password
16:FF773344208EE8F4600696F285936B0EA81B6F6598204A061B631F68E2

======================================================================
ERROR: test_hidden_services_conf
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/ports/security/py-stem/work/stem-1.5.4/test/runner.py", line 127, in wrapped
    return func(self, *args, **kwargs)
  File "/usr/ports/security/py-stem/work/stem-1.5.4/test/integ/control/controller.py", line 549, in test_hidden_services_conf
    controller.set_hidden_service_conf(initialconf)
  File "/usr/ports/security/py-stem/work/stem-1.5.4/stem/control.py", line 2582, in set_hidden_service_conf
    self.set_options(hidden_service_options)
  File "/usr/ports/security/py-stem/work/stem-1.5.4/stem/control.py", line 2419, in set_options
    raise stem.InvalidRequest(response.code, response.message)
InvalidRequest: Unacceptable option value: Failed to configure rendezvous options. See logs for details.

----------------------------------------------------------------------

Interesting. Maybe port 8020, 8021, 8030, 8031, or 8032 are already in use? Odd ports to have a conflict with but possible I suppose. If you don't think that's it mind attaching the output of the following?

% run_tests.py --integ --log TRACE --verbose --test test.integ.control.controller

STATIC CHECKS
460	* /usr/ports/security/py-stem/work/stem-1.5.4/stem/descriptor/remote.py
461	  line 445  - E722 do not use bare except'             | except:
462	  line 1013 - E722 do not use bare except'             | except:

These look ok. Pycodestyle added some new assertions since our last release. If you test against the git version these should be suppressed.

comment:5 Changed 22 months ago by yurivict271

You are welcome!
Thank you for working on this!

comment:6 Changed 22 months ago by atagar

Keywords: testing freebsd added

Hi yurivict271. Just to be very clear ball's in your court. I don't have a FreeBSD system so I'll need to rely on someone who can answer the above questions to make progress.

Changed 22 months ago by yurivict271

comment:7 Changed 22 months ago by yurivict271

What do you get when you run the following?

% echo "hello world" > /tmp/test_file

Getting exactly the same as you.


$ man --encoding=ascii -P cat /usr/ports/security/py-stem/work/stem-1.5.4/test/unit/tor_man_with_unknown
Illegal option --
Usage:

man [-adho] [-t | -w] [-M manpath] [-P pager] [-S mansect]

[-m arch[:machine]] [-p [eprtv]] [mansect] page [...]

man -f page [...] -- Emulates whatis(1)
man -k page [...] -- Emulates apropos(1)


"ps -o pid -u {username}" should be 'ps -o pid -U {username}'.


Does the output of these two commands differ for you? (os.path.expanduser)

No


What do you get from the following?

$ tor --hush --hash-password my_password
16:919400F2374C552760B38DFF6F23280ADB7C67FAA66AC72323C69B7B88


Attaching the output of 'python2 run_tests.py --integ --log TRACE --verbose --test test.integ.control.controller'

comment:8 Changed 22 months ago by atagar

Oh wait, I see part of the issue. You're running Stem 1.5.4, not the latest git codebase. The assertion about 'tor --hush --hash-password my_password' has changed since what you're running. Please test against the latest codebase instead.

% git clone https://git.torproject.org/stem.git

$ man --encoding=ascii -P cat /usr/ports/security/py-stem/work/stem-1.5.4/test/unit/tor_man_with_unknown
Illegal option --

Great! Turns out we had this issue with OSX in the past too. Fix pushed...

https://gitweb.torproject.org/stem.git/commit/?id=fe1512c

"ps -o pid -u {username}" should be 'ps -o pid -U {username}'.

Perfect. Pushed a fix for this too. Does this do the trick?

https://gitweb.torproject.org/stem.git/commit/?id=fe1512c

comment:9 Changed 22 months ago by yurivict271

Now, with the current version, these tests still fail with python-2.7:

$ python2 run_tests.py -a
TESTING FAILED (2 seconds)

[UNIT TEST] test_from_man_when_manual_is_unavailable (test.unit.manual.TestManual) ... FAIL
[UNIT TEST] test_parsing_with_example (test.unit.manual.TestManual) ... ERROR
[UNIT TEST] test_parsing_with_unknown_options (test.unit.manual.TestManual) ... ERROR
[UNIT TEST] test_saving_manual_as_config (test.unit.manual.TestManual) ... ERROR
[UNIT TEST] test_saving_manual_as_sqlite (test.unit.manual.TestManual) ... ERROR

These tests fail with python-3.6:

$ python3.6 run_tests.py -a
...skipped...
TESTING FAILED (16 seconds)

[UNIT TEST] test_router_status_entry (test.unit.descriptor.server_descriptor.TestServerDescriptor) ... FAIL
[UNIT TEST] test_from_man_when_manual_is_unavailable (test.unit.manual.TestManual) ... FAIL
[UNIT TEST] test_parsing_with_example (test.unit.manual.TestManual) ... ERROR
[UNIT TEST] test_parsing_with_unknown_options (test.unit.manual.TestManual) ... ERROR
[UNIT TEST] test_saving_manual_as_config (test.unit.manual.TestManual) ... ERROR
[UNIT TEST] test_saving_manual_as_sqlite (test.unit.manual.TestManual) ... ERROR
[RUN_OPEN] test_set_conf_when_immutable (test.integ.control.controller.TestController) ... ERROR

You can re-run just these tests with:

run_tests.py -a --test test.integ.control.controller
run_tests.py -a --test test.unit.manual
run_tests.py -a --test test.unit.descriptor.server_descriptor

Attaching the full test log with python-3.6

Changed 22 months ago by yurivict271

Attachment: tests-log-python-36.txt added

comment:10 Changed 22 months ago by atagar

Thanks! Manual tests failures are due to recent changes, and I was able to repro several python3 failures on my system. Great catches, I'll correct 'em tomorrow.

comment:11 Changed 22 months ago by atagar

Thanks yurivict271! Think I finally got 'em all. Do the tests now pass for you?

======================================================================
FAIL: test_router_status_entry
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/stem/test/unit/descriptor/server_descriptor.py", line 296, in test_router_status_entry
    self.assertEqual('4F0069BF91C04581B7C3CA9272E2D3228D4EA571', desc.digest)
AssertionError: '4F0069BF91C04581B7C3CA9272E2D3228D4EA571' != 'A863EFE8395C41C880782B89B850D20EDD242BDA'
- 4F0069BF91C04581B7C3CA9272E2D3228D4EA571
+ A863EFE8395C41C880782B89B850D20EDD242BDA

----------------------------------------------------------------------

Fixed: https://gitweb.torproject.org/stem.git/commit/?id=982c36e

======================================================================
ERROR: test_saving_manual_as_config
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/stem/test/require.py", line 58, in wrapped
    return func(self, *args, **kwargs)
  File "/tmp/stem/test/unit/manual.py", line 205, in test_saving_manual_as_config
    manual = stem.manual.Manual.from_man(EXAMPLE_MAN_PATH)
  File "/tmp/stem/stem/manual.py", line 495, in from_man
    categories, config_options = _get_categories(man_output), OrderedDict()
  File "/tmp/stem/stem/manual.py", line 666, in _get_categories
    if lines[-1] == '':
IndexError: list index out of range

----------------------------------------------------------------------

Fixed: https://gitweb.torproject.org/stem.git/commit/?id=336a2a8

======================================================================
ERROR: test_set_conf_when_immutable
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/stem/test/require.py", line 58, in wrapped
    return func(self, *args, **kwargs)
  File "/tmp/stem/test/integ/control/controller.py", line 778, in test_set_conf_when_immutable
    self.assertRaisesRegexp(stem.InvalidArguments, "DisableAllSwap cannot be changed while tor's running", controller.set_conf, 'DisableAllSwap', '1')
  File "/tmp/stem/stem/util/test_tools.py", line 272, in assertRaisesRegexp
    return super(original_type, self).assertRaisesRegexp(exc_type, exc_msg, func, *args, **kwargs)
  File "/usr/local/lib/python3.6/unittest/case.py", line 1320, in deprecated_func
    return original_func(*args, **kwargs)
  File "/usr/local/lib/python3.6/unittest/case.py", line 1267, in assertRaisesRegex
    return context.handle('assertRaisesRegex', args, kwargs)
  File "/usr/local/lib/python3.6/unittest/case.py", line 178, in handle
    callable_obj(*args, **kwargs)
  File "/tmp/stem/stem/control.py", line 2303, in set_conf
    self.set_options({param: value}, False)
  File "/tmp/stem/stem/control.py", line 2412, in set_options
    raise stem.InvalidRequest(response.code, response.message)
stem.InvalidRequest: Transition not allowed: While Tor is running, changing DisableAllSwap is not allowed.

----------------------------------------------------------------------

Fixed: https://gitweb.torproject.org/stem.git/commit/?id=1b9a68b

STATIC CHECKS
* /tmp/stem/cache_manual.py
  line 51   - undefined name 'exc'                     | print('Cached database schema is out of date (was %s, but current version is %s)' % (exc.database_schema, stem.manual.SCHEMA_VERSION))

Fixed: https://gitweb.torproject.org/stem.git/commit/?id=1b9a68b

comment:12 Changed 22 months ago by yurivict271

All tests pass now with both python-2.7 and python-3.6.

Thank you for fixing them, atagar!

comment:13 Changed 22 months ago by atagar

Resolution: fixed
Status: needs_informationclosed

My pleasure, thanks for all your patience and help yurivict271!

Note: See TracTickets for help on using tickets.