Opened 6 years ago

Closed 5 years ago

#9865 closed task (fixed)

Create a test harness for BridgeDB

Reported by: isis Owned by: isis
Priority: Medium Milestone:
Component: Obfuscation/BridgeDB Version:
Severity: Keywords: bridgedb-0.1.0, bridgedb-unittests
Cc: sysrqb, isis@… Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

BridgeDB's tests are currently stored in lib/BridgeDB/Tests.py and use the standard library's unittest module. Normally, Twisted code is tested with twisted.trial, for the reason that if something returns a twisted.internet.defer.Deferred the stdlib unittest module doesn't know to wait for the result (nor does it understand how to clear the reactor).

Some things which should be done:

  • We need a test runner which can pass on commands (i.e. doing "python setup.py test" would actually run "trial coverage [args] [tests/test directory]" and pass the arguments and test cases to coverage if need be. See tahoe-lafs's code for an example) and environment variables (i.e. PYTHONPATH).
  • twisted.trial has a conversion utility class for translating test results from stdlib unittest into something which trial's reporter can use. I've never used it, but either some sort of conversion should be done of the old tests, or they should be rewritten.
  • Find a way to write integration tests for the web interface.Windmill (more docs) and twill look promising. Windmill probably wouldn't run very nicely on a headless server, however, as it drives a browser.
  • Find a way to write tests for the email responder.

If we get all this running, we can setup Travis-CI and get automated continuous integration, which should help prevent bugs like #9626 and #9156.

Useful Resources:
http://pycheesecake.org/wiki/PythonTestingToolsTaxonomy

Child Tickets

TicketStatusOwnerSummaryComponent
#9872closedisisCreate a test runner for BridgeDB unittestsObfuscation/BridgeDB
#9873closedisisConvert BridgeDB's old unittests to use twisted.trialObfuscation/BridgeDB
#9937closedisisCreate fake non-sanitised bridge descriptors for BridgeDB testing purposesObfuscation/BridgeDB
#10723closedisisCreate a staging instance for BridgeDBObfuscation/BridgeDB

Change History (3)

comment:1 Changed 6 years ago by isis

Cc: isis@… added
Owner: set to isis
Status: newassigned

comment:2 Changed 5 years ago by isis

Update:

BridgeDB now has CI testing.
It also has twisted.trial-based unittests.
And a trial-based test runner which works by doing "bridgedb test".
The trial-based test runner also includes support for running BridgeDB's old unittests.
And automatic code coverage reports.

comment:3 Changed 5 years ago by isis

Keywords: bridgedb-0.1.0 added; bridgedb removed
Resolution: fixed
Status: assignedclosed

The runner for the old unittests is here, and it additionally runs tests for:

  • doctest examples
  • the old unittests in lib/bridgedb/Tests.py, with current code
  • the old unittests above, but monkeypatched to use some older pieces of code

While many modules still need more unittest coverage, I believe this ticket is done.

Note: See TracTickets for help on using tickets.