Opened 5 years ago

Closed 15 months ago

#12243 closed task (wontfix)

Document running integration tests and link to their reports

Reported by: nathan-at-least Owned by: nathan-at-least
Priority: Medium Milestone:
Component: Archived/Ooni Version:
Severity: Normal Keywords: archived-closed-2018-07-04
Cc: Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

On a private Least Authority issue tracker, earthrise wrote:

Tasks:

Try to run the integration tests with a PASS. Also note which unit tests fail.
When running the tests by trial ooni after vagrant up, the test_oonibclient.py tests are skipped, because the backend isn't present. test_oonicli.py apparently passes, although I'm not sure which backend it's using. Answer: The HTTP header manipulation test is using the hard-coded http://64.9.225.221 test helper. The other tests don't need a backend (i.e. they hit websites on the internet or test against the 8.8.8.8 DNS server). We should recommend that the http header test be run against a local copy of oonib, so that it tests the http header part of oonib as well.
WARNING: Running vagrant up does NOT install your local copy of ooni-probe, it just installs the .deb file from their repository.
Okay, I got most of them to pass! I had to install it myself in a Vagrant VM. I'll create a VagrantFile so that it can be reproduced. Some of the integration tests fail because the backend doesn't have the deck, nettest, etc.
Find out if ooni/travis actually are running these tests.
They are being run on travis-ci, although the test_oonibclient.py is being skipped (due to lack of bouncer).
[ ] Figure out which decks and nettests the bouncer must have so that the integration tests pass. Then modify the Vagrantfile so they actually do pass. (Moved to #25).
[ ] See if there is an easy way to separate coverage measurement for unit tests and integration tests. (Moved to #27).
Add this information to assessment.rst.

Child Tickets

Attachments (1)

Vagrantfile (1.3 KB) - added by nathan-at-least 5 years ago.
from ooni-releng commit e4dab6c34b62b440ab310f7c3383b42c6d43ee25

Download all attachments as: .zip

Change History (5)

Changed 5 years ago by nathan-at-least

Attachment: Vagrantfile added

from ooni-releng commit e4dab6c34b62b440ab310f7c3383b42c6d43ee25

comment:1 Changed 5 years ago by nathan-at-least

earthrise created the Vagrantfile attachment and says:

I added a Vagrantfile for running the integration tests in a reproducible way...

comment:2 Changed 5 years ago by nathan-at-least

Component: - Select a componentOoni

comment:3 Changed 22 months ago by teor

Severity: Normal

Set all open tickets without a severity to "Normal"

comment:4 Changed 15 months ago by teor

Keywords: archived-closed-2018-07-04 added
Resolution: wontfix
Status: newclosed

Close all tickets in archived components

Note: See TracTickets for help on using tickets.