Add travis CI for the Chutney repository

added actualpoints::1.5 component::core tor/chutney owner::teor parent::20647 points::0.5 priority::high resolution::fixed severity::normal sponsor::19 status::closed type::enhancement labels

Trac:
Priority: Medium to High

This ticket involves two tasks:

• add another stage to tor's travis config that runs chutney (eventually, tor's second stage could run RAM usage, performance, and other runtime tests)
• add a travis config to chutney that installs whichever tor is available in the package manager, then runs a chutney verify. We should run chutney tests on python 2 and 3, and Linux and macOS

#20647 (moved) is a duplicate, let's use this ticket for the chutney changes

Trac:
Parent: N/A to #20647 (moved)
Component: Core Tor/Tor to Core Tor/Chutney

Taylor, could you try out the minimal version here? That is, just add a 'test-network' step and see if it works? If there are bugs, don't feel you need to chase them: you can just open tickets.

Trac:
Points: N/A to 0.5
Status: new to assigned
Owner: N/A to catalyst

Replying to teor:

This ticket involves two tasks:

• add another stage to tor's travis config that runs chutney (eventually, tor's second stage could run RAM usage, performance, and other runtime tests) Stages and jobs on Travis don't share storage, so I think we'd have to find a way to persist stuff if we want to try this.
• add a travis config to chutney that installs whichever tor is available in the package manager, then runs a chutney verify. We should run chutney tests on python 2 and 3, and Linux and macOS Sounds good. At least on Debian OSes, the system tor package seems to start a tor daemon by default. I'm not sure we want this.

#28013 (moved) is for the tor repository parts of this work. Let's leave this ticket for the chutney repository parts.

Trac:
Status: assigned to new

Trac:
Owner: catalyst to N/A
Status: new to assigned

Trac:
Status: assigned to new

Trac:
Milestone: Tor: 0.3.5.x-final to Tor: 0.3.6.x-final

Make title clearer

Trac:
Summary: Add Chutney to travis-ci if possible to Add travis CI for the Chutney repository

I have an experimental version in my chutney-travis branch at https://github.com/teor2345/chutney.git

It's still going through CI.

Trac:
Owner: N/A to teor
Status: new to assigned

Trac:
Status: assigned to needs_revision

I have a working travis config, and much better debugging: https://github.com/torproject/chutney/pull/2

0.2.9, 0.3.3, and 0.3.4 fail less than 1% of the time. Example failure on 0.2.9: https://travis-ci.org/torproject/chutney/jobs/445982217

0.3.5 and 0.3.6 fail about 90% of the time, so I put them in allow_failures, and opened #28192 (moved) to track down the bug.

I want to copy the make test-network-all networks from each tor version into the travis config. Then I'll be done with this branch.

We can't fix 0.3.4 and earlier, so I want to run them twice (or however long it takes to get an acceptable error rate), and make the test pass if either succeeds.

Replying to teor:

... I want to copy the make test-network-all networks from each tor version into the travis config. Then I'll be done with this branch.

Since this is chutney, I'll just run the networks from tor stable on stable, and any extra networks in master on nightly. That's enough.

Then I need to decide on a default network: either basic-min, or chutney's default bridges-etc-etc-etc.

Edit: I also want to split the debug and travis changes into separate branches for review.

Edit: I also want to split the debug and travis changes into separate branches for review.

I merged the debug branch to master, rebased the chutney-travis branch, and closed the relevant tickets.

Tor 0.3.6.x has been renamed to 0.4.0.x.

Trac:
Milestone: Tor: 0.3.6.x-final to Tor: 0.4.0.x-final

Like #28922 (moved), we should think about copying the torproject key into the chutney repository, to reduce Travis network errors.

Trac:
Milestone: Tor: 0.4.0.x-final to N/A