Opened 4 years ago

Last modified 2 months ago

#20647 new enhancement

Tor and Chutney CI improvements

Reported by: chelseakomlo Owned by:
Priority: High Milestone: Tor: unspecified
Component: Core Tor/Tor Version:
Severity: Normal Keywords: test, chutney, consistency, jenkins, integration-testing, continuous-integration, postfreeze-ok, teor-unreached-2019-03-08, tor-ci, 041-deferred-20190530, 042-can, 044-deferred
Cc: Actual Points:
Parent ID: Points:
Reviewer: weasel Sponsor:

Description (last modified by teor)

We currently run chutney tests on every Tor and Chutney branch and pull request. We deal with non-determinism by allowing a small number of chutney failures, before reporting an overall test failure.

But sometimes failures are hard to diagnose. This ticket is a parent ticket for improvements that help us diagnose chutney failures.

Child Tickets

TicketTypeStatusOwnerSummary
#21573defectnewFix tor warnings that chutney is currently ignoring
#22132defectclosednickmChutney should avoid waiting for set times: wait for conditions instead
#22133enhancementnewWhen CHUTNEY_DEBUG is set, make tor log a debug log
#27912enhancementclosedteorAdd travis CI for the Chutney repository
#27965defectclosedteorChutney's single-onion-ipv6-md test fails on master
#28192defectclosedWork out why 0.3.5 and later fail chutney (but 0.3.4 and earlier do not)
#28204defectnewMake chutney log the verify progress for each circuit
#28220enhancementnewCreate a chutney network with a single authority, and make sure it bootstraps
#28228enhancementnewIn Chutney's debug mode, dump all tor warning logs to stderr as soon as they appear

Change History (27)

comment:1 Changed 4 years ago by chelseakomlo

If we wanted to use docker for this, I did a proof of concept here: git@github.com:chelseakomlo/tor-integration-ci.git, master branch

We can also test using different OS/versions as we do with unit tests.

This runs both stem and chutney tests, but I see that we already have a stem task in jenkins, so I can change this to run only chutney tests if we want to keep them separate.

comment:2 Changed 4 years ago by chelseakomlo

After a short chat with weasel, we talked about:

  1. Docker doesn't fit into our current setup but the ideas from the docker file should translate well
  2. Chutney tests should be their own task
  3. Ideally we will use the tor binary that has already been built in prior tasks

For 3, it looks like the location of the tor binary needs to be set by the environment variable CHUTNEY_TOR

comment:3 Changed 4 years ago by chelseakomlo

Component: - Select a componentCore Tor/Tor

comment:4 Changed 4 years ago by dgoulet

Keywords: test added
Milestone: Tor: 0.2.???

comment:5 Changed 4 years ago by teor

Milestone: Tor: 0.2.???Tor: 0.3.???

Milestone renamed

comment:6 Changed 4 years ago by nickm

Keywords: tor-03-unspecified-201612 added
Milestone: Tor: 0.3.???Tor: unspecified

Finally admitting that 0.3.??? was a euphemism for Tor: unspecified all along.

comment:7 Changed 3 years ago by nickm

Keywords: tor-03-unspecified-201612 removed

Remove an old triaging keyword.

comment:8 Changed 3 years ago by nickm

Keywords: chutney consistency jenkins integration-testing continuous-integration added

comment:9 Changed 2 years ago by teor

Milestone: Tor: unspecifiedTor: 0.3.6.x-final
Owner: set to teor
Status: newassigned
Summary: Run chutney tests in JenkinsRun chutney tests in Jenkins or Travis

I want to do this in 0.3.6, to catch more bugs like #27080.

As well as chelsea's implementation, meejah has a travis chutney here: https://travis-ci.org/tahoe-lafs/tahoe-lafs/jobs/417471760#L797

comment:10 Changed 2 years ago by teor

Parent ID: #27230

comment:11 Changed 2 years ago by teor

Parent ID: #27230#27146

We should make this change to test #27146.

comment:12 Changed 23 months ago by teor

Parent ID: #27146

comment:13 Changed 23 months ago by teor

Milestone: Tor: 0.3.6.x-finalTor: 0.3.5.x-final
Priority: MediumHigh

#27912 is a duplicate, applying properties

comment:14 Changed 22 months ago by catalyst

Milestone: Tor: 0.3.5.x-finalTor: 0.3.6.x-final

comment:15 Changed 21 months ago by nickm

Milestone: Tor: 0.3.6.x-finalTor: 0.4.0.x-final

Tor 0.3.6.x has been renamed to 0.4.0.x.

comment:16 Changed 19 months ago by nickm

Keywords: postfreeze-ok added

Mark some tickets as postfreeze-ok, to indicate that I think they are okay to accept in 0.4.0 post-freeze. Does not indicate that they are all necessary to do postfreeze.

comment:17 Changed 18 months ago by nickm

Milestone: Tor: 0.4.0.x-finalTor: 0.4.1.x-final

IIUC this is roadmapped for 0.4.1.x

comment:18 Changed 17 months ago by teor

Keywords: teor-unreached-2019-03-08 added
Owner: teor deleted

I'd like to do these tickets, but not in the next few months.

comment:19 Changed 15 months ago by nickm

Keywords: tor-ci added

comment:20 Changed 15 months ago by nickm

Keywords: 041-deferred-20190530 added

Marking these tickets as deferred from 041.

comment:21 Changed 15 months ago by nickm

Milestone: Tor: 0.4.1.x-finalTor: 0.4.2.x-final

comment:22 Changed 14 months ago by teor

Summary: Run chutney tests in Jenkins or TravisTor and Chutney CI improvements

Renaming ticket, because chutney CI works quite well in the Tor and Chutney repositories.

comment:23 Changed 11 months ago by nickm

Keywords: 042-can added

comment:24 Changed 7 months ago by teor

Description: modified (diff)

Update description.

comment:25 Changed 6 months ago by teor

Status: assignednew

Change tickets that are assigned to nobody to "new".

comment:26 Changed 3 months ago by nickm

Milestone: Tor: 0.4.2.x-finalTor: 0.4.4.x-final

comment:27 Changed 2 months ago by nickm

Keywords: 044-deferred added
Milestone: Tor: 0.4.4.x-finalTor: unspecified

Bulk-remove tickets from 0.4.4. Add the 044-deferred label to them.

Note: See TracTickets for help on using tickets.