Opened 4 months ago

Last modified 4 days ago

#30860 merge_ready enhancement

Add a chutney job that runs on macOS, so that IPv6 chutney tests work

Reported by: teor Owned by: teor
Priority: Medium Milestone: Tor: 0.4.2.x-final
Component: Core Tor/Tor Version:
Severity: Normal Keywords: CI, PTs, sponsor-28-maybe, consider-backport-after-ci-passes, 029-backport, 035-backport, 040-backport, 041-backport, network-team-roadmap-2019-Q1Q2, tor-ci, teor-backlog-ci, 042-deferred-20190918, teor-merge
Cc: catalyst Actual Points: 1.0
Parent ID: #31851 Points: 0.5
Reviewer: nickm Sponsor: Sponsor27-can

Description

Travis Linux doesn't support IPv6, so we should add a macOS chutney job to Tor's CI.

If that's too slow, we can:

  • add the chutney tests to the end of an existing macOS job
  • change the chutney job to macOS

Remember: each build has a limit of 2 concurrent macOS jobs.

We can do this task after #29280 merges.

Child Tickets

TicketTypeStatusOwnerSummary
#31859taskclosedteorDelete similar CI jobs

Change History (18)

comment:1 Changed 4 months ago by teor

Parent ID: #29280#29267
Sponsor: Sponsor28-can

comment:2 Changed 4 months ago by teor

Cc: gaba added
Keywords: sponsor-28-maybe added
Sponsor: Sponsor28-can

Hi Gaba, I put this task in sponsor 28-can because it is related to #29280, PTs, and chutney.

It should be a quick fix, please put it back in if you think it is important.

comment:3 Changed 3 months ago by teor

Keywords: teor-backlog-ci added
Owner: teor deleted
Sponsor: Sponsor31-can

Some of these tickets are part of the sponsor 31 CI improvements task, we will pick the specific tickets later, and assign them to people.

comment:4 Changed 4 weeks ago by nickm

Keywords: 042-deferred-20190918 added
Milestone: Tor: 0.4.2.x-finalTor: unspecified

Deferring various tickets from 0.4.2 to Unspecified.

comment:5 Changed 3 weeks ago by teor

Cc: cohosh gaba removed
Parent ID: #29267#31851

I'm going to redesign the test matrix in #31851 to allow for the new module tests.

This would also be a good opportunity to do:

  • macOS -> macOS chutney
  • remove Linux chutney

We should not change:

  • macOS rust

Because we've had bugs in macOS rust compilation on some previous rustc versions.

comment:6 Changed 3 weeks ago by teor

Sponsor: Sponsor31-canSponsor27-can

We need this ticket to test the sponsor 27 HSv3 IPv6 code. But it can also be part of the Sponsor 31 CI improvements.

comment:7 Changed 3 weeks ago by teor

Actual Points: 0.4
Milestone: Tor: unspecifiedTor: 0.4.2.x-final
Status: assignedneeds_review

Please review and merge these PRs:

Clean merges - testing only:

comment:8 Changed 3 weeks ago by teor

Owner: set to teor
Status: needs_reviewassigned

comment:9 Changed 3 weeks ago by teor

Status: assignedneeds_review

(Wanted: ticket system that changes owners in one step)

comment:10 Changed 2 weeks ago by catalyst

Cc: catalyst added

comment:11 Changed 2 weeks ago by teor

Status: needs_reviewneeds_revision

This change decreases the total time spent on the CI infrastructure, but it actually increases the wall-clock time.

I want to try these changes:

  1. Re-run all the builds, so they benefit from the build caches
  2. Reorder the builds so the slowest builds run first
  3. De-merge the chutney/check and distcheck/rust builds

Edited to add:

Decreasing the total time spent on the CI infrastructure *does* speed up the *next* build, because we are using fewer CI builders. But that only really benefits backports and alpha+master merges.

Last edited 2 weeks ago by teor (previous) (diff)

comment:12 in reply to:  11 ; Changed 2 weeks ago by teor

Replying to teor:

I want to try these changes:

  1. Re-run all the builds, so they benefit from the build caches

The build caches aren't the issue.

  1. Reorder the builds so the slowest builds run first
  2. De-merge the chutney/check and distcheck/rust builds

Rather than de-merging the builds, I disabled make check on them. I also put the slowest compilers (macOS/gcc, Linux/clang) on the fastest build on each platform.

I activated fast_finish, but I don't think it will make much difference: the chutney job is the slowest job.

Here are the latest PRs:

Clean merges - testing only:

But macOS chutney is really slow, so I might need to make a separate make target that only runs the IPv6 networks.

comment:13 Changed 2 weeks ago by teor

I think the fastest solution to this issue is to:

  • add a linux chutney job
  • allow failures on the macOS chutney job

It would be nice to have a Travis config for "finish before this job is done, but fail the build if it fails". But maybe we should just speed up our tests :-)

comment:14 in reply to:  12 Changed 12 days ago by teor

Actual Points: 0.41.0
Keywords: consider-backport-after-ci-passes 041-backport added
Status: needs_revisionneeds_review

Please review my PRs:

Clean merges - testing only:

macOS chutney and Rust are really slow, so I use fast_finish and allow_failure to let the job finish before they are done. Unfortunately, that means we don't see failures for those jobs in the build summary or IRC notifications. But we have enough fast tests, so I expect failures to be rare.

comment:15 Changed 12 days ago by teor

Here are the CI wallclock build time improvements, in minutes.

Release Current Speed PR Speed Improvement
0.2.9 16 9 44%
0.3.5 11 30 63%
0.4.0 13 20 35%
0.4.1 20 32 38%
master 13 35 63%

These times vary depending on server load, but they are clearly better.

comment:16 Changed 7 days ago by dgoulet

Reviewer: nickm

comment:17 Changed 5 days ago by nickm

Status: needs_reviewmerge_ready

This LGTM. Let's give it a try. One thing I don't understand, though, is the table above with different speeds. Are the columns reversed or partially reversed? 1-(9/16) == 44%, but 1-(35/13) != 63%.

Since this is CI, I think we should merge it all at once. Would you be okay doing the merges?

comment:18 Changed 4 days ago by nickm

Keywords: teor-merge added
Note: See TracTickets for help on using tickets.