Opened 5 months ago

Closed 5 weeks ago

#30860 closed enhancement (fixed)

Add a chutney job that runs on macOS, so that IPv6 chutney tests work

Reported by: teor Owned by: teor
Priority: Medium Milestone: Tor: 0.2.9.x-final
Component: Core Tor/Tor Version:
Severity: Normal Keywords: CI, PTs, sponsor-28-maybe, consider-backport-after-ci-passes, 029-backport, 035-backport, 040-backport, 041-backport, network-team-roadmap-2019-Q1Q2, tor-ci, teor-backlog-ci, 042-deferred-20190918, teor-merge
Cc: catalyst Actual Points: 1.5
Parent ID: #31851 Points: 0.5
Reviewer: nickm Sponsor: Sponsor27-can

Description

Travis Linux doesn't support IPv6, so we should add a macOS chutney job to Tor's CI.

If that's too slow, we can:

  • add the chutney tests to the end of an existing macOS job
  • change the chutney job to macOS

Remember: each build has a limit of 2 concurrent macOS jobs.

We can do this task after #29280 merges.

Child Tickets

TicketTypeStatusOwnerSummary
#31859taskclosedteorDelete similar CI jobs

Change History (21)

comment:1 Changed 5 months ago by teor

Parent ID: #29280#29267
Sponsor: Sponsor28-can

comment:2 Changed 5 months ago by teor

Cc: gaba added
Keywords: sponsor-28-maybe added
Sponsor: Sponsor28-can

Hi Gaba, I put this task in sponsor 28-can because it is related to #29280, PTs, and chutney.

It should be a quick fix, please put it back in if you think it is important.

comment:3 Changed 4 months ago by teor

Keywords: teor-backlog-ci added
Owner: teor deleted
Sponsor: Sponsor31-can

Some of these tickets are part of the sponsor 31 CI improvements task, we will pick the specific tickets later, and assign them to people.

comment:4 Changed 2 months ago by nickm

Keywords: 042-deferred-20190918 added
Milestone: Tor: 0.4.2.x-finalTor: unspecified

Deferring various tickets from 0.4.2 to Unspecified.

comment:5 Changed 8 weeks ago by teor

Cc: cohosh gaba removed
Parent ID: #29267#31851

I'm going to redesign the test matrix in #31851 to allow for the new module tests.

This would also be a good opportunity to do:

  • macOS -> macOS chutney
  • remove Linux chutney

We should not change:

  • macOS rust

Because we've had bugs in macOS rust compilation on some previous rustc versions.

comment:6 Changed 8 weeks ago by teor

Sponsor: Sponsor31-canSponsor27-can

We need this ticket to test the sponsor 27 HSv3 IPv6 code. But it can also be part of the Sponsor 31 CI improvements.

comment:7 Changed 8 weeks ago by teor

Actual Points: 0.4
Milestone: Tor: unspecifiedTor: 0.4.2.x-final
Status: assignedneeds_review

Please review and merge these PRs:

Clean merges - testing only:

comment:8 Changed 8 weeks ago by teor

Owner: set to teor
Status: needs_reviewassigned

comment:9 Changed 8 weeks ago by teor

Status: assignedneeds_review

(Wanted: ticket system that changes owners in one step)

comment:10 Changed 8 weeks ago by catalyst

Cc: catalyst added

comment:11 Changed 8 weeks ago by teor

Status: needs_reviewneeds_revision

This change decreases the total time spent on the CI infrastructure, but it actually increases the wall-clock time.

I want to try these changes:

  1. Re-run all the builds, so they benefit from the build caches
  2. Reorder the builds so the slowest builds run first
  3. De-merge the chutney/check and distcheck/rust builds

Edited to add:

Decreasing the total time spent on the CI infrastructure *does* speed up the *next* build, because we are using fewer CI builders. But that only really benefits backports and alpha+master merges.

Last edited 8 weeks ago by teor (previous) (diff)

comment:12 in reply to:  11 ; Changed 8 weeks ago by teor

Replying to teor:

I want to try these changes:

  1. Re-run all the builds, so they benefit from the build caches

The build caches aren't the issue.

  1. Reorder the builds so the slowest builds run first
  2. De-merge the chutney/check and distcheck/rust builds

Rather than de-merging the builds, I disabled make check on them. I also put the slowest compilers (macOS/gcc, Linux/clang) on the fastest build on each platform.

I activated fast_finish, but I don't think it will make much difference: the chutney job is the slowest job.

Here are the latest PRs:

Clean merges - testing only:

But macOS chutney is really slow, so I might need to make a separate make target that only runs the IPv6 networks.

comment:13 Changed 7 weeks ago by teor

I think the fastest solution to this issue is to:

  • add a linux chutney job
  • allow failures on the macOS chutney job

It would be nice to have a Travis config for "finish before this job is done, but fail the build if it fails". But maybe we should just speed up our tests :-)

comment:14 in reply to:  12 ; Changed 7 weeks ago by teor

Actual Points: 0.41.0
Keywords: consider-backport-after-ci-passes 041-backport added
Status: needs_revisionneeds_review

Please review my PRs:

Clean merges - testing only:

macOS chutney and Rust are really slow, so I use fast_finish and allow_failure to let the job finish before they are done. Unfortunately, that means we don't see failures for those jobs in the build summary or IRC notifications. But we have enough fast tests, so I expect failures to be rare.

comment:15 Changed 7 weeks ago by teor

Here are the CI wallclock build time improvements, in minutes.

Release Current Speed PR Speed Improvement
0.2.9 16 9 44%
0.3.5 30 11 63%
0.4.0 20 13 35%
0.4.1 32 20 38%
master 35 13 63%

These times vary depending on server load, but they are clearly better.

Last edited 5 weeks ago by teor (previous) (diff)

comment:16 Changed 6 weeks ago by dgoulet

Reviewer: nickm

comment:17 Changed 6 weeks ago by nickm

Status: needs_reviewmerge_ready

This LGTM. Let's give it a try. One thing I don't understand, though, is the table above with different speeds. Are the columns reversed or partially reversed? 1-(9/16) == 44%, but 1-(35/13) != 63%.

Since this is CI, I think we should merge it all at once. Would you be okay doing the merges?

comment:18 Changed 6 weeks ago by nickm

Keywords: teor-merge added

comment:19 in reply to:  17 Changed 5 weeks ago by teor

Replying to nickm:

This LGTM. Let's give it a try. One thing I don't understand, though, is the table above with different speeds. Are the columns reversed or partially reversed? 1-(9/16) == 44%, but 1-(35/13) != 63%.

Yes, I think I made a mistake, I'll fix the table now.

Since this is CI, I think we should merge it all at once. Would you be okay doing the merges?

I can do the merges.

comment:20 Changed 5 weeks ago by nickm

I can do the merges.

Great! Please do when you have the chance.

comment:21 in reply to:  14 Changed 5 weeks ago by teor

Actual Points: 1.01.5
Milestone: Tor: 0.4.2.x-finalTor: 0.2.9.x-final
Resolution: fixed
Status: merge_readyclosed

Replying to teor:

Please review my PRs:

Clean merges - testing only:

Merged to 0.2.9 - 0.4.2: the master branch now corresponds to maint-0.4.2, and merges cleanly. And maint-0.4.2 merges cleanly to master.

Note: See TracTickets for help on using tickets.