Opened 3 months ago

Closed 2 months ago

#27252 closed enhancement (implemented)

Reduce the number of travis jobs

Reported by: teor Owned by: teor
Priority: Medium Milestone: Tor: 0.3.5.x-final
Component: Core Tor/Tor Version:
Severity: Normal Keywords: rust tor-ci 029-backport-maybe 032-backport 033-backport 034-backport
Cc: dmr, catalyst Actual Points:
Parent ID: Points:
Reviewer: catalyst Sponsor:

Description

After #24629, we have some follow-up work.

In 0.2.9 and later:

  • work out if we really need clang and gcc on Linux and macOS

In 0.3.2 and later:

  • reduce the number of rust online travis jobs, to reduce travis network failures

Child Tickets

Change History (11)

comment:1 Changed 3 months ago by teor

Reducing the number of online rust jobs won't stop Travis network failures - it just reduces the number of network connections we make.

For example, this job errored at the VM initialisation stage, probably due to network issues. Then I restarted it:
https://travis-ci.org/torproject/tor/jobs/418948444

We should also look for any other opportunities to reduce network usage.
(But really, Travis should fix their infrastructure.)

comment:2 in reply to:  description ; Changed 3 months ago by catalyst

Replying to teor:

In 0.2.9 and later:

  • work out if we really need clang and gcc on Linux and macOS

I think if Xcode comes with only clang, and Tor Browser doesn't build on macOS with gcc, we don't need to test with gcc on macOS.

comment:3 Changed 3 months ago by dmr

Cc: dmr added

comment:4 in reply to:  2 Changed 3 months ago by teor

Replying to catalyst:

Replying to teor:

In 0.2.9 and later:

  • work out if we really need clang and gcc on Linux and macOS

I think if Xcode comes with only clang, and Tor Browser doesn't build on macOS with gcc, we don't need to test with gcc on macOS.

Here are some reasons why I'd like to keep gcc on macOS:

  • sometimes gcc on macOS finds warnings or errors that don't appear on Linux
  • while most native builders use clang, gcc can still cross-compile for macOS
  • the macOS package managers MacPorts and Fink support building tor with gcc, if it is the first compiler in the path

I don't know if these reasons are good enough to keep a separate macOS gcc build. I will see if we can combine some the macOS builds.

In general, I'd like all the macOS builds, and all the clang builds, to be orthogonal, because they are slower than the Linux gcc builds.

comment:5 Changed 3 months ago by teor

Reviewer: catalyst
Status: assignedneeds_review

Here's what I changed:

  • 0.2.9:
    • skip macOS gcc (-1 job on 0.2.9, -2 jobs on 0.3.2 and later)
    • skip the non-hardened build, because coverage also turns off hardening (-1 job on 0.2.9)
  • 0.3.2 and later:
    • do offline rust builds by default, and only do one online rust build
    • skip offline rust builds for Linux gcc, because we do an online rust build for Linux gcc (-1 job on 0.3.2 and later)
  • 0.3.4 and later:
    • Don't do a distcheck with --disable-module-dirauth (-1 configure/make/test on 0.3.4 and later)

Please review:

  • 0.2.9: ticket27252-029
  • 0.3.2: ticket27252-032
  • 0.3.3: ticket27252-033 (non-trivial merge)
  • 0.3.4: ticket27252-034 (merges cleanly to master)

I'm assigning catalyst as the reviewer, because they asked for this ticket. But feel free to delay the review for high-priority work.

comment:6 Changed 3 months ago by teor

Cc: catalyst added

Oh, the Travis CI is here:
https://travis-ci.org/teor2345/tor/branches

And the branches are here:
https://github.com/teor2345/tor.git

comment:7 Changed 3 months ago by teor

I missed the most obvious build to remove:

  • 0.2.9:
    • skip Linux gcc with the default env, because all the non-default builds use Linux gcc (-1 job on 0.2.9 and later)

So we're down to:

  • 9 jobs on 0.3.4 and later
  • 7 jobs on 0.3.2 and 0.3.3
  • 4 jobs on 0.2.9

I really do think we have a minimal set now.

comment:8 Changed 3 months ago by teor

It turns out that OS X gcc is an alias for clang, so our OS X gcc builds are redundant:
https://github.com/travis-ci/docs-travis-ci-com/pull/2002

Let's fix that in #27376, if we decide we want gcc on OS X.

comment:9 Changed 3 months ago by teor

Reviewer: catalyst

I'm throwing this back in the review pool, because 1/3 travis jobs are now failing due to network errors.

comment:10 Changed 2 months ago by catalyst

Reviewer: catalyst
Status: needs_reviewmerge_ready

Looks good to me. Good work! It looks like it will improve things significantly.

comment:11 Changed 2 months ago by nickm

Resolution: implemented
Status: merge_readyclosed

Merged each branch to its corresponding target, then merged forward and pushed!

Note: See TracTickets for help on using tickets.