wiki:org/teams/NetworkTeam/CIFailures

CI Failures

This page lists the CI failures for the network team.

It's based on the Network Team's 0.4.0 Release Status page, and teor's user page.

CI Builders

Jenkins builds our supported branches and Debian packages for deb.torproject.org. Notifications go to #tor-bots on irc.oftc.net.

Travis and Appveyor build GitHub pull requests, personal repositories, and our supported branches. Notifications go to #tor-ci on irc.oftc.net.

Handling CI Failures

We want to log every CI failure in trac, so we can see which CI issues we need to fix first.

Supported Branches

We want CI to always pass on supported branches.

When CI fails on:

  • master, a supported maint branch, or a supported release branch, or
  • a pull request, but it's not related to the changes in the pull request.
  1. Find the failure log message, the failed branch, and the merge head (if it's a pull request).
  2. If there is an existing ticket, log that information on the existing ticket.
  3. Otherwise, log a ticket tagged "tor-ci-fail" with the information, and mark it as high priority.
  4. If the failure is on an earlier supported branch, update the version and backport tags to that branch.

Pull Requests

When CI fails on a pull request, and it was caused by the changes in the pull request, mark the ticket for that pull request as needs_revision.

CI Failure Tickets

Unassigned tickets

These open tickets are tagged ci-fail, and they don't have an owner.

TODO: who will assign them to the best person to fix the ticket?

Owner: (none) (4 matches)

Ticket Summary Status Owner Reviewer Priority Severity Modified
#31921 Wrap our Travis commands with travis_retry, to mitigate network timeouts new Medium Normal 13 days ago
#30909 sbws consensus timestamp updates incorrectly use the current time needs_revision juga Medium Normal 3 weeks ago
#30235 Stem hangs waiting for control port data needs_information Medium Normal 4 months ago
#29437 test-stem times out intermittently assigned Medium Normal 4 months ago

Owner Action tickets

These tickets are tagged ci-fail, they have an owner, and they are in a status that needs action from the owner.

Anyone can change the assignments:

  • assign themselves one of these tickets,
  • un-assign ticket from themselves, or
  • ask someone else to take one of their tickets.

Owner: teor (1 match)

Ticket Summary Status Owner Reviewer Priority Severity Modified
#30901 Add control port trace logging to tor assigned teor nickm Medium Normal 12 days ago

Needs Review tickets

These tickets are tagged ci-fail, and they are in needs_review.

Ticket Summary Status Owner Reviewer Priority Severity Modified
No tickets found

Merge Ready tickets

These tickets are tagged ci-fail, and they are in merge_ready.

Milestone: Tor: 0.4.1.x-final (1 match)

Ticket Summary Status Owner Reviewer Priority Severity Modified
#31884 Define ExecuteBash in the Appveyor error block merge_ready teor asn Medium Normal 11 days ago

CI failures being diagnosed

As of 23 Sep 2019:

  • tor-ci-linux-0.2.9 (old-ish) transient; cleared up by itself in the following build

see https://jenkins.torproject.org/job/tor-ci-linux-0.2.9/292/console This looks like a travis failure. There's a java exception caused by a timeout, and no builders actually happen.

  • tor-ci-linux-0.4.1 (new-ish) transient; cleared up by itself in the following build

see https://jenkins.torproject.org/job/tor-ci-linux-0.4.1/ARCHITECTURE=amd64,SUITE=buster/28/console Buster amd64 is failing with what appears to be

08:50:53 08:50:53 FAIL: src/test/test_workqueue_efd.sh
08:50:53 ====================================
08:50:53 08:50:53 Illegal instruction
08:50:53 FAIL src/test/test_workqueue_efd.sh (exit status: 132)

I wonder if this is transient, or if it is a real problem? We didn't change this code any time recently.

  • tor-ci-linux-master-clang (new-ish) transient? seems to have fixed itself

https://jenkins.torproject.org/job/tor-ci-linux-master-clang/3640/ARCHITECTURE=i386,SUITE=sid/consoleFull

This is only happening on i686 sid, and it is a compiler segfault. This makes me think it is a bug in the clang package in debian unstable.

Looks like there's now a complaint about an unrecognized warning flag? -catalyst see #31769

15:39:37 cc1: error: command line option '-Wextra-semi' is valid for C++/ObjC++ but not for C [-Werror]
15:39:37 cc1: all warnings being treated as errors
15:39:37 make[1]: *** [Makefile:11538: src/feature/control/control_bootstrap.o] Error 1
  • tor-ci-mingwcross-* (old-ish)

https://jenkins.torproject.org/job/tor-ci-mingwcross-master/2499/ARCHITECTURE=amd64,SUITE=buster/console

This looks like a misconfiguration to me -- it seems to be complaining about not having found a zlib package maybe?

  • tor-ci-windows-master (old?)

https://jenkins.torproject.org/job/tor-ci-windows-master/78/console This looks like it could be a real bug: a unit test is failing. It's in "slow/process/callbacks" -- I believe ahf knows that code best.

Last modified 3 weeks ago Last modified on Sep 23, 2019, 2:44:15 PM