Opened 4 years ago

Last modified 3 years ago

#15994 new task

Running Tor Browser unit tests as part of our gitian-based build process

Reported by: boklm Owned by: boklm
Priority: Medium Milestone:
Component: Applications/Quality Assurance and Testing Version:
Severity: Normal Keywords:
Cc: mcs, gk, brade Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

The Tor Browser patches add or modify some unit tests. Running those tests as part of the normal build process would allow us to test the main Tor Browser features using the same binaries that we ship. Running all Mozilla unit tests as part of the build would take a lot of time, so it's better done using Mozilla Try, but if we limit that to the tests that we add/modify, the time to run the tests is probably more reasonable.

Child Tickets

Change History (6)

comment:1 Changed 4 years ago by boklm

The branch bug15994-v0.1 in my user repository contains a script to run the unit tests:
https://gitweb.torproject.org/user/boklm/tor-browser-bundle.git/commit/?h=bug15994-v0.1

This script:

  • starts an Xorg server with a dummy driver and a window manager
  • uses git to find the tests that Tor Browser commits add or modify
  • run all modified/added unit tests and save logs
  • stops the Xorg server

What remains to be done:

  • remove --disable-tests from .mozconfig
  • add the packages used by the script and calls to the script in the linux gitian descriptor
  • add instructions to copy the tests logs archive to the build directory, and update gitian/hash-bundles.sh to not include them in sha256sums.txt
  • compare the build time with and without running the tests to see if that's acceptable

comment:2 Changed 4 years ago by mcs

Cc: mcs added

comment:3 Changed 4 years ago by gk

Cc: gk added

What additional time would that cost us?

comment:4 Changed 4 years ago by boklm

An other thing pointed by mikeperry during the weekly meeting: we should make sure that the unit tests cannot leak onto the non-tor net of a gitian builder.

comment:5 Changed 3 years ago by gk

Severity: Normal

I think this is the ideal thing for our nightly builds. At least we should try to get that deployed there first.

comment:6 Changed 3 years ago by mcs

Cc: brade added
Note: See TracTickets for help on using tickets.