Changes between Initial Version and Version 1 of org/meetings/2014WinterDevMeeting/notes/TBBReleaseProcess


Ignore:
Timestamp:
Feb 17, 2014, 7:53:18 PM (6 years ago)
Author:
arma
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • org/meetings/2014WinterDevMeeting/notes/TBBReleaseProcess

    v1 v1  
     1TBB Release Process. Feb 17 2014.
     2
     3Release process currently: a new version of something comes out (e.g. Firefox), then Mike looks over the changes to see what needs fixing. At some point Mike makes a tag. Then Mike notifies people in an ad hoc way. Or Linus monitors the commit logs.
     4
     5Once your 'make' is done, 'make match' compares your sha256sums to those of other makers.
     6
     7Then 'make sign' to upload signatures of your packages.
     8
     9(Colin has some Selenium tests that he's working on. What's the status there?
     10
     11We also have access to the Mozilla Mercurial "tryserver" test servers. Only Mike's ssh key is authorized for now. Also they're the
     12standard Mozilla tests, and probably many of them fail on Tor Browser.)
     13
     14Then Mike writes a blog post, uploads to www-master, updates RecommendedTBBVersions.
     15
     16But: support has no visibility into impending releases, and as soon as a new one comes out, there's a support spike. Plus, the support team doesn't know how to resolve the new issues yet either.
     17
     18Tails has a habit where they can't mark a bug fixed until the documentation has been fixed.
     19
     20What do we mean by this documentation? Browser design document, web pages with screenshots, ...
     21
     22Videos in particular are out of date and hard to recreate.
     23
     24TBB doesn't know what issues the support team hears about.
     25
     26TODO: Make a list this week of all the documentation that still mentions Vidalia, mis-states Windows TBB as a zip file, etc.
     27
     28What part of the release process takes the most time?
     29
     30Answer: building, especially Firefox. It takes 24 to 48 hours before we get enough builds to have confidence. Though we can and do tell QA about builds once there's just one.
     31
     32Firefox has some sort of security fix every six weeks. Lately we've been doing TBB releases every two to three weeks.
     33
     34Erinn estimates 4-5 days of wall clock time to do a TBB release.
     35
     36Idea: rotating release duties. Solves the hit-by-bus problem, but also lets people focus on their actual work rather than being surprised by needing to do a release. Also it will force us to make a release checklist that is accurately documented.
     37
     38Blockers:
     39- Erinn still signs individual packages. (Teaching normal users how to check sha256sum files is not going to happen.) We could instead put a shared key online in a safe place, and it generates those signatures.
     40
     41- The release process involves updating the website version file, and so every releaser needs access for that.
     42
     43- Everyone needs access to www-master to actually publish packages.
     44
     45Integration with Support:
     46
     47Support team wants a calendar of releases plus the features that are planned to accompany them.
     48
     49Mike likes the independence of choosing at the last minute what other fixes to merge in.
     50
     51TODO: Comb through Trac tickets and make a list of known issues in normal English, intended for the support team.
     52
     53What about the roadmap? Does that constrain us too much?
     54
     55Roger guesses Lunar doesn't need a roadmap more than one release into the future. Also, incremental changelogs: once you've merged stuff, there should be a way for him to know about it.
     56
     57How should TBB team better predict upcoming releases of its components?
     58New Tors, Python fixes, etc shouldn't all be surprises.
     59
     60Support needs to have input into the 'known issues' list.
     61
     62QA woe: All our manual QA people use English. Oops.
     63
     64Should we get more people doing QA? Why not tell tor-talk about upcoming bundles? All our users are surprised by upcoming TBB releases, every time.
     65Erinn suggests a blog post publicizing tor-qa list.
     66
     67There's a tbb-testcase trac keyword for things that would be good for automation.
     68
     69When Mike put out 3.5.2, he unrecommended 3.5.1, which caused the spike in support. A delay in unrecommending the old version would
     70give more chance to find problems before forcing every user to notice. But of course that delays upgrades for (known) security issues.
     71
     72How to reduce time-to-release?
     73
     741) Only rebuild components when they have changed, so you don't recompile stuff that hasn't changed since last time. On the way.
     75But tradeoff: keeping the VM around, with state on it, opens us up for security issues. This is a fine example where simplicity may be better.
     76
     772) Nightlies will help make time-to-release faster, especially if the build triggers from the commit so you don't have to wait for the builder's human to wake up and launch it manually. Also nightlies will help with QA since some people will run the nightlies and find bugs early.
     78
     793) Publish hashes of partial build components?
     80
     81We need to get Nicolas a build machine that's as big as Erinn's, for nightlies among other things. Erinn has a plan for that.
     82
     83TBB team needs to strengthen their communication. And, how should the tbb team communicate with the support team?
     84
     85Spurious virus warnings! We need to submit all our packages to VirusTotal before release. In fact uploading to VirusTotal should be automated as part of the build-publish process, to make it happen as early as possible. Is uploading to VirusTotal sufficient to resolve all our QA false positives?