Opened 5 years ago

Closed 4 years ago

#8416 closed defect (fixed)

Programmatize PT bundle building

Reported by: dcf Owned by: dcf
Priority: Medium Milestone:
Component: Obfuscation/Pluggable transport Version:
Severity: Keywords: flashproxy
Cc: erinn, asn, aallai, adrelanos@… Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

We have good instructions for taking a TBB and turning it into a PT bundle:

https://gitweb.torproject.org/flashproxy.git/blob/HEAD:/doc/bundle-gnulinux.txt
https://gitweb.torproject.org/flashproxy.git/blob/HEAD:/doc/bundle-macosx.txt
https://gitweb.torproject.org/flashproxy.git/blob/HEAD:/doc/bundle-windows.txt

But the instructions are manual and tedious to follow. They should rather be shell scripts.

The scripts can assume all the necessary components are already in place in some well-known locations. The torrc fragments need to be broken out into separate files.

The PT transport bundle is wider than flash proxy, so the build scripts don't really belong in the flash proxy repository. But they can go there for now.

Child Tickets

Change History (8)

comment:1 Changed 5 years ago by dcf

Related to #8644, I want these scripts to have separate variables TBB_VERSION, which will be used to name and find the TBB archive to extract into a temporary directory, and PT_BUNDLE_RELEASE, which will be used to append an extra version number to our filenames. We will increment PT_BUNDLE_RELEASE with each batch of bundles we make with the same TBB_VERSION, and reset it to 1 whenever TBB_VERSION is increased.

comment:2 Changed 5 years ago by dcf

Keywords: flashproxy added

comment:3 Changed 5 years ago by dcf

Owner: changed from asn to dcf
Status: newassigned

I'm going to try this as I reproduce Alex's bundles from today.

comment:4 Changed 5 years ago by dcf

Resolution: fixed
Status: assignedclosed

This is as automated now as I want it. The procedure is now

  1. Run make dist in flashproxy as before.
  2. Update version numbers at the top of doc/Makefile.
  3. Run one of:
    • make windows
    • make macosx-i686
    • make gnulinux-i686
    • make gnulinux-x86_64

The makefile does the downloading, extracting, copying, and rezipping.

comment:5 Changed 5 years ago by weasel

Resolution: fixed
Status: closedreopened

Good work!

A couple comments/wishlist items:

  • You probably shouldn't rely on gpg's exit codes, ever. gpgv is there if you really need this, else maybe grep gpg's --status-fd output.
  • It'd be nice if there was a means to tell the makefile to never ever download stuff.
  • find | xargs constructs probably want -print0 | xargs -0, or you use -exec rm {} \+
  • Can you make the installed location of pyptlib configurable? Right now building the bundle would require root.

comment:6 Changed 5 years ago by proper

Cc: adrelanos@… added

When using gpg --keyring, it's better to also add --no-default-keyring.

comment:7 Changed 4 years ago by dcf

Status: reopenedneeds_review

What do you think of the current state in https://gitweb.torproject.org/pluggable-transports/bundle.git?

In building the 2.4.12-alpha-2-pt1 bundles, I made some changes that make the process a bit nicer.

  1. Only the fetch-* targets download from the network, and they are not run implicitly.
  2. pyptlib doesn't have to be installed. Like flashproxy and obfsproxy, you just need to have a clone of it somewhere.
  3. The makefile makes another temporary local clone of each of those three clones. The temporary clones are blown away on each rebuild and the originals are never written to. This also means that you can do e.g. git checkout 1.1 to build against a certain tag, even if there are newer commits in the repository.
  4. Uses find -print0 and gpgv.

The manual steps are:

  • Installing a build environment and dependencies (bundle-*.txt instructions);
  • Making clones of pyptlib, obfsproxy, and flashproxy; and
  • Downloading and verifying a plain TBB.

Then automatic steps done by the makefile are

  • Extracting the TBB;
  • Making a copy of each of pyptlib, obfsproxy, and flashproxy, and building them;
  • Copying pyptlib, obfsproxy, and flashproxy files;
  • Modifying torrc; and
  • Re-zipping the bundle with a pluggable transports file name.

comment:8 Changed 4 years ago by dcf

Resolution: fixed
Status: needs_reviewclosed
Note: See TracTickets for help on using tickets.