Opened 5 years ago

Closed 5 years ago

#12903 closed task (fixed)

Integrate obfs4proxy into the TBB bundle.

Reported by: yawning Owned by: yawning
Priority: Medium Milestone:
Component: Circumvention/Pluggable transport Version:
Severity: Keywords: obfs4, tbb, tbb-4.5-alpha
Cc: gk Actual Points:
Parent ID: #12130 Points:
Reviewer: Sponsor:

Description

I did some of the preliminary work based off dcf's meek branch a while ago, now that meek has been merged into TBB, and obfs4proxy is approaching what I consider deployable, it is the time to change my patch into a real branch.

Open questions that I need to figure out:

  • Is it ok to just mirror dependencies at people.tp.o/~yawning/mirrors/source?
  • My patch only edited versions.nightly since I wasn't (and still am not quite) at the point where I was comfortable tagging obfs4proxy. Is there a way to say "use this commit" for the other files?
  • The LICENSE file needs to be copied.

Child Tickets

Change History (15)

comment:1 in reply to:  description Changed 5 years ago by dcf

Replying to yawning:

I was comfortable tagging obfs4proxy. Is there a way to say "use this commit" for the other files?

Just set VERIFY_TAGS=0. Cf. https://gitweb.torproject.org/user/dcf/tor-browser-bundle.git/commitdiff/659a0d10206af8d1337b13967d32322f3693d996.

comment:2 in reply to:  description ; Changed 5 years ago by gk

Cc: gk added

Replying to yawning:

  • Is it ok to just mirror dependencies at people.tp.o/~yawning/mirrors/source?

I think the reason for doing so was, originally, just to be sure the dependencies are really available when needed. Thus, if you have reasons to fear/assume the infrastructure they are currently on is flaky etc. it might indeed be better to host them on tpo infra.

comment:3 in reply to:  2 Changed 5 years ago by yawning

Replying to dcf:

Replying to yawning:

I was comfortable tagging obfs4proxy. Is there a way to say "use this commit" for the other files?

Just set VERIFY_TAGS=0. Cf. https://gitweb.torproject.org/user/dcf/tor-browser-bundle.git/commitdiff/659a0d10206af8d1337b13967d32322f3693d996.

Ok. The nightly target currently fails to build due to things that I assume aren't my fault (it fails when adding the TBB specific tor patches) which is way before my added madness, so I'll try this with the alpha target next.

Replying to gk:

Replying to yawning:

  • Is it ok to just mirror dependencies at people.tp.o/~yawning/mirrors/source?

I think the reason for doing so was, originally, just to be sure the dependencies are really available when needed. Thus, if you have reasons to fear/assume the infrastructure they are currently on is flaky etc. it might indeed be better to host them on tpo infra.

A hah. The reason I put them in my ~ was that the gitian scripts currently don't support pulling code in from mercurial and go.net/go.crypto currently only live in hg on code.google.com. For now I'll keep doing things this way, but it should be easy to change later as needed.

comment:4 Changed 5 years ago by yawning

FWIW my current branch is now at: https://github.com/Yawning/tor-browser-bundle/tree/bug12903

Thanks to Mike providing me with access to a VM, I've verified that at least the Linux part works, though as I screwed up and left out a ';', the UI integration can't be tested past "it works when I copy/paste an obfs4 bridge line into the box", and "it uses obfs4prosy for obfs3".

TODO:

  • Ensure that the Macintoy/Wintendo builds at least finish. I have access to neither platforms currently so I can't test the output.
  • Redo the build with the fixed bridge_prefs.js file, and make sure obfs4 appears in the drop down and that it bootstraps.
  • Figure out how to add the LICENSE file to the bundle.
  • Finish #12606, tag a release and switch the targets to use said tag.
Version 0, edited 5 years ago by yawning (next)

comment:5 Changed 5 years ago by yawning

Ok, I need to regenerate the test bundles because obfs4proxy has changed a good amount and the bridgeline that people should be using for my test bridge also is slightly different. My last snapshot series should still work, but it won't accept custom bridge entries for a few reasons.

I will make it bundle the obfs4proxy license file and make another snapshot with the newer code soon-ish.

comment:6 Changed 5 years ago by yawning

Ok, I ended up just adding the obfs4 license to the LICENSE file that has all the other pt licenses. We're also currently missing the Go license, so that was added as LICENSE.GO (the runtime meek/obfs4 link against and some of the obfs4 dependencies all use that license).

Currently generating another set of bundles to make sure my changes are solid (also temporarily pulling in my #12535 patch for testing). Assuming the output looks good, all that remains here is to switch to using a taged version of obfs4proxy, and get more default obfs4 bridges.

comment:7 in reply to:  6 ; Changed 5 years ago by dcf

Replying to yawning:

Ok, I ended up just adding the obfs4 license to the LICENSE file that has all the other pt licenses. We're also currently missing the Go license, so that was added as LICENSE.GO (the runtime meek/obfs4 link against and some of the obfs4 dependencies all use that license).

You should probably break out the addition of the Go license to a separate ticket, because it's really should be a part of the bundles even without obfs4, and it will make the obfs4 integration diff smaller.

The diff at https://github.com/Yawning/tor-browser-bundle/tree/bug12903 looks pretty good to me.

comment:8 in reply to:  7 ; Changed 5 years ago by yawning

Replying to dcf:

Replying to yawning:

Ok, I ended up just adding the obfs4 license to the LICENSE file that has all the other pt licenses. We're also currently missing the Go license, so that was added as LICENSE.GO (the runtime meek/obfs4 link against and some of the obfs4 dependencies all use that license).

You should probably break out the addition of the Go license to a separate ticket, because it's really should be a part of the bundles even without obfs4, and it will make the obfs4 integration diff smaller.

Done as #13039, and I even attached a patch.

The diff at https://github.com/Yawning/tor-browser-bundle/tree/bug12903 looks pretty good to me.

I rebased on master, backed out the temporary goptlib SOCKS5 testing stuff, and changed my branch to only include the obfs4 LICENSE (as the Go license will be included with the patch attached to #13039).

As far as I can tell the required integration work is done apart from having new obfs4 and goptlib tags (once I finalize the SOCKS5 stuff) and probably different obfs4 bridges (instead of the one on my test box, though I might be able to keep that running, not sure yet).

comment:9 in reply to:  8 Changed 5 years ago by dcf

Replying to yawning:

As far as I can tell the required integration work is done apart from having new obfs4 and goptlib tags (once I finalize the SOCKS5 stuff) and probably different obfs4 bridges (instead of the one on my test box, though I might be able to keep that running, not sure yet).

I should be clear that I don't intend to merge #12535 (SOCKS5) to goptlib until after alpha bundles with the SOCK5 code have been released and gotten user testing. I want there to be a chance for bug reports to come in. The risk of screwing something up isn't high, but I would still rather see release–fix–merge than merge–release–fix. You should plan on using goptlib 0.2 in your #12903 patch.

comment:10 Changed 5 years ago by yawning

Aight. I'll have the #12903 code use SOCKS4a then, and at a later date file a ticket for alphas that pull in the newer SOCKS code. For the most part (apart from added features), the newer SOCKS implementation is a drop in replacement, so it's easy to test at a later date.

comment:11 Changed 5 years ago by yawning

Ok, per discussion in IRC and #12130 2 bridges may be enough for now for the branch to get merged.

Current issues:

  • I'm not sure if I want to be in the default TBB bridge business, but it's easy to remove my bridge as long as it's before we deploy this right?
  • As noted by dcf, obfs4proxy currently uses SOCKS4a. I changed the default obfs2/obfs3 providers from obfsproxy (python) to obfs4proxy for testing purposes, so that may need to be reverted if people use those with IPv6 (alternatively, my #12535 goptlib branch should be good to go, but won't get merged till it sees field testing in a TBB alpha). Using obfs4proxy where possible would be better because it implements the PT_PROXY stuff without py2exe brain damage.

gk I assume you're going to squash the branch if/when you merge it? Having a billion wip commits probably is neither useful nor interesting (apart from documenting my struggles with gitian for posterity). I could make a squashed branch as well.

comment:12 Changed 5 years ago by mikeperry

Keywords: tbb-4.5-alpha added

comment:13 Changed 5 years ago by mikeperry

Yawning - We're going to include this in a 4.5-alpha release on Thursday/Friday (which means we're building on Wednesday). Can you squash this down and remove any bridges you don't want in it?

comment:14 in reply to:  13 Changed 5 years ago by yawning

Replying to mikeperry:

Yawning - We're going to include this in a 4.5-alpha release on Thursday/Friday (which means we're building on Wednesday). Can you squash this down and remove any bridges you don't want in it?

gk made a squashed branch a while ago:
https://gitweb.torproject.org/user/gk/tor-browser-bundle.git/shortlog/refs/heads/bug_12903_squashed_v3

Assuming that branch is ok and I want to remove a bridge, should I make another branch and re-squash? Or is just telling someone which bridge to remove sufficient?

comment:15 Changed 5 years ago by mikeperry

Resolution: fixed
Status: newclosed

Ok, this branch was merged into origin/master.

Note: See TracTickets for help on using tickets.