Integrate obfs4proxy into the TBB bundle.
I did some of the preliminary work based off dcf's meek branch a while ago, now that meek has been merged into TBB, and obfs4proxy is approaching what I consider deployable, it is the time to change my patch into a real branch.
Open questions that I need to figure out:
- Is it ok to just mirror dependencies at
people.tp.o/~yawning/mirrors/source? - My patch only edited
versions.nightlysince I wasn't (and still am not quite) at the point where I was comfortable tagging obfs4proxy. Is there a way to say "use this commit" for the other files? - The LICENSE file needs to be copied.
Activity
Replying to yawning:
I was comfortable tagging obfs4proxy. Is there a way to say "use this commit" for the other files?
Just set VERIFY_TAGS=0. Cf. https://gitweb.torproject.org/user/dcf/tor-browser-bundle.git/commitdiff/659a0d10206af8d1337b13967d32322f3693d996.
Replying to yawning:
- Is it ok to just mirror dependencies at
people.tp.o/~yawning/mirrors/source?
I think the reason for doing so was, originally, just to be sure the dependencies are really available when needed. Thus, if you have reasons to fear/assume the infrastructure they are currently on is flaky etc. it might indeed be better to host them on tpo infra.
Trac:
Cc: N/A to gk- Is it ok to just mirror dependencies at
Replying to dcf:
Replying to yawning:
I was comfortable tagging obfs4proxy. Is there a way to say "use this commit" for the other files?
Just set VERIFY_TAGS=0. Cf. https://gitweb.torproject.org/user/dcf/tor-browser-bundle.git/commitdiff/659a0d10206af8d1337b13967d32322f3693d996.
Ok. The nightly target currently fails to build due to things that I assume aren't my fault (it fails when adding the TBB specific tor patches) which is way before my added madness, so I'll try this with the alpha target next.
Replying to gk:
Replying to yawning:
- Is it ok to just mirror dependencies at
people.tp.o/~yawning/mirrors/source?
I think the reason for doing so was, originally, just to be sure the dependencies are really available when needed. Thus, if you have reasons to fear/assume the infrastructure they are currently on is flaky etc. it might indeed be better to host them on tpo infra.
A hah. The reason I put them in my ~ was that the gitian scripts currently don't support pulling code in from mercurial and
go.net/go.cryptocurrently only live in hg on code.google.com. For now I'll keep doing things this way, but it should be easy to change later as needed.- Is it ok to just mirror dependencies at
FWIW my current branch is now at: https://github.com/Yawning/tor-browser-bundle/tree/bug12903
Thanks to Mike providing me with access to a VM, I've verified that at least the Linux part works, though as I screwed up and left out a ';', the UI integration can't be tested past "it works when I copy/paste an obfs4 bridge line into the box", and "it uses obfs4prosy for obfs3".
TODO:
Ensure that the Macintoy/Wintendo builds at least finish. I have access to neither platforms currently so I can't test the output.Redo the build with the fixed bridge_prefs.js file, and make sure obfs4 appears in the drop down and that it bootstraps.- Figure out how to add the LICENSE file to the bundle.
- Finish #12606 (moved), tag a release and switch the targets to use said tag.
Edit: As far as I am aware (and to the limit of what myself and a few friends could test), the output from my branch works as intended.
Ok, I need to regenerate the test bundles because obfs4proxy has changed a good amount and the bridgeline that people should be using for my test bridge also is slightly different. My last snapshot series should still work, but it won't accept custom bridge entries for a few reasons.
I will make it bundle the obfs4proxy license file and make another snapshot with the newer code soon-ish.
Ok, I ended up just adding the obfs4 license to the LICENSE file that has all the other pt licenses. We're also currently missing the Go license, so that was added as LICENSE.GO (the runtime meek/obfs4 link against and some of the obfs4 dependencies all use that license).
Currently generating another set of bundles to make sure my changes are solid (also temporarily pulling in my #12535 (moved) patch for testing). Assuming the output looks good, all that remains here is to switch to using a taged version of obfs4proxy, and get more default obfs4 bridges.
Replying to yawning:
Ok, I ended up just adding the obfs4 license to the LICENSE file that has all the other pt licenses. We're also currently missing the Go license, so that was added as LICENSE.GO (the runtime meek/obfs4 link against and some of the obfs4 dependencies all use that license).
You should probably break out the addition of the Go license to a separate ticket, because it's really should be a part of the bundles even without obfs4, and it will make the obfs4 integration diff smaller.
The diff at https://github.com/Yawning/tor-browser-bundle/tree/bug12903 looks pretty good to me.
Replying to dcf:
Replying to yawning:
Ok, I ended up just adding the obfs4 license to the LICENSE file that has all the other pt licenses. We're also currently missing the Go license, so that was added as LICENSE.GO (the runtime meek/obfs4 link against and some of the obfs4 dependencies all use that license).
You should probably break out the addition of the Go license to a separate ticket, because it's really should be a part of the bundles even without obfs4, and it will make the obfs4 integration diff smaller.
Done as #13039 (moved), and I even attached a patch.
The diff at https://github.com/Yawning/tor-browser-bundle/tree/bug12903 looks pretty good to me.
I rebased on master, backed out the temporary goptlib SOCKS5 testing stuff, and changed my branch to only include the obfs4 LICENSE (as the Go license will be included with the patch attached to #13039 (moved)).
As far as I can tell the required integration work is done apart from having new obfs4 and goptlib tags (once I finalize the SOCKS5 stuff) and probably different obfs4 bridges (instead of the one on my test box, though I might be able to keep that running, not sure yet).
Replying to yawning:
As far as I can tell the required integration work is done apart from having new obfs4 and goptlib tags (once I finalize the SOCKS5 stuff) and probably different obfs4 bridges (instead of the one on my test box, though I might be able to keep that running, not sure yet).
I should be clear that I don't intend to merge #12535 (moved) (SOCKS5) to goptlib until after alpha bundles with the SOCK5 code have been released and gotten user testing. I want there to be a chance for bug reports to come in. The risk of screwing something up isn't high, but I would still rather see release–fix–merge than merge–release–fix. You should plan on using goptlib 0.2 in your #12903 (moved) patch.
Aight. I'll have the #12903 (moved) code use SOCKS4a then, and at a later date file a ticket for alphas that pull in the newer SOCKS code. For the most part (apart from added features), the newer SOCKS implementation is a drop in replacement, so it's easy to test at a later date.
Ok, per discussion in IRC and #12130 (moved) 2 bridges may be enough for now for the branch to get merged.
Current issues:
- I'm not sure if I want to be in the default TBB bridge business, but it's easy to remove my bridge as long as it's before we deploy this right?
- As noted by dcf, obfs4proxy currently uses SOCKS4a. I changed the default obfs2/obfs3 providers from obfsproxy (python) to obfs4proxy for testing purposes, so that may need to be reverted if people use those with IPv6 (alternatively, my #12535 (moved) goptlib branch should be good to go, but won't get merged till it sees field testing in a TBB alpha). Using obfs4proxy where possible would be better because it implements the PT_PROXY stuff without py2exe brain damage.
gk I assume you're going to squash the branch if/when you merge it? Having a billion wip commits probably is neither useful nor interesting (apart from documenting my struggles with gitian for posterity). I could make a squashed branch as well.
Replying to mikeperry:
Yawning - We're going to include this in a 4.5-alpha release on Thursday/Friday (which means we're building on Wednesday). Can you squash this down and remove any bridges you don't want in it?
gk made a squashed branch a while ago: https://gitweb.torproject.org/user/gk/tor-browser-bundle.git/shortlog/refs/heads/bug_12903_squashed_v3
Assuming that branch is ok and I want to remove a bridge, should I make another branch and re-squash? Or is just telling someone which bridge to remove sufficient?
mentioned in issue #13039 (moved)
mentioned in issue tpo/applications/tor-browser#13039 (closed)
