Add snowflake pt to alpha linux builds
Please see the attached.
Reproducibility of cgo depends on a compiler that supports,
-gno-record-gcc-switches
-fdebug-prefix-map=$WORK=/tmp/go-buildwhich landed in go 1.7.3
(see https://github.com/golang/go/commit/5bbb98df0960f57dca73cb7640456608d4cc0917)
Activity
Here is some feedback for the attached patch:
- I'd like to have an enhanced commit message. At least the bug number should be a part.
- I heard there were snags (comment:19:ticket:19001) like the need for
apt-get install curl pkg-config libgtk2.0-dev libglib2.0-dev. I hit that one, too. This should be handled in ourcheck-prerequisites.shscript. - Snowflake is not working:
2016/11/25 08:28:57 BrokerChannel Error: No snowflake proxies currently available. 2016/11/25 08:28:57 Failed to retrieve answer. Retrying in 10 seconds 2016/11/25 08:29:07 Negotiating via BrokerChannel... Target URL: snowflake-reg.appspot.com Front URL: www.google.com 2016/11/25 08:29:07 BrokerChannel Response: 503 Service Unavailable- Is there really no way to avoid cluttering the disk with all that stuff to just build the webrtc part? Think about us builders who usually have a dev, hardened, alpha and stable tor-browser-bundle tree to get releases faster out.
Trac:
Status: needs_review to needs_revision
Keywords: TorBrowserTeam201611R deleted, N/A added-
- Snowflake is not working:
The browser proxy has not been distributed widely yet, so noone is coming along in time to answer your offer. Try visiting https://keroserene.net/snowflake/snowflake.html in another browser simultaneously, or run one locally https://gitweb.torproject.org/pluggable-transports/snowflake.git/tree/proxy/README.md#n18
-
- Is there really no way to avoid cluttering the disk with all that stuff to just build the webrtc part? Think about us builders who usually have a dev, hardened, alpha and stable tor-browser-bundle tree to get releases faster out.
Yeah, webrtc is a beast to build. In the short term, this ticket is only about the alpha channel. In the long term, switching to rbm will ameliorate the issue, in that it makes better use of reusable components.
But, your objection is noted and we'll try and think of how to improve the situation.
Trac:
Cc: serene, dcf, arma to serene, dcf, arma, boklm -
- I'd like to have an enhanced commit message. At least the bug number should be a part.
- I heard there were snags (comment:19:ticket:19001) like the need for apt-get install curl pkg-config libgtk2.0-dev libglib2.0-dev. I hit that one, too. This should be handled in our check-prerequisites.sh script.
Ok, see the newly attached
0001-Bug-20735-Add-snowflake-pt-to-alpha-Linux-builds.patch -
Replying to arlolra:
- Is there really no way to avoid cluttering the disk with all that stuff to just build the webrtc part? Think about us builders who usually have a dev, hardened, alpha and stable tor-browser-bundle tree to get releases faster out.
Yeah, webrtc is a beast to build. In the short term, this ticket is only about the alpha channel. In the long term, switching to rbm will ameliorate the issue, in that it makes better use of reusable components.
But, your objection is noted and we'll try and think of how to improve the situation.
FWIW: it is not an objection in the sense that a thing needs to get fixed to get the patch merged. I am just annoyed that this thing causes quite a big part of my hard disk filled with crap I don't want (and IMO should not be needed in an ideal world) :)
rbmnot really mitigates this concern (although, sure, I don't have to check out all of this four times). -
Replying to arlolra:
- Snowflake is not working:
The browser proxy has not been distributed widely yet, so noone is coming along in time to answer your offer. Try visiting https://keroserene.net/snowflake/snowflake.html in another browser simultaneously, or run one locally https://gitweb.torproject.org/pluggable-transports/snowflake.git/tree/proxy/README.md#n18
So, how are users in our alphas then supposed to use Snowflake? Wouldn't it be smarter to have, say, Snowflake in our nightly builds first for a while, propagate that to get the browser proxy more widely distributed and then ship it in alphas? Right now all the users get is Snowflake not working which is sort of counterintuitive if we ship it in an alpha build (as showing it in the PT drop down box indicates that it is something which is actually usable).
-
Wouldn't it be smarter to have, say, Snowflake in our nightly builds first for a while, propagate that to get the browser proxy more widely distributed and then ship it in alphas?
Maybe there's a misunderstanding here.
The proxies (snowflakes) are run by random internet users in uncensored regions. They just need to navigate to a certain website and then their browser takes over. Distributing it widely means hosting the badge on say, torproject.org, or some other highly trafficked site.
It's the same model as flashproxy (https://crypto.stanford.edu/flashproxy/#how-it-works).
-
Replying to arlolra:
Wouldn't it be smarter to have, say, Snowflake in our nightly builds first for a while, propagate that to get the browser proxy more widely distributed and then ship it in alphas?
Maybe there's a misunderstanding here.
I don't think so.
The proxies (snowflakes) are run by random internet users in uncensored regions. They just need to navigate to a certain website and then their browser takes over. Distributing it widely means hosting the badge on say, torproject.org, or some other highly trafficked site.
Yes, let's do that. What I want to avoid is alpha users having basically zero chances of getting a working Snowflake and coming instead to us claiming it is broken.
-
I don't think so.
Ok, I guess I was confused by your "propagate that to get the browser proxy more widely distributed". The
snowflake-clientbuilt here is independent of the proxy. Anyways, we're on the same page. Sorry.Yes, let's do that. What I want to avoid is alpha users having basically zero chances of getting a working Snowflake and coming instead to us claiming it is broken.
Gotcha, see #20813 (moved)
-
Replying to arlolra:
I don't think so.
Ok, I guess I was confused by your "propagate that to get the browser proxy more widely distributed". The
snowflake-clientbuilt here is independent of the proxy. Anyways, we're on the same page. Sorry.No worries, I should have framed it better. :)
Yes, let's do that. What I want to avoid is alpha users having basically zero chances of getting a working Snowflake and coming instead to us claiming it is broken.
Gotcha, see #20813 (moved)
Thanks. How about this: We get snowflake into the nightlies for now and plan to ship it into the alpha after the next one (which is due in January 2017). And you are trying to take care of #20813 (moved) meanwhile? We need to freeze the code for the next alpha next week and it seems to me that timeframe might be too short to get #20813 (moved) done sufficiently. Or am I wrong in that regard?
-
How about this: We get snowflake into the nightlies for now and plan to ship it into the alpha after the next one (which is due in January 2017).
Sounds good. To be clear, is it the next one that goes out in January, or the one after? (Words are hard.)
And you are trying to take care of #20813 (moved) meanwhile?
Yes, I've initiated a discussion with the team.
We need to freeze the code for the next alpha next week and it seems to me that timeframe might be too short to get #20813 (moved) done sufficiently. Or am I wrong in that regard?
No, that's probably fair to say.
-
Replying to arlolra:
How about this: We get snowflake into the nightlies for now and plan to ship it into the alpha after the next one (which is due in January 2017).
Sounds good. To be clear, is it the next one that goes out in January, or the one after? (Words are hard.)
Indeed. So, the next release will be on Dec 13/Dec 14. And snowflake could then get included in the alpha that goes out in January.
-
I am sorry about that but could you rebase your patch one last time? I needed to get the sandbox bundling and building landed for our SponsorU deadline and it changed a bunch of things that are affecting the snowflake integration.
Trac:
Keywords: TorBrowserTeam201612R deleted, N/A added
Status: needs_review to needs_revision -
Replying to serene:
Hi -- checking in on this.
@gk: Do you have everything you need for the integration, and are we on track for the January alpha release?
Yes, and yes, thanks. I just did the annual christmas traveling and had some days off which is the only reason for this delay. I applied the patch to
masterand merged the previous one intohardened-builds(ashardened-buildsdoes not ship the sandbox). The result is commit 761b0dbabd8fd14e8d9149634b52869f0c68375b, f78866cad71122fa851ebdf6c63053605c565826 and 2eca5d42245a98b4572943b0307a74392bfa4f5c.I was a bit confused as the last patch from arlolra indicated that it is Patch 1/3. (That's what is contained in the Subject line). Thus, if there is anything else that needs to get addressed on our side let me know.
Trac:
Status: needs_review to closed
Resolution: N/A to fixed -
I was a bit confused as the last patch from arlolra indicated that it is Patch 1/3. (That's what is contained in the Subject line). Thus, if there is anything else that needs to get addressed on our side let me know.
Sorry, patches 2 and 3 are the macOS and Windows bits, respectively, which we aren't doing here yet (see https://github.com/arlolra/tor-browser-bundle/commits/sf-set). So, no, nothing else to address.
Thanks for merging!
-
Our nightly build is failing for some reason (ln5 reported "sha256sum: webrtc.tar.gz: No such file or directory" and "fatal: ambiguous argument 'master': unknown revision or path not in the working tree."). I tried to look at it on my regular build machine and things are already broken while fetching the source code:
ACTION REQUIRED: Before running `gclient sync|runhooks` again, you must run: src/setup_links.py --force Which will replace all directories which now must be symlinks, after prompting with a summary of the work-to-be-done. Error: Command '/usr/bin/python src/setup_links.py' returned non-zero exit status 1 in /home/gk/dev/gitian-builder/inputs/webrtc Makefile:97: recipe for target 'prep-nightly' failed make: *** [prep-nightly] Error 2I wonder if there is something that changed here recently as I remember building Snowflake on that machine but then got rid of all the related sources again.
Could you double-check that a clean tor-browser-bundle checkout (commit 761b0dbabd8fd14e8d9149634b52869f0c68375b) + a
make prep-nightlyis working for you? -
Ok, confirmed that I get the same thing as you. And yes, this is the result of a recent upstream change.
I believe the problem is that
fetch --nohooks --no-history webrtccallsgclient syncwithout passing a revision (-r), and therefore checkouts the head, which we then up reverting in the next step.Working on a fix.
-
Okay, this works for me (as in:
make prep-nightlydoes not break anymore) and is applied tomasterandhardened-builds(commit 3cf7b5af18a4fa0ad0813c0e16148dc1e578c04a and 96836418e9686ea775759bbe15c39cbc3855969d).Trac:
Status: reopened to closed
Resolution: N/A to fixed
Keywords: TorBrowserTeam201612R deleted, TorBrowserTeam201701R added mentioned in issue #20813 (moved)
mentioned in issue #21185 (closed)
mentioned in issue tpo/anti-censorship/pluggable-transports/snowflake#20813 (closed)