Opened 6 years ago

Closed 4 years ago

#10006 closed task (wontfix)

Build an obfs-flash PT bundle

Reported by: dcf Owned by: dcf
Priority: Medium Milestone:
Component: Circumvention/Pluggable transport Version:
Severity: Keywords:
Cc: asn, infinity0 Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

Create a test pluggable transports bundle with the obfs3_flashproxy transport in place of the flashproxy transport.

Child Tickets

TicketStatusOwnerSummaryComponent
#9444closederinnCreate deterministic TorBrowserBundles with Pluggable TransportsApplications/Tor bundles/installation
#10005closedinfinity0obfs-flash-client incompatible with Twisted 12Circumvention/Pluggable transport
#10016closedasnTypeError in pyptlib subproc on WindowsCircumvention/Pluggable transport
#10017closedinfinity0Windows twisted.internet.stdio.StandardIO.__init__ lacks "stdin" keyword argument.Circumvention/Pluggable transport
#10088closedpyptlib - use JobObjects on windows to automatically kill PT children when PT itself diesCircumvention/Pluggable transport

Attachments (11)

proctree-1.png (19.3 KB) - added by dcf 6 years ago.
Process tree after staring Vidalia
proctree-2.png (4.6 KB) - added by dcf 6 years ago.
Process tree after stopping Vidalia
proctree-3.png (12.8 KB) - added by dcf 6 years ago.
Process tree after staring obfs-flash-client-env.bat from the Cygwin shell
proctree-4.png (6.8 KB) - added by dcf 6 years ago.
Process tree after ctrl-C of obfs-flash-client-env.bat from Cygwin shell
proctree-5.png (10.4 KB) - added by dcf 6 years ago.
Process tree after starting obfs-flash-client from the Cygwin shell
proctree-6.png (4.5 KB) - added by dcf 6 years ago.
Process tree after ctrl-C of obfs-flash-client from Cygwin shell
proctree-7.png (17.0 KB) - added by dcf 6 years ago.
Process tree after Vidalia starting obfs-flash-client directly (no batch file)
vidalia-notresponding.png (31.3 KB) - added by dcf 6 years ago.
How Vidalia looks when you try to restart it and an old obfsproxy process is running
proctree-8.png (2.3 KB) - added by dcf 6 years ago.
Process tree after stopping Vidalia (no batch file)
pca.2.png (15.7 KB) - added by dcf 6 years ago.
Program Compatibility Assistant popup
pca.png (15.7 KB) - added by dcf 6 years ago.
Program Compatibility Assistant popup

Download all attachments as: .zip

Change History (45)

comment:1 Changed 6 years ago by dcf

I have code for the gnulinux build here:

https://gitweb.torproject.org/pluggable-transports/bundle.git/shortlog/refs/heads/obfs-flash

The resulting bundles work, except for #10005. That is, if you unzip the bundle and edit the file App/obfs-flash-client to remove , reactor=reactor, then it works.

I currently have it set up to use the siteb facilitator, until the main facilitator is running the new transport-aware code. So in testing, you will want to open a browser to http://siteb.fp-facilitator.org/proxy/embed.html?debug&initial_facilitator_poll_interval=0.

obfs-flash-client finds out where obfsproxy and flashproxy-client are through environment variables. To get it to work, I made a wrapper script called obfs-flash-client-env.sh that sets the environment and execs obfs-flash-client. obfs-flash-client-env.sh is what gets called from torrc. There might be a better way to accomplish it.

comment:2 Changed 6 years ago by infinity0

Does windows have an env.exe? Then you could use that. I'll check it next time I get access to my windows machine. If that's not available, I'll add some extra flags to obfs-flash-client. I was trying to encourage people to use PATH, but I guess it's less feasible on windows.

This will be a non-issue once we do #9744, of course.

comment:3 in reply to:  2 Changed 6 years ago by dcf

Replying to infinity0:

Does windows have an env.exe? Then you could use that. I'll check it next time I get access to my windows machine. If that's not available, I'll add some extra flags to obfs-flash-client. I was trying to encourage people to use PATH, but I guess it's less feasible on windows.

Good thought. I just checked and there's no env for Windows (at least not such that is called "env".) There is set, that looks like it will work from a batch file. I'm going to try the Windows build tonight so I'll see what I can figure out.

Worst case, we can make a little Python wrapper and py2exe it.

comment:4 Changed 6 years ago by dcf

I think I now have the Windows build mostly sorted out, including a batch file wrapper obfs-flash-client-env.bat to set up the environment.

However, I hit a snag with #10017 that prevents obfs-flash-client from running for me, so I can't yet fully test.

Changed 6 years ago by dcf

Attachment: proctree-1.png added

Process tree after staring Vidalia

Changed 6 years ago by dcf

Attachment: proctree-2.png added

Process tree after stopping Vidalia

Changed 6 years ago by dcf

Attachment: proctree-3.png added

Process tree after staring obfs-flash-client-env.bat from the Cygwin shell

Changed 6 years ago by dcf

Attachment: proctree-4.png added

Process tree after ctrl-C of obfs-flash-client-env.bat from Cygwin shell

Changed 6 years ago by dcf

Attachment: proctree-5.png added

Process tree after starting obfs-flash-client from the Cygwin shell

Changed 6 years ago by dcf

Attachment: proctree-6.png added

Process tree after ctrl-C of obfs-flash-client from Cygwin shell

Changed 6 years ago by dcf

Attachment: proctree-7.png added

Process tree after Vidalia starting obfs-flash-client directly (no batch file)

Changed 6 years ago by dcf

Attachment: vidalia-notresponding.png added

How Vidalia looks when you try to restart it and an old obfsproxy process is running

comment:5 Changed 6 years ago by dcf

With the patch in #10017, I got the Windows build running and bootstrapped over obfs-flash proxies. Hooray!

However, I am having trouble with subprocesses continuing to run after Vidalia and Tor stop running. I consistently see obfsproxy.exe continuing to run, using the configuration that is currently in the obfs-flash bundle branch, where Tor invokes obfs-flash-client.exe directly, without an intermediate batch file. One process (obfsproxy.exe) is the fewest surviving processes I've been able to get; with other configuration I get some more; see below. The extra process stops Vidalia from being launched a second time.

Process Explorer is the tool I'm using to look at the process trees.

Vidalia, no batch file

This is what happen with the code currently in the obfs-flash branch of the bundle repo. Here is the process tree after Tor is fully bootstrapped and the browser window is open:

Process tree after Vidalia starting obfs-flash-client directly (no batch file)

Here is the process tree after closing the browser and exiting Vidalia. Notice that obfsproxy.exe is unparented and now its own root. All the other processes disappeared.

Process tree after stopping Vidalia (no batch file)

If I try to relaunch Vidalia while obfsproxy.exe is still running, it enters a "Not Responding" state and I have to forcibly kill it. It remains that way until I manually kill the old obfsproxy.exe process.

How Vidalia looks when you try to restart it and an old obfsproxy process is running

Vidalia with batch file

Originally I had a batch file wrapper invoking obfs-flash-client.exe. That turned out not to be necessary, but it may help to see how the processes looked in that configuration.

Here cmd.exe is the batch file.

Process tree after staring Vidalia

obfs-flash-client.exe and obfsproxy.exe survive. Interestingly, flashproxy-client.exe was killed, even though it was also started by obfs-flash-client.exe.

Process tree after stopping Vidalia

This situation is presumably a little worse than when only obfsproxy.exe survives, because obfs-flash-client.exe will still have bound the local port 2334.

I also tried a little Python wrapper instead of a batch file, and it made no difference.

Invoking obfs-flash-client.exe directly

You can reproduce it without building a full bundle. Here is how I tried running obfs-flash-client directly from a Cygwin shell.

TOR_PT_STATE_LOCATION=. TOR_PT_MANAGED_TRANSPORT_VER=1 TOR_PT_CLIENT_TRANSPORTS=obfs3_flashproxy ./App/obfs-flash-client --fp-arg=--facilitator=http://siteb.fp-facilitator.org/fac/ --fp-arg=--register-methods=http 2334 :9000

Process tree after starting obfs-flash-client from the Cygwin shell

This is how it looks after ctrl-C. This time obfs-flash-client.exe was killed, but flashproxy-client.exe survived alongside obfsproxy.exe. (They are now both root processes without a parent.)

Process tree after ctrl-C of obfs-flash-client from Cygwin shell

Invoking batch file directly

Finally, here is how it is slightly different if you call the batch file from the shell. The batch process is killed, but obfs-flash-client.exe and its two descendants survive.

TOR_PT_STATE_LOCATION=. TOR_PT_MANAGED_TRANSPORT_VER=1 TOR_PT_CLIENT_TRANSPORTS=obfs3_flashproxy ./App/obfs-flash-client-env.bat --fp-arg=--facilitator=http://siteb.fp-facilitator.org/fac/ --fp-arg=--register-methods=http 2334 :9000

Process tree after staring obfs-flash-client-env.bat from the Cygwin shell

After ctrl-C:

Process tree after ctrl-C of obfs-flash-client-env.bat from Cygwin shell


Are you able to reproduce the result of obfsproxy.exe surviving using one of the shell commands above? Do you have any ideas why obfsproxy.exe is surviving and flashproxy-client.exe is being killed? Maybe as a stopgap workaround, we could have obfs-flash-client be more forcible about killing its children after the first time it gets a signal.

Changed 6 years ago by dcf

Attachment: proctree-8.png added

Process tree after stopping Vidalia (no batch file)

comment:6 Changed 6 years ago by infinity0

Without looking in more detail, my suspicion is that this is #9330 showing up. When Tor kills obfs-flash-client using ProcessTerminate, exit handlers don't complete and so it never gets a chance to kill all the children completely (which would explain why you're seeming seemingly random children living/dying).

BTW, the PT spec states that you need to SIGINT *twice* for a PT to shut down. When you do Ctrl-C inside the terminal on GNU/Linux, it sends SIGINT to *all* processes including the children, which is not what you want since obfsproxy/flashproxy don't yet follow the spec and die on just one SIGINT - you must "kill -INT" the obfs-flash-client directly, twice. Alternatively, obfs-flash-client (and anything using pyptlib.util.subproc.auto_killall) will die and clean up on one SIGTERM.

I'm not sure how that affects Cygwin. But on windows yesterday, I was able to do a clean shutdown using a standalone obfs-flash-client with Ctrl-C twice (although I just did that in the console, so possibly it sent it to the children as well).

I'll explore this in more detail today.

Last edited 6 years ago by infinity0 (previous) (diff)

comment:7 in reply to:  6 ; Changed 6 years ago by dcf

Replying to infinity0:

Without looking in more detail, my suspicion is that this is #9330 showing up. When Tor kills obfs-flash-client using ProcessTerminate, exit handlers don't complete and so it never gets a chance to kill all the children completely (which would explain why you're seeming seemingly random children living/dying).

#9330 is a good find. I think this problem must be a bit deeper, though. The thing with Vidalia refusing to run a second time would have been noticed in a released bundle before now. What children survive isn't random; it is different in each configuration above, but it was reproducible within each configuration. (So in the first configuration, it was always obfsproxy.exe and only obfsproxy.exe that survived.) Also note that there are two obfsproxy.exe processes: one started by tor.exe for obfs2 and obfs3 (PID 1852 in the first graphic above), and one started by obfs-flash-client.exe for obfs3_flashproxy (PID 2144); and only the second one (PID 2144) is continuing to run. So I think it has something to do with the process nesting.

BTW, the PT spec states that you need to SIGINT *twice* for a PT to shut down. When you do Ctrl-C inside the terminal on GNU/Linux, it sends SIGINT to *all* processes including the children, which is not what you want since obfsproxy/flashproxy don't yet follow the spec and die on just one SIGINT - you must "kill -INT" the obfs-flash-client directly, twice. Alternatively, obfs-flash-client (and anything using pyptlib.util.subproc.auto_killall) will die and clean up on one SIGTERM.

I think one SIGINT suffices if the transport program is not currently handling any connections. At least that's how I read "Proxies should respond to a single INT signal by closing their listener ports and not accepting any new connections, but keeping all connections open, then terminating when connections are all closed." obfs-flash-server implements it as I understand it. But you're right, neither flashproxy-client nor obfsproxy seem to have any fancy signal handling, so it must not be the case that obfsproxy.exe is simply waiting for a second signal. C-obfsproxy used to have fancy SIGINT handling (#3473) and there is a ticket (#9741) for py-obfsproxy. Handling of SIGTERM (vs. SIGINT) is #9732. I should mention that in the Python wrapper I tested, I registered handlers for both SIGINT and SIGTERM, and never saw either (probably because of #9330).

I'm not sure how that affects Cygwin. But on windows yesterday, I was able to do a clean shutdown using a standalone obfs-flash-client with Ctrl-C twice (although I just did that in the console, so possibly it sent it to the children as well).

Ah, that's good news. What I'll do is upload one of the bundles I built, and then we can at least test whether this happens on other machines than mine.

The other thing I would try is to have obfs-flash-client spawn a third dummy process, both before and after the usual two, and see if it is killed, or it remains alongside obfsproxy.exe, or it remains and obfsproxy.exe does not, etc.

Last edited 6 years ago by dcf (previous) (diff)

comment:8 in reply to:  7 Changed 6 years ago by dcf

Replying to dcf:

What children survive isn't random; it is different in each configuration above, but it was reproducible within each configuration. (So in the first configuration, it was always obfsproxy.exe and only obfsproxy.exe that survived.)

Let me amend this; I have now seen it a few times where both children of obfs-flash-client.exe (obfsproxy.exe and flashproxy-client.exe) survived. The other obfsproxy.exe was still killed. It may be my imagination, but it seems to happen more often that both processes survive, when I configure torrc to remove all the obfs2 and obfs3 bridges.

comment:9 Changed 6 years ago by dcf

Mac OS bundle is A-OK.

As soon as we sort out the Windows issue, what I'd like to do is push a set of packages to tor-qa for testing.

comment:10 Changed 6 years ago by infinity0

Looking at this right now with my own slightly different setup and I see python.exe (for obfsproxy) remaining. It seems I was wrong before and the 2 Ctrl-Cs did not shut down everything.

comment:11 Changed 6 years ago by dcf

GITNE on IRC suggested that it might help to set CREATE_NEW_PROCESS_GROUP in Tor's call to CreateProcess. I tested this by patching the binary (don't tell). Unfortunately, it didn't seem to work.

Using the test bundle from https://people.torproject.org/~dcf/pt-bundle/2.4.17-beta-2-obfs-flash20131025/, a partial disassembly of the call to CreateProcess is

objdump -d tor.exe
  55a179:       c7 44 24 14 00 00 00    movl   $0x0,0x14(%esp)
  55a180:       00
  55a181:       c7 44 24 10 01 00 00    movl   $0x1,0x10(%esp)
  55a188:       00
  55a189:       c7 44 24 0c 00 00 00    movl   $0x0,0xc(%esp)
  55a190:       00
  55a191:       c7 44 24 08 00 00 00    movl   $0x0,0x8(%esp)
  55a198:       00
  55a199:       8b 45 f0                mov    -0x10(%ebp),%eax
  55a19c:       89 44 24 04             mov    %eax,0x4(%esp)
  55a1a0:       8b 85 74 ff ff ff       mov    -0x8c(%ebp),%eax
  55a1a6:       89 04 24                mov    %eax,(%esp)
  55a1a9:       e8 42 06 05 00          call   5aa7f0 <_CreateProcessA@40>

The first line sets dwCreationFlags. I changed byte 1415039 of the file from 00 to 02, so the disassembly becomes

objdump -d tor-hacked.exe
  55a179:       c7 44 24 14 00 02 00    movl   $0x200,0x14(%esp)
  55a180:       00
  55a181:       c7 44 24 10 01 00 00    movl   $0x1,0x10(%esp)
  55a188:       00
  55a189:       c7 44 24 0c 00 00 00    movl   $0x0,0xc(%esp)
  55a190:       00
  55a191:       c7 44 24 08 00 00 00    movl   $0x0,0x8(%esp)
  55a198:       00
  55a199:       8b 45 f0                mov    -0x10(%ebp),%eax
  55a19c:       89 44 24 04             mov    %eax,0x4(%esp)
  55a1a0:       8b 85 74 ff ff ff       mov    -0x8c(%ebp),%eax
  55a1a6:       89 04 24                mov    %eax,(%esp)
  55a1a9:       e8 42 06 05 00          call   5aa7f0 <_CreateProcessA@40>

The resulting binaries are

sha256sum tor.exe tor-hacked.exe
30099504de82281adde709a537e7ffe600d8c72021c9af459730f103b1376502  tor.exe
5aedaba498852ba0728193ee36aca21bcdd3975e71aeaff45362d1766930f0af  tor-hacked.exe

I copied tor-hacked.exe over tor.exe, ran the bundle, and then exited it. flashproxy-client.exe and obfsproxy.exe were still running.

I also tried tor-hacked.exe in combination with removing the CREATE_NEW_PROCESS_GROUP flag from subproc, supposing that maybe another new process group was insulating the children from the parent. That didn't seem to have any effect either.

comment:12 Changed 6 years ago by infinity0

In my interpretation, that is necessary but not sufficient - since Tor should not be doing TerminateProcess as its first course of action when trying to close a PT. See #9330 for more discussion.

I am also going through pyptlib's subproc module on windows in order to try to get as many tests passing as possible.

comment:13 Changed 6 years ago by dcf

Here are some things I found while web searching around, that may have been also touched on in IRC.

How do I automatically destroy child processes in Windows?
The Windows API supports objects called "Job Objects". The following code will create a "job" that is configured to shut down all processes when the main application ends (when its handles are cleaned up).

The code sample there uses Windows API functions:

These API function are available in the pywin32 library, which we already include in the browser bundle because it is a dependency of Twisted.

infinity0, what do you think of trying the CreateJobObject technique in subproc as a workaround until #9330 is fixed? The problem is that I foresee a solution to #9330 being several weeks off. I'm not sure if it will work, but if it does, (1) even if not 100% correct, it's correct enough to solve the "run Vidalia twice" problem, and (2) we can implement it locally, without necessitating new Tor and vanilla bundle releases.

The main difficulty I foresee is that the subprocess module may not give visibility to the handle under the Popen object (maybe msvcrt.get_osfhandle will work?). It'd be great to be able to test this idea, without completely rewriting subproc to use CreateProcess etc.

Last edited 6 years ago by dcf (previous) (diff)

comment:14 Changed 6 years ago by infinity0

This sounds feasible, I will have a go at it tomorrow. Thanks for the detailed research!

comment:15 Changed 6 years ago by dcf

I built bundles 2.4.17-beta-2-obfs-flash1 combining all our fixes so far, for posterity and so we have something concrete to share. They should work on GNU/Linux and Mac OS X, and on Windows have the problem of persistent subprocesses.

Packaged software's tags are as in https://gitweb.torproject.org/pluggable-transports/bundle.git/blob/d4632105e10e56e3d88c75fbc3d7bc2f78dfd924:/Makefile.

comment:16 Changed 6 years ago by infinity0

Early report that that JobObject thing works just as we want. :D I will tidy up the code and commit soon!

comment:17 Changed 6 years ago by infinity0

Code committed here: https://github.com/infinity0/pyptlib/commit/2c8cea7a077124b8e49ea45bd3886c6b2df1f09b

It does depend on a bunch of other changes (full diff from current tpo head) but they are mostly refactoring/bugfixes.. you can ignore/revert the ctrl-break stuff if you don't like it.

Runs (and dies correctly) for me on Windows XP but I did not test actually connecting to Tor due to the earlier "hanging" problems. Let me know if it works for you! We especially need to test on Windows Vista/7 due to that comment about Access Denied in that link, I only have XP at hand.

comment:18 Changed 6 years ago by infinity0

Oh, one thing, you should use this commit of obfs-flash. (I realise this is a bit untidy, suggestions for alternatives are welcome.)

Last edited 6 years ago by infinity0 (previous) (diff)

comment:19 Changed 6 years ago by asn

Hm. Do we want this merged in pyptlib master? Or should we keep it as a side patch for building PTTBs till we solve #9330?

comment:20 in reply to:  18 ; Changed 6 years ago by dcf

Replying to infinity0:

Oh, one thing, you should use this commit of obfs-flash. (I realise this is a bit untidy, suggestions for alternatives are welcome.)

It works for me from the command line as in comment:5. But it doesn't work in the bundle. In the Vidalia message log, I see obfs-flash-client emit "VERSION 1" but nothing else. I see it stop running after that. It doesn't seem to be spawning its subprocesses (or if it is, it's doing it so fast I can't see it in Process Explorer).

Traceback (most recent call last):
  File "obfs-flash-client", line 318, in <module>
  File "obfs-flash-client", line 313, in main
  File "obfs-flash-client", line 263, in obfs3_flashproxy
  File "obfs-flash-client", line 229, in pt_launch_child
  File "pyptlib\util\subproc.pyc", line 121, in __init__
pywintypes.error: (5, 'AssignProcessToJobObject', 'Access is denied.')
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "atexit.pyc", line 24, in _run_exitfuncs
  File "pyptlib\util\subproc.pyc", line 297, in killall
  File "twisted\internet\base.pyc", line 582, in stop
ReactorNotRunning: Can't stop reactor that isn't running.
Error in sys.exitfunc:
Traceback (most recent call last):
  File "atexit.pyc", line 24, in _run_exitfuncs
  File "pyptlib\util\subproc.pyc", line 297, in killall
  File "twisted\internet\base.pyc", line 582, in stop
twisted.internet.error.ReactorNotRunning: Can't stop reactor that isn't running.

I guess this is what you were referring to in comment:17, the same as in this discussion:

"On Vista, the Program Compatibility Assistant was added to help diagnose application compatibility problems. To allow us to accurately monitor applications, we leverage job objects for a certain class of applications. Specifically, this class of applications include anything launched from the Windows shell. Applications launched from a command-prompt are excluded."

(That explains why it works from the command line but not when you double-click.) Also from that page:

"If the application is marked with a UAC manifest, the application will be excluded permanently from the Program Compatibility Assistant."

Taking that advice, I tried adding a manifest to obfs-flash-client by adding this to setup.exe:

 setup(
-    console=["obfs-flash-client"],
+    console=[{
+        "script": "obfs-flash-client",
+        "uac_info": "asInvoker",
+    }],

It seems to properly embed the manifest (I can run mt.exe -inputresource:obfs-flash-client.exe -out:obfs-flash-client.manifest), but I get the same access error as before. Maybe flashproxy-client and obfsproxy need manifests too? I didn't try that. I also tried changing to "uac_info": "requireAdministrator"--it causes a little shield to appear on the obfs-flash-client.exe icon, and Tor fails to launch it with

Warning: Managed proxy at 'App\obfs-flash-client' failed at launch.

It seems like we're on the right track. Are you able to test embedding a manifest in flashproxy-client and obfsproxy? That's the next thing I'd try. Then you have to test it some way that doesn't involve running it from the command line (like building a wrapper that sets environment variables and runs obfs-flash-client).

comment:21 in reply to:  19 Changed 6 years ago by dcf

Replying to asn:

Hm. Do we want this merged in pyptlib master? Or should we keep it as a side patch for building PTTBs till we solve #9330?

I think having it in a side branch is fine. We don't want to be hasty committing to new code developed under time pressure. The patch also adds pywin32 as a dependency to pyptlib, which might not be desirable. As long as the branch exists in a repo somewhere at torproject.org, it's easy to build bundles from it.

comment:22 in reply to:  20 ; Changed 6 years ago by dcf

Replying to dcf:

It seems like we're on the right track. Are you able to test embedding a manifest in flashproxy-client and obfsproxy? That's the next thing I'd try. Then you have to test it some way that doesn't involve running it from the command line (like building a wrapper that sets environment variables and runs obfs-flash-client).

I tried doing the same uac_info thing for flashproxy-client.exe and obfsproxy.exe, and there was no change. I thought that maybe "Start Tor Browser.exe", vidalia.exe, and tor.exe might also need to have manifests, but I didn't know how to do that. I tried going into their properties, and put them in "Windows 7" compatibility mode, but nothing changed.

Also, when I did the test with "uac_info": "requireAdministrator" on obfs-flash-client.exe, I got this popup after closing Vidalia:

Program Compatibility Assistant popup

I ran it again, and this time a UAC dialog popped up when tor.exe tried to run obfs-flash-client.exe "...needs to make changes on the computer." I said yes, but the program failed as before. Now I wonder if the compatibility assistant is magically turned on for everything using the same path...

Last edited 6 years ago by dcf (previous) (diff)

Changed 6 years ago by dcf

Attachment: pca.2.png added

Program Compatibility Assistant popup

Changed 6 years ago by dcf

Attachment: pca.png added

Program Compatibility Assistant popup

comment:23 Changed 6 years ago by infinity0

Just did a force-push to remove a bit of an overengineered workaround - got rid of the "usesnewobject" flag.

Have not run into the UAC issues since I'm on XP, so managed to get everything working, and successfully connected to Tor!

One thing is that this workaround only works if you kill obfs-flash-client.exe or if tor.exe shuts down cleanly (to be able to kill obfs-flash-client.exe). If you kill tor.exe or if it crashes, obfs-flash-client.exe sticks around. We can solve that, e.g. by implementing 10047 but it's a fair bit of extra work.

I will look at the UAC issues today, but I'm on XP so will have a hard time reproducing it. I'll try to find a Vista/7 machine, though.

Oh and my setup involves calling obfsproxy with a batch file, the JobObject autokill stuff still works, so you ought to be able to put in as many wrappers as you want.

Last edited 6 years ago by infinity0 (previous) (diff)

comment:24 Changed 6 years ago by dcf

Here's maybe a dumb idea: If we suspect that flashproxy-client.exe is being killed because by default it logs a bunch of stuff to stderr, and its stderr gets closed (causing whatever the Windows equivalent of "broken pipe" is), what if we just had obfsproxy write some stuff to stderr?

I managed mode obfsproxy insists that logging be to a file. I tried hacking the debug log function to also print to stderr, but it didn't cause the process to die. I don't know if that's because the idea doesn't work, or because it wasn't logging anything.

comment:25 Changed 6 years ago by dcf

Also I think that #10047 would be an expedient and reasonable solution, one we can test in obfsproxy even before having to write up a specification.

comment:26 in reply to:  22 ; Changed 6 years ago by infinity0

Replying to dcf:

Replying to dcf:

It seems like we're on the right track. Are you able to test embedding a manifest in flashproxy-client and obfsproxy? That's the next thing I'd try. Then you have to test it some way that doesn't involve running it from the command line (like building a wrapper that sets environment variables and runs obfs-flash-client).

I tried doing the same uac_info thing for flashproxy-client.exe and obfsproxy.exe, and there was no change. I thought that maybe "Start Tor Browser.exe", vidalia.exe, and tor.exe might also need to have manifests, but I didn't know how to do that. I tried going into their properties, and put them in "Windows 7" compatibility mode, but nothing changed.

It's cool, we don't need to use UACs or any of that bullshit. We just need to have tor.exe start the PT with the CREATE_BREAKAWAY_FROM_JOB (0x01000000) flag. I do this in subproc.py, and the underlying reason is the same, explained in detail here.

You could try the trick you did above with tor-hacked.exe - change byte 1415041 to 0x01. (Perhaps also add CREATE_NEW_PROCESS_GROUP (0x00000200) for good karma, but I don't think it's necessary.)

Summary of that MSDN explanation, from subproc.py:

        # If we already belong to a JobObject, our children are auto-assigned
        # to that and AssignProcessToJobObject(ch, _chJob) fails. This flag
        # prevents this auto-assignment, as long as the parent JobObject has
        # JOB_OBJECT_LIMIT_BREAKAWAY_OK set on it as well.
        _Popen_creationflags |= _CREATE_BREAKAWAY_FROM_JOB

Two cases where "already belong to a JobObject" occurs:

  • in test_util_subproc.py, we spawn a main process that sometimes spawns a child. the main process already belongs to the JobObject of the test, since the test is using subproc.Popen too.
  • the Vista PCA system decides to assign nearly everything to its own JobObject. It seems this does indeed have JOB_OBJECT_LIMIT_BREAKAWAY_OK, so we can apply the same workaround.

So actually I already solved this problem, it's just cropping up in a different form. :p

Last edited 6 years ago by infinity0 (previous) (diff)

comment:27 in reply to:  26 Changed 6 years ago by dcf

Replying to infinity0:

Replying to dcf:

Replying to dcf:

It seems like we're on the right track. Are you able to test embedding a manifest in flashproxy-client and obfsproxy? That's the next thing I'd try. Then you have to test it some way that doesn't involve running it from the command line (like building a wrapper that sets environment variables and runs obfs-flash-client).

I tried doing the same uac_info thing for flashproxy-client.exe and obfsproxy.exe, and there was no change. I thought that maybe "Start Tor Browser.exe", vidalia.exe, and tor.exe might also need to have manifests, but I didn't know how to do that. I tried going into their properties, and put them in "Windows 7" compatibility mode, but nothing changed.

It's cool, we don't need to use UACs or any of that bullshit. We just need to have tor.exe start the PT with the CREATE_BREAKAWAY_FROM_JOB (0x01000000) flag. I do this in subproc.py, and the underlying reason is the same, explained in detail here.

You could try the trick you did above with tor-hacked.exe - change byte 1415041 to 0x01. (Perhaps also add CREATE_NEW_PROCESS_GROUP (0x00000200) for good karma, but I don't think it's necessary.)

Sweet, good find. I'll give this a try tonight.

comment:28 Changed 6 years ago by dcf

sha256sum tor.exe tor-01000000.exe tor-01000200.exe
30099504de82281adde709a537e7ffe600d8c72021c9af459730f103b1376502  tor.exe
95636e24137c526ce5e26dfb6fe5391605887ee4ec42c4da9b2ad3b49bd3075b  tor-01000000.exe
9da2504d30a7909e2f4cf696fe8b9c75f921f0a58e54dffeccb4187089925002  tor-01000200.exe
dwCreationFlags=0x01000200, UAC enabled in obfs-flash, flashproxy-client, obfsproxy
Works! All processes terminated.
dwCreationFlags=0x01000000, UAC enabled in obfs-flash, flashproxy-client, obfsproxy
Works!
dwCreationFlags=0x01000200, UAC enabled in obfs-flash only
Works!
dwCreationFlags=0x01000000, UAC enabled in obfs-flash only
Works!
dwCreationFlags=0x01000200, UAC not enabled anywhere
Works!
dwCreationFlags=0x01000000, UAC not enabled anywhere
Works!

So it seems that CREATE_BREAKAWAY_FROM_JOB in tor.exe is the important thing, and neither CREATE_NEW_PROCESS_GROUP nor a UAC in the subprocesses matters.

comment:29 Changed 6 years ago by infinity0

Great! I've updated the build scripts to apply that patch automatically. I built it on wine and tested it on Windows XP, and pleased to say it works out-of-the-box. :D

comment:30 Changed 6 years ago by asn

FWIW, I made #10088 for the source-code implementation of the CREATE_BREAKAWAY_FROM_JOB idea.

comment:31 Changed 6 years ago by infinity0

Parent ID: #7167#9444

I believe all major issues within obfs-flash have been dealt with that might affect a deployment - we've tested it with the previous non-deterministic PTTBB on all platforms. The only thing left, is to add this into the deterministic gitian build of PTTBB (soon-to-be just "TBB").

Therefore, I'm re-parenting this ticket from #7167 to #9444.

comment:32 Changed 6 years ago by infinity0

We should do some more testing about #10082 and #10242, see how critical they are for a normal user session.

comment:33 Changed 6 years ago by dcf

Parent ID: #9444

Reversing the parent relationship; we want to be able to close #9444 without having to close this one first.

comment:34 Changed 4 years ago by dcf

Resolution: wontfix
Status: newclosed

I'm closing this because we built a test bundle (comment:15) and it doesn't look as though this project will get further.

Note: See TracTickets for help on using tickets.