While PTs + proxy are working fine on Linux and Mac OS X they are broken on Windows:
12.06.2014 14:22:10.893 [WARN] Managed proxy failed to configure the pluggable transport's outgoing proxy. (We don't have txsocksx. Can't do proxy. Please install txsocksx.) 12.06.2014 14:22:10.894 [WARN] Managed proxy at 'Tor\PluggableTransports\obfsproxy' failed the configuration protocol and will be destroyed.
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information
Child items ...
Show closed items
Linked items 0
Link issues together to show that they're related.
Learn more.
This isn't a packaging issue, it's a obfsproxy build one. Apparently I forgot to add txsocksx in the setup_py2exe.py's includes list. I'm not sure if py2exe is smart enough to pick up parsely if only txsocksx is added.
fteproxy's build process also needs similar changes, but I'm not sure where the best place to put them are.
Something to chalk up to "none of the obfsproxy developers do development on Windows".
This isn't a packaging issue, it's a obfsproxy build one. Apparently I forgot to add txsocksx in the setup_py2exe.py's includes list. I'm not sure if py2exe is smart enough to pick up parsely if only txsocksx is added.
Hm, that should be easy to fix. However, I'm also not sure whether py2exe will recursively determine the dependencies of txsocksx, and I can't find the answer in the py2exe docs:
http://www.py2exe.org/index.cgi/ListOfOptions
I also don't know whether I can specify a lowest module version, as we are doing with pyptlib in setup.py: pyptlib >= 0.0.6.
OK. For obfsproxy, I made a branch that adds parsley and txsocksx to the py2exe dependencies.
Let's see if it works! You can find it in bug12381 at https://git.torproject.org/pluggable-transports/obfsproxy.git.
For fteproxy, I sent an email to Kevin. If he does not respond, I will look into it.
Trac: Cc: asn, yawning to asn, yawning, kpdyer Status: assigned to needs_review
+ cd obfsproxy + patch -tp1 patching file setup_py2exe.py + find -type f + xargs touch '--date=2000-01-01 00:00:00' + LD_PRELOAD= + wine /home/ubuntu/install/python/python.exe setup_py2exe.py py2exe fixme:msvcrt:MSVCRT__wsopen_s : pmode 0x81b6 ignored running py2exe^M creating Z:\home\ubuntu\build\obfsproxy\py2exe_bundle^M creating Z:\home\ubuntu\build\obfsproxy\py2exe_bundle\build^M creating Z:\home\ubuntu\build\obfsproxy\py2exe_bundle\build\bdist.win32^M creating Z:\home\ubuntu\build\obfsproxy\py2exe_bundle\build\bdist.win32\winexe^M creating Z:\home\ubuntu\build\obfsproxy\py2exe_bundle\build\bdist.win32\winexe\bundle-2.7^M creating Z:\home\ubuntu\build\obfsproxy\py2exe_bundle\build\bdist.win32\winexe\temp^M creating Z:\home\ubuntu\build\obfsproxy\py2exe_bundle\dist^M *** searching for required modules ***^M Traceback (most recent call last):^M File "setup_py2exe.py", line 18, in <module>^M "dist_dir": dist_path,^M File "Z:\home\ubuntu\install\python\lib\distutils\core.py", line 152, in setup^M dist.run_commands()^M File "Z:\home\ubuntu\install\python\lib\distutils\dist.py", line 953, in run_commands^M self.run_command(cmd)^M File "Z:\home\ubuntu\install\python\lib\distutils\dist.py", line 972, in run_command^M cmd_obj.run()^M File "Z:\home\ubuntu\install\python\lib\site-packages\py2exe\build_exe.py", line 243, in run^M self._run()^M File "Z:\home\ubuntu\install\python\lib\site-packages\py2exe\build_exe.py", line 296, in _run^M self.find_needed_modules(mf, required_files, required_modules)^M File "Z:\home\ubuntu\install\python\lib\distutils\dist.py", line 972, in run_command^M cmd_obj.run()^M File "Z:\home\ubuntu\install\python\lib\site-packages\py2exe\build_exe.py", line 243, in run^M self._run()^M File "Z:\home\ubuntu\install\python\lib\site-packages\py2exe\build_exe.py", line 296, in _run^M self.find_needed_modules(mf, required_files, required_modules)^M File "Z:\home\ubuntu\install\python\lib\site-packages\py2exe\build_exe.py", line 1297, in find_needed_modules^M mf.import_hook(mod)^M File "Z:\home\ubuntu\install\python\lib\site-packages\py2exe\mf.py", line 719, in import_hook^M return Base.import_hook(self,name,caller,fromlist,level)^M File "Z:\home\ubuntu\install\python\lib\site-packages\py2exe\mf.py", line 136, in import_hook^M q, tail = self.find_head_package(parent, name)^M File "Z:\home\ubuntu\install\python\lib\site-packages\py2exe\mf.py", line 204, in find_head_package^M raise ImportError, "No module named " + qname^M ImportError: No module named txsocksx^M
I've merged the tor-browser-bundle bits in the commits 5c307a89c22cfd2d3b38d8215539324213fafcfb, 5400da654020a34edb9edee70a0583a89231c4fe (both master) and 34aedfbae28ed04ad7fe1df0695d246e8e14313a (maint-3.6).
asn, kpdyer: We need rather soonish new tags for obfsproxy and fte adding txsockxs and parsley to the requirements in setup_py2exe.py and setup_tbb.py respectively. Otherwise we would have to do the patching in the descriptor itself for the time being, a thing I'd rather like to avoid...
I've merged the tor-browser-bundle bits in the commits 5c307a89c22cfd2d3b38d8215539324213fafcfb, 5400da654020a34edb9edee70a0583a89231c4fe (both master) and 34aedfbae28ed04ad7fe1df0695d246e8e14313a (maint-3.6).
asn, kpdyer: We need rather soonish new tags for obfsproxy and fte adding txsockxs and parsley to the requirements in setup_py2exe.py and setup_tbb.py respectively. Otherwise we would have to do the patching in the descriptor itself for the time being, a thing I'd rather like to avoid...
Hmm, I think asn might be traveling right now. We tagged a new obfsproxy release a few days ago but it looks like asn didn't merge bug12381 before we tagged. asn? (The latest release is mostly scramblesuit related changes, not sure if TBB ships that PT yet or not.).
I've merged the tor-browser-bundle bits in the commits 5c307a89c22cfd2d3b38d8215539324213fafcfb, 5400da654020a34edb9edee70a0583a89231c4fe (both master) and 34aedfbae28ed04ad7fe1df0695d246e8e14313a (maint-3.6).
asn, kpdyer: We need rather soonish new tags for obfsproxy and fte adding txsockxs and parsley to the requirements in setup_py2exe.py and setup_tbb.py respectively. Otherwise we would have to do the patching in the descriptor itself for the time being, a thing I'd rather like to avoid...
Okay, here is a debug log showing what is going on:
2014-07-23 15:34:25,217 [WARNING] Obfsproxy (version: unknown) starting up.2014-07-23 15:34:25,217 [DEBUG] argv: ['Tor\\PluggableTransports\\obfsproxy', '--log-min-severity=debug', '--log-file=C:\\Users\\AMO_Test\\Desktop\\log.log', 'managed']2014-07-23 15:34:25,217 [DEBUG] args: Namespace(data_dir=None, log_file='C:\\Users\\[scrubbed]\\Desktop\\log.log', log_min_severity='debug', name='managed', no_log=False, no_safe_logging=False, proxy=None)2014-07-23 15:34:25,217 [INFO] Entering client managed-mode.2014-07-23 15:34:25,217 [DEBUG] pyptlib gave us the following data:'{'__class__': <class 'pyptlib.client.ClientTransportPlugin'>, 'config': {'allTransportsEnabled': False, 'managedTransportVer': ['1'], 'proxy': SplitResult(scheme='http', netloc='[scrubbed]', path='', query='', fragment=''), 'stateLocation': 'C:\\Users\\[scrubbed]\\Tor Browser\\Data\\Tor\\pt_state/', 'transports': ['obfs2', 'obfs3']}, 'served_transports': ['obfs2', 'obfs3'], 'served_version': '1', 'stdout': <open file '<stdout>', mode 'w' at 0x00436078>}'2014-07-23 15:34:25,217 [INFO] OBFSSOCKSv5Factory starting on 492992014-07-23 15:34:25,217 [INFO] Starting factory <obfsproxy.network.socks.OBFSSOCKSv5Factory instance at 0x02CCD030>2014-07-23 15:34:25,217 [DEBUG] socks_fact_0x2ccd030: Starting up SOCKS server factory.2014-07-23 15:34:25,217 [DEBUG] Successfully launched 'obfs2' at '[scrubbed]'2014-07-23 15:34:25,217 [INFO] OBFSSOCKSv5Factory starting on 493002014-07-23 15:34:25,217 [INFO] Starting factory <obfsproxy.network.socks.OBFSSOCKSv5Factory instance at 0x02CCD1C0>2014-07-23 15:34:25,217 [DEBUG] socks_fact_0x2ccd1c0: Starting up SOCKS server factory.2014-07-23 15:34:25,217 [DEBUG] Successfully launched 'obfs3' at '[scrubbed]'2014-07-23 15:34:25,217 [INFO] Starting up the event loop.2014-07-23 15:34:26,917 [DEBUG] socks_fact_0x2ccd1c0: New connection.2014-07-23 15:34:27,010 [DEBUG] socks_fact_0x2ccd1c0: New connection.2014-07-23 15:34:27,072 [DEBUG] socks_fact_0x2ccd1c0: New connection.2014-07-23 15:34:27,151 [DEBUG] socks_fact_0x2ccd1c0: New connection.2014-07-23 15:34:27,229 [ERROR] Unhandled ErrorTraceback (most recent call last): File "twisted\python\log.pyc", line 88, in callWithLogger File "twisted\python\log.pyc", line 73, in callWithContext File "twisted\python\context.pyc", line 118, in callWithContext File "twisted\python\context.pyc", line 81, in callWithContext--- <exception caught here> --- File "twisted\internet\selectreactor.pyc", line 151, in _doReadOrWrite File "twisted\internet\tcp.pyc", line 215, in doRead File "twisted\internet\tcp.pyc", line 221, in _dataReceived File "obfsproxy\network\socks5.pyc", line 170, in dataReceived File "obfsproxy\network\socks5.pyc", line 242, in processAuthentication File "obfsproxy\network\socks5.pyc", line 139, in <lambda> File "obfsproxy\network\socks5.pyc", line 328, in processNoAuthRequired File "obfsproxy\network\socks5.pyc", line 396, in processRequest File "obfsproxy\network\socks5.pyc", line 411, in processCmdConnect File "obfsproxy\network\socks.pyc", line 156, in connectClass File "obfsproxy\network\network.pyc", line 409, in create_proxy_client File "twisted\internet\endpoints.pyc", line 42, in <module> File "twisted\internet\stdio.pyc", line 28, in <module> File "twisted\internet\_win32stdio.pyc", line 7, in <module>exceptions.ImportError: No module named win32api
My try to make obfsproxy working with a proxy started here: !https://trac.torproject.org/projects/tor/ticket/12381
gk's log said that win32api is missing. So I've compiled obfsproxy on Windows, changing setup_py2exe.py to include pywin32's win32api.
After replacing 3.6.3's obfsproxy with my compiled one, I got the following error on log:
!------------LOG PIECE START---------------!2014-07-26 14:43:15,470 [ERROR] Unhandled ErrorTraceback (most recent call last): File "twisted\python\log.pyc", line 88, in callWithLogger File "twisted\python\log.pyc", line 73, in callWithContext File "twisted\python\context.pyc", line 118, in callWithContext File "twisted\python\context.pyc", line 81, in callWithContext--- <exception caught here> --- File "twisted\internet\selectreactor.pyc", line 151, in _doReadOrWrite File "twisted\internet\tcp.pyc", line 215, in doRead File "twisted\internet\tcp.pyc", line 221, in _dataReceived File "obfsproxy\network\socks5.pyc", line 170, in dataReceived File "obfsproxy\network\socks5.pyc", line 242, in processAuthentication File "obfsproxy\network\socks5.pyc", line 139, in <lambda> File "obfsproxy\network\socks5.pyc", line 328, in processNoAuthRequired File "obfsproxy\network\socks5.pyc", line 396, in processRequest File "obfsproxy\network\socks5.pyc", line 411, in processCmdConnect File "obfsproxy\network\socks.pyc", line 156, in connectClass File "obfsproxy\network\network.pyc", line 396, in create_proxy_client File "txsocksx\client.pyc", line 12, in <module> File "parsley.pyc", line 3, in <module> File "ometa\grammar.pyc", line 32, in <module> File "ometa\grammar.pyc", line 25, in loadGrammarexceptions.IOError: [Errno 2] No such file or directory: 'C:\\[MyPath]\\TorBrowser\\Tor\\PluggableTransports\\obfsproxy.zip\\ometa\\parsley_termactions.parsley'2014-07-26 14:43:15,471 [DEBUG] socks_up_0x2336e50: Connection was lost ([Errno 2] No such file or directory: 'C:\\[MyPath]\\TorBrowser\\Tor\\PluggableTransports\\obfsproxy.zip\\ometa\\parsley_termactions.parsley').2014-07-26 14:43:15,471 [DEBUG] socks_up_0x2336e50: Closing connection.2014-07-26 14:43:15,471 [DEBUG] circ_0x2348eb8: Tearing down circuit.!------------LOG PIECE END---------------!
So, I've looked on the zip file, and I didn't find that file. So, I'have downloaded every *.parsley file from here (!https://github.com/python-parsley/parsley/tree/master/ometa) and injected them inside obfsproxy.zip, into "/ometa/". No lucky, it haven't worked.
On that moment that I thought a simple idea: Why not create a folder "obfsproxy.zip" and extract Obfsproxy's py2exe files to it? I did it, and the error was another:
!------------LOG PIECE START---------------!2014-07-26 15:17:04,184 [ERROR] Unhandled ErrorTraceback (most recent call last): File "twisted\python\log.pyc", line 88, in callWithLogger File "twisted\python\log.pyc", line 73, in callWithContext File "twisted\python\context.pyc", line 118, in callWithContext File "twisted\python\context.pyc", line 81, in callWithContext--- <exception caught here> --- File "twisted\internet\selectreactor.pyc", line 151, in _doReadOrWrite File "twisted\internet\tcp.pyc", line 215, in doRead File "twisted\internet\tcp.pyc", line 221, in _dataReceived File "obfsproxy\network\socks5.pyc", line 170, in dataReceived File "obfsproxy\network\socks5.pyc", line 242, in processAuthentication File "obfsproxy\network\socks5.pyc", line 139, in <lambda> File "obfsproxy\network\socks5.pyc", line 328, in processNoAuthRequired File "obfsproxy\network\socks5.pyc", line 396, in processRequest File "obfsproxy\network\socks5.pyc", line 411, in processCmdConnect File "obfsproxy\network\socks.pyc", line 156, in connectClass File "obfsproxy\network\network.pyc", line 409, in create_proxy_client File "twisted\internet\endpoints.pyc", line 42, in <module> File "twisted\internet\stdio.pyc", line 28, in <module> File "twisted\internet\_win32stdio.pyc", line 15, in <module> File "twisted\internet\_pollingfile.pyc", line 104, in <module> File "pywintypes.pyc", line 124, in <module> File "pywintypes.pyc", line 61, in __import_pywin32_system_module__exceptions.ImportError: Module 'pywintypes' isn't in frozen sys.path ['C:\\[MyPath]\\TorBrowser\\Tor', 'C:\\[MyPath]\\TorBrowser\\Tor\\PluggableTransports\\obfsproxy.zip']2014-07-26 15:17:04,184 [DEBUG] socks_up_0x23f3d50: Connection was lost (Module 'pywintypes' isn't in frozen sys.path ['C:\\[MyPath]\\TorBrowser\\Tor', 'C:\\[MyPath]\\TorBrowser\\Tor\\PluggableTransports\\obfsproxy.zip']).2014-07-26 15:17:04,184 [DEBUG] socks_up_0x23f3d50: Closing connection.2014-07-26 15:17:04,184 [DEBUG] circ_0x2429d50: Tearing down circuit.!------------LOG PIECE END---------------!
After doing some research, I found this link (!http://stackoverflow.com/questions/17891071/importerror-with-cx-freeze-and-pywin32-module-pythoncom-isnt-in-frozen-sys-p) and a user suggested to put pythoncom33.dll on the build directory. I've searched for that file in python's directory and found a folder with 3 dll files (pycom27.dll, pycomloader27.dll, pywintypes27.dll) in (C:\Python27\Lib\site-packages\pywin32_system32). So, thinking on the "put on the build directory", I've put these DLL's on my new obfsproxy.zip folder and:
!------------LOG PIECE START---------------!2014-07-26 15:22:01,470 [ERROR] Unhandled ErrorTraceback (most recent call last): File "twisted\python\log.pyc", line 88, in callWithLogger File "twisted\python\log.pyc", line 73, in callWithContext File "twisted\python\context.pyc", line 118, in callWithContext File "twisted\python\context.pyc", line 81, in callWithContext--- <exception caught here> --- File "twisted\internet\selectreactor.pyc", line 151, in _doReadOrWrite File "twisted\internet\tcp.pyc", line 215, in doRead File "twisted\internet\tcp.pyc", line 221, in _dataReceived File "obfsproxy\network\socks5.pyc", line 170, in dataReceived File "obfsproxy\network\socks5.pyc", line 242, in processAuthentication File "obfsproxy\network\socks5.pyc", line 139, in <lambda> File "obfsproxy\network\socks5.pyc", line 328, in processNoAuthRequired File "obfsproxy\network\socks5.pyc", line 396, in processRequest File "obfsproxy\network\socks5.pyc", line 411, in processCmdConnect File "obfsproxy\network\socks.pyc", line 156, in connectClass File "obfsproxy\network\network.pyc", line 410, in create_proxy_client File "txsocksx\client.pyc", line 12, in <module> File "parsley.pyc", line 5, in <module> File "terml\parser.pyc", line 88, in <module> File "ometa\grammar.pyc", line 25, in loadGrammarexceptions.IOError: [Errno 2] No such file or directory: 'C:\\[MyPath]\\TorBrowser\\Tor\\PluggableTransports\\obfsproxy.zip\\terml\\terml.parsley'2014-07-26 15:22:01,470 [DEBUG] socks_up_0x2458c30: Connection was lost ([Errno 2] No such file or directory: 'C:\\[MyPath]\\TorBrowser\\Tor\\PluggableTransports\\obfsproxy.zip\\terml\\terml.parsley').2014-07-26 15:22:01,470 [DEBUG] socks_up_0x2458c30: Closing connection.2014-07-26 15:22:01,470 [DEBUG] circ_0x24751c0: Tearing down circuit.!------------LOG PIECE END---------------!
I came back to parsley GitHub's repository, downloaded all *.parsley files from "/terml/", and put them on the obfsproxy.zip folder, and:
Obfsproxy worked sucessfully with an HTTP Proxy!
I don't know if my solution is correct, but it's working, which is a great advance. Thanks!
Realistically the PT team needs to decide this is important at this point, I think. I'm not even sure this will matter anymore with TBB 4.5 and obfs4 anyway.
Realistically the PT team needs to decide this is important at this point, I think. I'm not even sure this will matter anymore with TBB 4.5 and obfs4 anyway.
I think with TBB 4.5/obfs4proxy, the only PT that this affects will be fte, so it doesn't matter much. This naturally assumes I merge my scramblesuit obfs4proxy branch by then (which I should). I'm open to being convinced otherwise, but fighting python packaging/deterministic build issues isn't all that fun.
I'm confused --- does anything need to be fixed in fteproxy?
No. I was just documenting that fteproxy will be the only obfsproxy user in TBB 4.5 (once I merge/tag some code), so it will be the only PT affected by the gigantic mess that is py2exe/python on Windows.
(obfsX/ScrambleSuit will all use a different Golang codebase that is a lot easier to build/package.)
Yes, but have you any indication this is different in Tor Browser 8?
Trac: Summary: Pluggable Transports + proxy is not working on Windows with TBB 3.6.2 to Pluggable Transports + proxy is not working on Windows with Tor Browser Resolution: wontfix toN/A Status: closed to reopened