Tor won't start on Windows when path contains non-ascii characters

changed milestone to %Tor: unspecified

added 042-should 043-deferred 044-deferred component::core tor/tor milestone::Tor: unspecified owner::ahf parent::25729 points::1 priority::high reporter::iktsuarpok severity::major status::assigned tbb-needs tor-client type::defect unicode version::tor 0.2.3.25 win32 windows labels

To reproduce - run it from Windows Explorer

Trac:
Username: iktsuarpok

Just to confirm -- you have this problem with TBB 3.5 also, yes? https://blog.torproject.org/blog/tor-browser-bundle-35rc1-released

What version of TBB is this? Does a more recent series (3.0 or 3.5) have the same issue?

Could this be a duplicate of #9593 (moved) ?

Trac:
Status: new to needs_information

@nickm When I created this issue I used tor-browser-2.3.25-15_de.exe.

I couldn't create exactly identical conditions to check because of 3.5rc1 changes (compared to tor-browser-2.3.25-15). At first sight it looks like TBB still doesn't process unicode paths correctly on win.

Here's what I did:

Downloaded https://archive.torproject.org/tor-package-archive/torbrowser/3.5rc1/torbrowser-install-3.5-rc-1_de.exe
Installed it to C:\Users\kuncaŭščyna\Downloads\3.5rc1 (instead it installed itself to C:\Users\kuncauscyna\Downloads\3.5rc1 having created the necessary path)
I moved the installed files back to C:\Users\kuncaŭščyna\Downloads\3.5rc1
Ran it from Win Explorer, it said that Tor was unexpectedly closed.

To compare I ran it from "ascii-only" path (I copied the data to "C:\Users\kuncauscyna;)\Downloads\3.5rc1\Tor Browser") and ran it from there in Win Explorer. It said that it couldn't find libevent*.dll. I closed Tor, opened Far, run

set PATH=Tor;%PATH%

and restarted "Start Tor Browser.exe" from Far. It opened firefox (== Tor Browser) without problems. Then I did the same for "C:\Users\kuncaŭščyna\Downloads\3.5rc1\Tor Browser" (set PATH and restarted from Far). But it still said that Tor was unexpectedly closed.

So I concluded that I can run TorBroswer-3.5rc1 from "C:\Users\kuncauscyna;)\Downloads\3.5rc1\Tor Browser" but not from "C:\Users\kuncaŭščyna\Downloads\3.5rc1\Tor Browser" i.e. I can run it only from ascii-only paths.

(I'm not yet used to the new UI and couldn't find the logs in 3.5rc1, so not sure what exactly went wrong)

Trac:
Username: iktsuarpok

Trac:
Username: iktsuarpok
Status: needs_information to new

TBB people -- can we trace what command line Tor is getting in this situation? Are we losing the non-ascii characters between the launcher and Tor, or between Tor and the filesystem call?

Also, could somebody refresh me on why this needs to be an absolute path? I know I've had it explained to me before, but I didn't see the explanation here or on #9593 (moved) when I checked.

Trac:
Cc: N/A to helix, mikeperry, mcs

I can confirm, nor TBB with Vidalia nor TBB3 do not works for Windows XP if path to torrc file contains non ascii chars.

for Windows XP if path to torrc file contains non ascii chars.

Those was non localized version without locales, so not sure it's reproducible for some European version with proper locales.

Replying to nickm:

TBB people -- can we trace what command line Tor is getting in this situation? Are we losing the non-ascii characters between the launcher and Tor, or between Tor and the filesystem call?

(parenthetical comment: If I place the downloaded exe (self-extracting installer) in a folder that contains non-ASCII characters in its name, I see "Error launching installer". So I had to extract it onto the Desktop and then move the "Tor Browser" folder into the non-ASCII folder. So maybe the installer has trouble with non-ASCII paths too.)

To see some debugging from Tor Launcher, edit ...\Tor Browser\Data\Browser\profile.default\prefs.js and add these two lines:

user_pref("extensions.torlauncher.loglevel", 0); user_pref("extensions.torlauncher.logmethod", 0);

Then open a cmd prompt and cd to ...\Tor Browser\Browser. For non-ASCII characters to be displayed correctly in a console window, you need to set the font to Lucida Console (which can be done from the console window's menu) and you should set the code page to UTF-8 by executing this command: chcp 65001

Then start the browser like this: firefox -console -no-remote

A console window should open and you will debugging output from Tor Launcher, including a dump of the args that are passed to tor. The problem is, I cannot tell if the paths being passed to tor are correct or not because the console window does not have the correct code page. I will have to devise a way to capture the command line args (maybe I can temporarily replace tor.exe with something that does just that).

Also, could somebody refresh me on why this needs to be an absolute path? I know I've had it explained to me before, but I didn't see the explanation here or on #9593 (moved) when I checked.

Tor Launcher passes absolute paths to tor because Tor Launcher cannot change the current working directory, and therefore the only (seemingly) safe choice is to use absolute paths. Given the kind of problem exposed by this bug though, it would be nice if it could figure out a way to use relative paths.

No problem with Vidalia or Torlauncher, they passes agruments in proper way, but Tor can handle only non unicode argv that OS converted using configured code-page. Proper way is to drop ~~windows~~ legacy code that depends code-page and to use unicode windows specific stuff and funcs.

Tor works with non-ascii stuff if used chars can be converted with configured code-page. It working for me with non-ascii chars for localized windows, but can't to start with reported 'kuncaŭščyna' directory even if language (via control panel, that changes code-page for non-unicode programs) defined as German. Everything works if renamed directory to word that can be converted and used for german code-page, for example: 'Europäische'

The kuncaŭščyna directory was used only for testing. (This is a name in Belarusian Latin alphabet - there's no code page for this alphabet, imo). I'm not sure how often this problem may occur in reality, but here's what actually happened. I was reported about this problem by a friend who moved to Italy (and I guess she changed her locale but not her login). She simply downloaded the TBB (latest stable release for en_US) into the default location (C:\Users<user>\Downloads) unpacked the exe and ran the launcher. It didn't start though and she didn't have any clue as to what was wrong. When I looked into her log, I saw the message: Unable to open configuration file: C:\Users????\Downloads... But with the new release candidate I might be also puzzled, because it doesn't show log, it simply says that "Tor was unexpectedly closed". So maybe if it's difficult to add proper unicode support on windows (I understand that windows differs much from unices), it's possible to notify the user about the actual problem along with instruction to move the files somewhere else (as an option).

Trac:
Username: iktsuarpok

the log message (stable release) for the issue

Trac:
Username: iktsuarpok

Replying to iktsuarpok:

... When I looked into her log, I saw the message: Unable to open configuration file: C:\Users????\Downloads... But with the new release candidate I might be also puzzled, because it doesn't show log, it simply says that "Tor was unexpectedly closed". So maybe if it's difficult to add proper unicode support on windows (I undestand that windows differs much from other unices), it's possible to notify the user about the actual problem along with instruction to move the files somewhere else (as an option).

I agree that better error reporting is needed. Unfortunately, Tor Launcher does not currently have a way to capture all of the log messages generated by tor. See #10059 (moved).

Trac:
Cc: helix, mikeperry, mcs to helix, mikeperry, mcs, brade

Replying to cypherpunks:

No problem with Vidalia or Torlauncher, they passes agruments in proper way, but Tor can handle only non unicode argv that OS converted using configured code-page. Proper way is to drop ~~windows~~ legacy code that depends code-page and to use unicode windows specific stuff and funcs.

Hm. If we can do this by just handling unicode command-line arguments (s/main/wmain/), this won't be hard. If we need to change every place that we touch the file system or manipulate a path, that will be a bit harder.

Hm. If we can do this by just handling unicode command-line arguments (s/main/wmain/), this won't be hard. If we need to change every place that we touch the file system or manipulate a path, that will be a bit harder.

If legacy program could to set UTF-8 code page to use by legacy functions that would solve all problems without any changes, but no such method documented and Microsoft want to drop code pages support someday in every piece of shipped code.

Windows native way is to use "wide" functions with "wide" chars to store stuff. More simpler solution for less changes is to convert UTF-16 arguments to UTF-8 and convert back only for windows UTF-16 funcs, but it would to break output and logs.

Trac:
Priority: normal to major
Keywords: windows, unicode deleted, tor-client windows unicode added
Milestone: N/A to Tor: 0.2.6.x-final

Adding 'erinn' to cc list of every ticket with 'helix' in its cc list -- erinn is helix's trac username.

Trac:
Cc: helix, mikeperry, mcs, brade to helix, mikeperry, mcs, brade, erinn

Tor won't start on Windows when path contains non-ascii characters

Child items 0

Activity