Opened 5 years ago

Last modified 3 months ago

#10416 new defect

Tor won't start on Windows when path contains non-ascii characters

Reported by: iktsuarpok Owned by:
Priority: High Milestone: Tor: unspecified
Component: Core Tor/Tor Version: Tor: 0.2.3.25
Severity: Normal Keywords: tor-client, windows, unicode, win32
Cc: mikeperry, mcs, brade, erinn, Executorvoodoo Actual Points:
Parent ID: #25729 Points:
Reviewer: Sponsor:

Description

When trying to start the TBB from the following path:

C:\Users\kuncaŭščyna\Downloads\Tor Browser\

got the following error:

Vidalia detected that the Tor software exited unexpectedly.
Please check the message log for recent warning or error messages.

Here's the log:

Dez 16 18:27:21.431 [Hinweis] Tor v0.2.3.25 (git-17c24b3118224d65) running on Windows 7.
Dez 16 18:27:21.431 [Hinweis] Tor can't help you if you use it wrong! Learn how to be safe at https://www.torproject.org/download/download#warning
Dez 16 18:27:21.431 [Warnung] Unable to open configuration file "C:\Users\kuncauscyna\Downloads\Tor Browser\Data\Tor\torrc".
Dez 16 18:27:21.431 [Fehler] Reading config failed--see warnings above.

Diacritics were obviously "simplified" which created inexisting path.

I know Windows is not recommened for anonymity. But some people use TBB on Win only for censorship circumvention. The easiest solution is simply to add a message to log like:

Unable to open configuration file "C:\Users\kuncauscyna\Downloads\Tor Browser\Data\Tor\torrc". Please check that the path contains only ascii symbols.

But much better would be to fix it to process correctly path in unicode.

Child Tickets

Attachments (1)

tor_nonascii_log.jpg (41.5 KB) - added by iktsuarpok 5 years ago.
the log message (stable release) for the issue

Download all attachments as: .zip

Change History (40)

comment:1 Changed 5 years ago by iktsuarpok

To reproduce - run it from Windows Explorer

comment:2 Changed 5 years ago by arma

Just to confirm -- you have this problem with TBB 3.5 also, yes?
https://blog.torproject.org/blog/tor-browser-bundle-35rc1-released

comment:3 Changed 5 years ago by nickm

Status: newneeds_information

What version of TBB is this? Does a more recent series (3.0 or 3.5) have the same issue?

Could this be a duplicate of #9593 ?

comment:4 Changed 5 years ago by iktsuarpok

@nickm
When I created this issue I used tor-browser-2.3.25-15_de.exe.

I couldn't create exactly identical conditions to check because of 3.5rc1 changes (compared to tor-browser-2.3.25-15). At first sight it looks like TBB still doesn't process unicode paths correctly on win.

Here's what I did:
1) Downloaded https://archive.torproject.org/tor-package-archive/torbrowser/3.5rc1/torbrowser-install-3.5-rc-1_de.exe
2) Installed it to C:\Users\kuncaŭščyna\Downloads\3.5rc1
(instead it installed itself to C:\Users\kuncauscyna\Downloads\3.5rc1 having created the necessary path)
3) I moved the installed files back to C:\Users\kuncaŭščyna\Downloads\3.5rc1
4) Ran it from Win Explorer, it said that Tor was unexpectedly closed.

To compare I ran it from "ascii-only" path (I copied the data to "C:\Users\kuncauscyna;)\Downloads\3.5rc1\Tor Browser") and ran it from there in Win Explorer. It said that it couldn't find libevent*.dll. I closed Tor, opened Far, run

set PATH=Tor;%PATH%

and restarted "Start Tor Browser.exe" from Far. It opened firefox (== Tor Browser) without problems. Then I did the same for "C:\Users\kuncaŭščyna\Downloads\3.5rc1\Tor Browser" (set PATH and restarted from Far). But it still said that Tor was unexpectedly closed.

So I concluded that I can run TorBroswer-3.5rc1 from "C:\Users\kuncauscyna;)\Downloads\3.5rc1\Tor Browser" but not from "C:\Users\kuncaŭščyna\Downloads\3.5rc1\Tor Browser" i.e. I can run it only from ascii-only paths.

(I'm not yet used to the new UI and couldn't find the logs in 3.5rc1, so not sure what exactly went wrong)

comment:5 Changed 5 years ago by iktsuarpok

Status: needs_informationnew

comment:6 Changed 5 years ago by nickm

Cc: helix mikeperry mcs added

TBB people -- can we trace what command line Tor is getting in this situation? Are we losing the non-ascii characters between the launcher and Tor, or between Tor and the filesystem call?

Also, could somebody refresh me on why this needs to be an absolute path? I know I've had it explained to me before, but I didn't see the explanation here or on #9593 when I checked.

comment:7 Changed 5 years ago by cypherpunks

I can confirm, nor TBB with Vidalia nor TBB3 do not works for Windows XP if path to torrc file contains non ascii chars.

comment:8 Changed 5 years ago by cypherpunks

for Windows XP if path to torrc file contains non ascii chars.

Those was non localized version without locales, so not sure it's reproducible for some European version with proper locales.

comment:9 in reply to:  6 Changed 5 years ago by mcs

Replying to nickm:

TBB people -- can we trace what command line Tor is getting in this situation? Are we losing the non-ascii characters between the launcher and Tor, or between Tor and the filesystem call?

(parenthetical comment: If I place the downloaded exe (self-extracting installer) in a folder that contains non-ASCII characters in its name, I see "Error launching installer". So I had to extract it onto the Desktop and then move the "Tor Browser" folder into the non-ASCII folder. So maybe the installer has trouble with non-ASCII paths too.)

To see some debugging from Tor Launcher, edit ...\Tor Browser\Data\Browser\profile.default\prefs.js and add these two lines:

user_pref("extensions.torlauncher.loglevel", 0);
user_pref("extensions.torlauncher.logmethod", 0);

Then open a cmd prompt and cd to ...\Tor Browser\Browser. For non-ASCII characters to be displayed correctly in a console window, you need to set the font to Lucida Console (which can be done from the console window's menu) and you should set the code page to UTF-8 by executing this command:
chcp 65001

Then start the browser like this:
firefox -console -no-remote

A console window should open and you will debugging output from Tor Launcher, including a dump of the args that are passed to tor. The problem is, I cannot tell if the paths being passed to tor are correct or not because the console window does not have the correct code page. I will have to devise a way to capture the command line args (maybe I can temporarily replace tor.exe with something that does just that).

Also, could somebody refresh me on why this needs to be an absolute path? I know I've had it explained to me before, but I didn't see the explanation here or on #9593 when I checked.

Tor Launcher passes absolute paths to tor because Tor Launcher cannot change the current working directory, and therefore the only (seemingly) safe choice is to use absolute paths. Given the kind of problem exposed by this bug though, it would be nice if it could figure out a way to use relative paths.

comment:10 Changed 5 years ago by cypherpunks

No problem with Vidalia or Torlauncher, they passes agruments in proper way, but Tor can handle only non unicode argv that OS converted using configured code-page. Proper way is to drop windows legacy code that depends code-page and to use unicode windows specific stuff and funcs.

Tor works with non-ascii stuff if used chars can be converted with configured code-page. It working for me with non-ascii chars for localized windows, but can't to start with reported 'kuncaŭščyna' directory even if language (via control panel, that changes code-page for non-unicode programs) defined as German. Everything works if renamed directory to word that can be converted and used for german code-page, for example: 'Europäische'

comment:11 Changed 5 years ago by iktsuarpok

The kuncaŭščyna directory was used only for testing. (This is a name in Belarusian Latin alphabet - there's no code page for this alphabet, imo). I'm not sure how often this problem may occur in reality, but here's what actually happened. I was reported about this problem by a friend who moved to Italy (and I guess she changed her locale but not her login). She simply downloaded the TBB (latest stable release for en_US) into the default location (C:\Users\<user>\Downloads) unpacked the exe and ran the launcher. It didn't start though and she didn't have any clue as to what was wrong. When I looked into her log, I saw the message:
Unable to open configuration file: C:\Users\????\Downloads\...
But with the new release candidate I might be also puzzled, because it doesn't show log, it simply says that "Tor was unexpectedly closed".
So maybe if it's difficult to add proper unicode support on windows (I understand that windows differs much from unices), it's possible to notify the user about the actual problem along with instruction to move the files somewhere else (as an option).

Last edited 5 years ago by iktsuarpok (previous) (diff)

Changed 5 years ago by iktsuarpok

Attachment: tor_nonascii_log.jpg added

the log message (stable release) for the issue

comment:12 in reply to:  11 Changed 5 years ago by mcs

Cc: brade added

Replying to iktsuarpok:

... When I looked into her log, I saw the message:
Unable to open configuration file: C:\Users\????\Downloads\...
But with the new release candidate I might be also puzzled, because it doesn't show log, it simply says that "Tor was unexpectedly closed".
So maybe if it's difficult to add proper unicode support on windows (I undestand that windows differs much from other unices), it's possible to notify the user about the actual problem along with instruction to move the files somewhere else (as an option).

I agree that better error reporting is needed. Unfortunately, Tor Launcher does not currently have a way to capture all of the log messages generated by tor. See #10059.

comment:13 in reply to:  10 Changed 5 years ago by nickm

Replying to cypherpunks:

No problem with Vidalia or Torlauncher, they passes agruments in proper way, but Tor can handle only non unicode argv that OS converted using configured code-page. Proper way is to drop windows legacy code that depends code-page and to use unicode windows specific stuff and funcs.

Hm. If we can do this by just handling unicode command-line arguments (s/main/wmain/), this won't be hard. If we need to change every place that we touch the file system or manipulate a path, that will be a bit harder.

comment:14 Changed 5 years ago by cypherpunks

Hm. If we can do this by just handling unicode command-line arguments (s/main/wmain/), this won't be hard. If we need to change every place that we touch the file system or manipulate a path, that will be a bit harder.

If legacy program could to set UTF-8 code page to use by legacy functions that would solve all problems without any changes, but no such method documented and Microsoft want to drop code pages support someday in every piece of shipped code.

Windows native way is to use "wide" functions with "wide" chars to store stuff. More simpler solution for less changes is to convert UTF-16 arguments to UTF-8 and convert back only for windows UTF-16 funcs, but it would to break output and logs.

comment:15 Changed 5 years ago by nickm

Keywords: tor-client added
Milestone: Tor: 0.2.6.x-final
Priority: normalmajor

comment:16 Changed 5 years ago by nickm

Cc: erinn added

Adding 'erinn' to cc list of every ticket with 'helix' in its cc list -- erinn is helix's trac username.

comment:17 Changed 5 years ago by nickm

Cc: helix removed

Removing helix from cc lists -- helix is not erinn's trac username.

comment:18 Changed 5 years ago by sschuerger

Workaround when running NTFS:

Create a junction named
C:\Users\kuncauscyna
pointing to
C:\Users\kuncaŭščyna

junction.exe is available from the MS Power Toys.

comment:19 Changed 5 years ago by nickm

Keywords: 026-triaged-1 added

comment:20 Changed 4 years ago by nickm

Milestone: Tor: 0.2.6.x-finalTor: 0.2.7.x-final

comment:21 Changed 4 years ago by nickm

Status: newassigned

comment:22 Changed 4 years ago by nickm

Keywords: 027-triaged-1-out added

Marking triaged-out items from first round of 0.2.7 triage.

comment:23 Changed 4 years ago by nickm

Milestone: Tor: 0.2.7.x-finalTor: 0.2.???

Make all non-needs_review, non-needs_revision, 027-triaged-1-out items belong to 0.2.???

comment:24 Changed 2 years ago by teor

Milestone: Tor: 0.2.???Tor: 0.3.???

Milestone renamed

comment:25 Changed 2 years ago by nickm

Keywords: tor-03-unspecified-201612 added
Milestone: Tor: 0.3.???Tor: unspecified

Finally admitting that 0.3.??? was a euphemism for Tor: unspecified all along.

comment:26 Changed 22 months ago by nickm

Keywords: tor-03-unspecified-201612 removed

Remove an old triaging keyword.

comment:27 Changed 22 months ago by nickm

Keywords: 027-triaged-in added

comment:28 Changed 22 months ago by nickm

Keywords: 027-triaged-in removed

comment:29 Changed 22 months ago by nickm

Keywords: 027-triaged-1-out removed

comment:30 Changed 22 months ago by nickm

Keywords: 026-triaged-1 removed

comment:31 Changed 22 months ago by nickm

Status: assignednew

Change the status of all assigned/accepted Tor tickets with owner="" to "new".

comment:32 Changed 22 months ago by nickm

Keywords: win32 added
Severity: Normal

comment:33 Changed 10 months ago by arma

Is this bug causing all Windows users with weird characters in their username to have their Tor Browser fail when they try to run it from their user directory?

I haven't heard from any affected users, so maybe it's not so bad -- or instead, maybe users like that are less likely to come usefully report a bug to us in English.

comment:34 Changed 5 months ago by teor

Parent ID: #25729

comment:35 Changed 5 months ago by teor

See #25729: this bug only happens when the torrc is encoded in a different encoding to the default system encoding.

We need a proposal for transitioning from Windows-codepage encoded torrcs to UTF-8 encoded torrcs.

comment:36 in reply to:  11 Changed 3 months ago by cypherpunks2

Replying to iktsuarpok:

But with the new release candidate I might be also puzzled, because it doesn't show log, it simply says that "Tor was unexpectedly closed".

This.

grep -R "unexpectedly exited" trac.torproject.org | wc -l
many-much

comment:37 Changed 3 months ago by cypherpunks2

Simple detector for tor-launcher

diff --git a/src/components/tl-process.js b/src/components/tl-process.js
index 3aa45e4..e12e523 100644
--- a/src/components/tl-process.js
+++ b/src/components/tl-process.js
@@ -355,6 +355,7 @@ TorProcessService.prototype =
   mLastTorWarningPhase: null,
   mLastTorWarningReason: null,
   mDefaultPreferencesAreLoaded: false,
+  mWindowsCodePageWrong: false,
 
   // Private Methods /////////////////////////////////////////////////////////
   _startTor: function(aForceDisableNetwork)
@@ -510,6 +511,12 @@ TorProcessService.prototype =
         if (env.exists("PATH"))
           path += ";" + env.get("PATH");
         env.set("PATH", path);
+        var check = env.get("PATH");
+        if (path != check) {
+           this.mWindowsCodePageWrong = true;
+           var s = TorLauncherUtil.getLocalizedString("wrong_windows_codepage")
+           this._notifyUserOfError(s, null, this.kTorProcessDidNotStartTopic);
+        }
       }
 
       this.mTorProcessStatus = this.kStatusStarting;

comment:38 Changed 3 months ago by gk

Cc: Executorvoodoo added

Resolved #28943 as duplicate.

comment:39 in reply to:  37 Changed 3 months ago by gk

Replying to cypherpunks2:

Simple detector for tor-launcher

diff --git a/src/components/tl-process.js b/src/components/tl-process.js
index 3aa45e4..e12e523 100644
--- a/src/components/tl-process.js
+++ b/src/components/tl-process.js
@@ -355,6 +355,7 @@ TorProcessService.prototype =
   mLastTorWarningPhase: null,
   mLastTorWarningReason: null,
   mDefaultPreferencesAreLoaded: false,
+  mWindowsCodePageWrong: false,
 
   // Private Methods /////////////////////////////////////////////////////////
   _startTor: function(aForceDisableNetwork)
@@ -510,6 +511,12 @@ TorProcessService.prototype =
         if (env.exists("PATH"))
           path += ";" + env.get("PATH");
         env.set("PATH", path);
+        var check = env.get("PATH");
+        if (path != check) {
+           this.mWindowsCodePageWrong = true;
+           var s = TorLauncherUtil.getLocalizedString("wrong_windows_codepage")
+           this._notifyUserOfError(s, null, this.kTorProcessDidNotStartTopic);
+        }
       }
 
       this.mTorProcessStatus = this.kStatusStarting;

Thanks! I opened #29020 for a potential warning mechanism in Tor Launcher.

Note: See TracTickets for help on using tickets.