Opened 3 years ago

Closed 3 years ago

Last modified 3 years ago

#17250 closed defect (fixed)

Japanese font(s) look ugly on websites

Reported by: gk Owned by: tbb-team
Priority: Medium Milestone:
Component: Applications/Tor Browser Version:
Severity: Normal Keywords: tbb-fingerprinting-fonts, TorBrowserTeam201512R
Cc: yawning, arthuredelstein Actual Points:
Parent ID: #18097 Points:
Reviewer: Sponsor:

Description

During the dev meeting we got some feedback wrt to our new Japanese bundle. It was said that there are websites that look ugly using a localized build. I am not sure about the OS and the websites that are affected though.

Child Tickets

Change History (19)

comment:1 Changed 3 years ago by yawning

A few things, by platform:

  • Linux uses NotoSansCJKsc-Regular.otf. I'm not sure why we ship the Simplified Chinese version of the font. But it's legible, though I haven't looked for differences/stuff that's broken very hard.
  • Under Windows, there's no fonts configured for Japanese (about:preferences#content->Advanced), the drop-downs for Serif, Sans-serif, and Monospace are empty. I have no idea how Firefox handles fonts named in Japanese (eg: MS 明朝 whitelisted as MS Mincho), but I can't select the fonts even though they're present on the system. Even the about:tor page looks kind of horrible.
  • I don't have a Macintoy.

comment:2 Changed 3 years ago by yawning

http://ipafont.ipa.go.jp/index.html#en has a quirky license, but looks OK if we're willing to bloat the bundle size. Not sure if there's anything better.

comment:3 in reply to:  1 ; Changed 3 years ago by arthuredelstein

Replying to yawning:

A few things, by platform:

  • Linux uses NotoSansCJKsc-Regular.otf. I'm not sure why we ship the Simplified Chinese version of the font. But it's legible, though I haven't looked for differences/stuff that's broken very hard.

The NotoSansCJKsc-Regular.otf font covers Japanese, Korean, and both Simplified and Traditional Chinese. The "sc" refers to the fact that it defaults to Simplified Chinese character variants if the language has not been specified by a page.

  • Under Windows, there's no fonts configured for Japanese (about:preferences#content->Advanced), the drop-downs for Serif, Sans-serif, and Monospace are empty. I have no idea how Firefox handles fonts named in Japanese (eg: MS 明朝 whitelisted as MS Mincho), but I can't select the fonts even though they're present on the system. Even the about:tor page looks kind of horrible.

Right now on Windows (in the alpha) only MS Mincho is whitelisted, so that's the only Japanese font available. I will look into whether it would be possible to whitelist more Windows fonts. Are there some you would prefer?

  • I don't have a Macintoy.

comment:4 in reply to:  3 ; Changed 3 years ago by yawning

Replying to arthuredelstein:

Replying to yawning:

A few things, by platform:

  • Linux uses NotoSansCJKsc-Regular.otf. I'm not sure why we ship the Simplified Chinese version of the font. But it's legible, though I haven't looked for differences/stuff that's broken very hard.

The NotoSansCJKsc-Regular.otf font covers Japanese, Korean, and both Simplified and Traditional Chinese. The "sc" refers to the fact that it defaults to Simplified Chinese character variants if the language has not been specified by a page.

Ah ok. Shouldn't we be shipping a separate Noto variant for each ov the CJKV (+TC) locales that we have bundles for? By default, someone that's using the ja_JP bundle is likely going to hit pages with no language specifier that are in Japanese...

If the browser automagically handles all of this, that's great. I don't read Japanese content that much, so I have no idea how modern browsers handle this stuff. Also, Han Unification is Stupid, and should be destroyed.

  • Under Windows, there's no fonts configured for Japanese (about:preferences#content->Advanced), the drop-downs for Serif, Sans-serif, and Monospace are empty. I have no idea how Firefox handles fonts named in Japanese (eg: MS 明朝 whitelisted as MS Mincho), but I can't select the fonts even though they're present on the system. Even the about:tor page looks kind of horrible.

Right now on Windows (in the alpha) only MS Mincho is whitelisted, so that's the only Japanese font available. I will look into whether it would be possible to whitelist more Windows fonts. Are there some you would prefer?

Err. If MS Mincho is whitelisted, why isn't it being used? As in, with the ja_JP bundle on Windows, there are no fonts configured for Japanese at all. Likewise, if it's whitelisted, why can't I select it as my Serif font (It should at least appear in the dropdown...)?

I assume if I hack up my profile to use: MS 明朝 for my Serif and MS ゴシック as my Sans-Serif font that things will actually look ok, but I haven't tried yet (Double width , , single-width ).

It's worth noting that despite the fonts having been shipped since Windows 3.1, we should not be whitelisting either if "the font has glyphs that changed between versions" is an issue.

ChangeLogs:
https://ja.wikipedia.org/wiki/MS_%E3%82%B4%E3%82%B7%E3%83%83%E3%82%AF
https://ja.wikipedia.org/wiki/MS_%E6%98%8E%E6%9C%9D

comment:5 in reply to:  4 Changed 3 years ago by arthuredelstein

Replying to yawning:

Replying to arthuredelstein:

Replying to yawning:

A few things, by platform:

  • Linux uses NotoSansCJKsc-Regular.otf. I'm not sure why we ship the Simplified Chinese version of the font. But it's legible, though I haven't looked for differences/stuff that's broken very hard.

The NotoSansCJKsc-Regular.otf font covers Japanese, Korean, and both Simplified and Traditional Chinese. The "sc" refers to the fact that it defaults to Simplified Chinese character variants if the language has not been specified by a page.

Ah ok. Shouldn't we be shipping a separate Noto variant for each ov the CJKV (+TC) locales that we have bundles for? By default, someone that's using the ja_JP bundle is likely going to hit pages with no language specifier that are in Japanese...

That would be ideal, but it's difficult to implement because we're trying not to leak the user's locale. We would need to ensure that the CJKVsc and CJKVjp have the same apparent name exposed to Firefox, and the bounding rectangles for every character are the same for each variant. It might be possible.

If the browser automagically handles all of this, that's great. I don't read Japanese content that much, so I have no idea how modern browsers handle this stuff. Also, Han Unification is Stupid, and should be destroyed.

Agreed.

  • Under Windows, there's no fonts configured for Japanese (about:preferences#content->Advanced), the drop-downs for Serif, Sans-serif, and Monospace are empty. I have no idea how Firefox handles fonts named in Japanese (eg: MS 明朝 whitelisted as MS Mincho), but I can't select the fonts even though they're present on the system. Even the about:tor page looks kind of horrible.

Right now on Windows (in the alpha) only MS Mincho is whitelisted, so that's the only Japanese font available. I will look into whether it would be possible to whitelist more Windows fonts. Are there some you would prefer?

Err. If MS Mincho is whitelisted, why isn't it being used? As in, with the ja_JP bundle on Windows, there are no fonts configured for Japanese at all.

Oops, I got confused -- in Windows TBB 5.5.3a, there are actually four fonts whitelisted specifically for Japanese: MS Mincho, MS PMincho, MS Gothic, MS PGothic.

Here's a page to check that all four Japanese fonts are available to content pages:

https://arthuredelstein.github.io/tordemos/windows-fonts.html

I notice that both serif and sans-serif choose MS PGothic. This seems wrong, but it's true for vanilla Firefox as well.

On Windows, do you see important situations where TBB 5.5.3a Japanese text is worse than Firefox? Not that we shouldn't fix vanilla Firefox text rendering problems, but I want to fix TBB-specific problems first.

For example, ja.wikipedia.org looks basically identical on my Windows machines on Firefox and TBB.

Likewise, if it's whitelisted, why can't I select it as my Serif font (It should at least appear in the dropdown...)?

I don't know why the dropdown is failing.

I assume if I hack up my profile to use: MS 明朝 for my Serif and MS ゴシック as my Sans-Serif font that things will actually look ok, but I haven't tried yet (Double width , , single-width ).

That would be very helpful if you find preference settings that make it look better for you.

It's worth noting that despite the fonts having been shipped since Windows 3.1, we should not be whitelisting either if "the font has glyphs that changed between versions" is an issue.

We decided, at this stage, not to try to protect users from that issue.

comment:6 Changed 3 years ago by yawning

Severity: Normal

I finally got around to digging more into this. Modifying the whitelist on a Windows 10 system to include MS 明朝 and MS ゴシック results in them being picked up and used to display Japanese.

Because #16739 exists, I assume the problem here is "We only whitelisted the English display names of the fonts, which do not exist when using Japanese as the system locale".

The short term workaround would be to whitelist

  • MS 明朝(MS Mincho)
  • MS ゴシック(MS Gothic)
  • MS P明朝(MS PMincho)
  • MS Pゴシック(MS PGothic)

Note: is also double width. Past that we get to move on to fixing "those fonts look like shit", but this is a easy thing we should do (the fonts are already whitelisted, just not completely whitelisted).

comment:7 in reply to:  6 Changed 3 years ago by arthuredelstein

Replying to yawning:

I finally got around to digging more into this. Modifying the whitelist on a Windows 10 system to include MS 明朝 and MS ゴシック results in them being picked up and used to display Japanese.

Because #16739 exists, I assume the problem here is "We only whitelisted the English display names of the fonts, which do not exist when using Japanese as the system locale".

The short term workaround would be to whitelist

  • MS 明朝(MS Mincho)
  • MS ゴシック(MS Gothic)
  • MS P明朝(MS PMincho)
  • MS Pゴシック(MS PGothic)

Note: is also double width. Past that we get to move on to fixing "those fonts look like shit", but this is a easy thing we should do (the fonts are already whitelisted, just not completely whitelisted).

Thanks for tracking this down. I was unaware of this issue, but I will check other languages (such as Chinese) for similar problems and post a patch with the corrected whitelist.

comment:8 Changed 3 years ago by arthuredelstein

Keywords: TorBrowserTeam201510R added
Status: newneeds_review

I found a list of localized font names for Windows here:
http://trigeminal.fmsinc.com/samples/font_choices.html

Here's a patch for review, where I have added the localized names for each CJK font to the font whitelist on Windows. I'm didn't find any localized font names for the non-CJK fonts included in this whitelist.

https://github.com/arthuredelstein/tor-browser/commit/17250

(I will also investigate if any localized font names are needed for OS X or Linux.)

comment:9 in reply to:  8 ; Changed 3 years ago by yawning

Replying to arthuredelstein:

Here's a patch for review, where I have added the localized names for each CJK font to the font whitelist on Windows. I'm didn't find any localized font names for the non-CJK fonts included in this whitelist.

https://github.com/arthuredelstein/tor-browser/commit/17250

(I will also investigate if any localized font names are needed for OS X or Linux.)

The patch looks ok for Japanese/Windows.

I'm hesitant to call this fixed since our Japanese about:tor page on Windows still looks horrific even when the correct fonts are specified, leading to poor first impressions and a bad out-of-the-box experience.

There's an upstream issue discussing this: https://bugzilla.mozilla.org/show_bug.cgi?id=1016792

Is there a particular reason why we whitelist YaHei/JhengHei but not Meiryo/Malgun Gothic (Korean)? Availability across the fonts should be similar (Vista+).

The Meiryo inline box breakage documented in comment 7 was resolved in Firefox 40 (https://bugzilla.mozilla.org/show_bug.cgi?id=1157083).

comment:10 in reply to:  9 ; Changed 3 years ago by mcs

Replying to yawning:

The patch looks ok for Japanese/Windows.

I'm hesitant to call this fixed since our Japanese about:tor page on Windows still looks horrific even when the correct fonts are specified, leading to poor first impressions and a bad out-of-the-box experience.

Is the above issue covered by #16735, or is there another issue with about:tor?

comment:11 in reply to:  10 Changed 3 years ago by yawning

Replying to mcs:

Replying to yawning:

The patch looks ok for Japanese/Windows.

I'm hesitant to call this fixed since our Japanese about:tor page on Windows still looks horrific even when the correct fonts are specified, leading to poor first impressions and a bad out-of-the-box experience.

Is the above issue covered by #16735, or is there another issue with about:tor?

No, it's a separate issue. It renders correctly, but "Congratulations! This browser is configured to use Tor." looks terrible because a font that's not great to begin with is getting scaled up to a large size.

There's not much that can be done without changing which fonts we use, I don't think.

comment:12 Changed 3 years ago by gk

Status: needs_reviewneeds_revision

I picked the fix from your 17250 branch. It is ea1c3ae18602b21bed1bcb6dad0f77703bfb6c4e on tor-browser-38.3.0esr-5.5-2. This ticket needs additional patches to be fixed. Thus, marking it as "needs revision".

comment:13 Changed 3 years ago by gk

Keywords: TorBrowserTeam201511 added

comment:14 Changed 3 years ago by mikeperry

Keywords: TorBrowserTeam201510R removed

comment:15 Changed 3 years ago by mikeperry

Keywords: TorBrowserTeam201512 added; TorBrowserTeam201511 removed

comment:16 Changed 3 years ago by arthuredelstein

Keywords: TorBrowserTeam201512R added; TorBrowserTeam201512 removed
Status: needs_revisionneeds_review

I posted a fix here:
https://github.com/arthuredelstein/tor-browser/commits/17250+1
This branch contains two commits, one for this bug and one for #17661.

Note: these patches just expand the font whitelists for OS X and Windows.

Last edited 3 years ago by arthuredelstein (previous) (diff)

comment:17 Changed 3 years ago by gk

Resolution: fixed
Status: needs_reviewclosed

Okay, assuming it helps: commit 2c2d625aef55b9bd1ed1b1ec459e4f1a8b11eadb on tor-browser-38.4.0esr-5.5-1. It will be in 5.5a5.

comment:18 Changed 3 years ago by arthuredelstein

Parent ID: 18097

comment:19 Changed 3 years ago by arthuredelstein

Parent ID: 18097#18097
Note: See TracTickets for help on using tickets.