Opened 12 months ago

Last modified 7 months ago

#8839 reopened defect

startpage.com filters search results by keyword now (due to Google change)

Reported by: cypherpunks Owned by: mikeperry
Priority: major Milestone:
Component: Firefox Patch Issues Version:
Keywords: MikePerry201309 Cc: mikeperry
Actual Points: Parent ID:
Points: 2

Description

"The Tor Browser Bundle uses startpage.com as the default search engine, but startpage.com has a so-called "family filter" enabled by default, which filters out many webpages"

This is pretty funny for a browser that people use to evade censorship and filters.

Child Tickets

Change History (16)

comment:1 follow-up: Changed 12 months ago by cypherpunks

And how exactly is this TBB's fault?

Anyway, there is a very simple solution involving a couple of mouse clicks to this via Startpage Settings.

comment:2 in reply to: ↑ 1 ; follow-up: Changed 12 months ago by cypherpunks

Replying to cypherpunks:

And how exactly is this TBB's fault?

TBB should set an example for others. There are two solutions here -

  • disable family filter by default
  • use some other search engine like duckdukgo

Anyway, there is a very simple solution involving a couple of mouse clicks to this via Startpage Settings.

refer above statement

comment:3 in reply to: ↑ 2 Changed 11 months ago by cypherpunks

You don't seem to grasp the fact that neither TBB nor the Tor Project have any control over what Startpage chooses to do or not do.

Duckduckgo has some useful features, but the quality of its search results are incredibly poor compared to Startpage.

"TBB should set an example for others." -- How would you propose the family filter could be disabled by default? Specifically.

comment:4 Changed 8 months ago by nickm

  • Cc mikeperry added
  • Priority changed from normal to major

You can manufature startpage URLs that have arbitrary combinations of preferences. Go to the settings page, click the settings you want, and click "Get URL." Use that as the search URL rather than the. Alternatively, I believe there's a cookie you can set. So the technical possibility exists.

Mike, I think this one is a good idea. I'm hearing reports that the "porn blocker" is not just blocking porn, but is being assholish about LGBT stuff, which is of course serious. I did a little investigation, and it also seems that their filter blocks many searches for nonmainstream sexual practices, regardless of the pornographic content of the webpage. This has some tremendously negative effects. For example, one can search for "lesbians" and get some okay-looking webpages, and for "kink" and get some webpages about hairstyling and bent ropes, but when one searches for "lesbians and kink", you get no results with the "safe" filter on. There are probably bunches of other examples, but that one is seriously problematic in my book.

Furthermore, when I search for that term, it gives me advertisements for at least one thing that it presumably would have filtered otherwise, which kind of implies to me that the whole filtering mechanism just sucks.

Can we try it with the filter off in the next alpha and see if it is finally the straw on the camel's back that sets off the promised pornopocalypse that has yet failed to destroy western civilization?

comment:5 Changed 8 months ago by nickm

And here's a less serious-business example. With the filter on, when you search for "tom lerher smut", you get a lot of generic Tom Lehrer results. If you want Mr. Lehrer's delightful anticensorship anthem, "Smut", you need to turn the filter off.

comment:6 Changed 8 months ago by nickm

(Err, and that result holds even when you are spelling 'lehrer' correctly in both cases.)

comment:7 follow-up: Changed 8 months ago by nickm

And here's another one. If you search for "The Owl and the Pussy-Cat" , hoping for the Edward Lear poem and punctuating its title properly, you get a bunch of cat and owl articles because that word for a cat is a naughty one. (Sometimes you get the right poem way down on the page.) The advertisers present you with the right poem, though. If you turn off filtering, you get the nice nonsense poem you wanted.

You cannot search for "pussy riot" either, because of the same forbidden word. WTH.

I remark in passing that "Moby-Dick" appears to be unfiltered, as "Dick's Sporting Goods", as is "Dick Cheney".

I could do this all night, but I shouldn't have to. The pussy riot one should clinch this. Let's kill this filter hard.

comment:8 in reply to: ↑ 7 Changed 8 months ago by nickm

Replying to nickm:

I remark in passing that "Moby-Dick" appears to be unfiltered, as "Dick's Sporting Goods", as is "Dick Cheney".

But "cock fight", "cock robin", and "gerald cock" all have the "cock" term filtered. At least searching for "Clifford Cocks" (plural) seems to work.

comment:9 Changed 8 months ago by tor_que

Could you *please* also set 100 as the default number of search results per page?

comment:10 Changed 8 months ago by mikeperry

  • Summary changed from startpage.com filters search results to startpage.com filters search results by keyword now (due to Google change)

What we're seeing here is actually a change in Google's Safesearch. It used to be on by default and quite a bit smarter about differentiating porn from non-porn. Now it seems to be keyword based and off by default. I will email the Startpage people and tell them that Google's filters have changed, and see what they say.

If they still want to keep this new junky version enabled, I guess we'll have to experiment with encoding Startpage preference URL in the omnibox.

I wonder if their video/image filters also got dumber.

comment:11 Changed 8 months ago by nickm

So, if their reaction is "I guess we won't filter by default any more", great, let's use their default. But if it's anything else (including "we'll ask google nicely" or "we will have better, smarter filters" or no prompt answer), I think we should try to use the preference URL.

comment:12 Changed 8 months ago by mikeperry

  • Component changed from Torbutton to Firefox Patch Issues
  • Keywords MikePerry201308 added
  • Owner set to mikeperry

nickm: Agreed. I just sent them a mail to see if they will change their default. I also asked if the preference URL was stable (so we can rely on it not to change format/names for prefs).

For now, it does appear we can disable only the search filter with https://startpage.com/do/mypage.pl?prfh=disable_family_filterEEE1N1N

comment:13 Changed 8 months ago by cypherpunks

This is a better way to disable the filter, by only adding this code to the current search plugin after the <Param name="q" value="{searchTerms}"/> code:

<Param name="prfh" value="disable_family_filterEEE1N1N"/>

If you decide to add any other options, just replace the first code like this:

<Param name="prfh" value="disable_family_filterEEE1N1Ndisable_video_family_filterEEE1N1N"/>

comment:14 Changed 8 months ago by mikeperry

  • Keywords MikePerry201309 added; MikePerry201308 removed

Ok. Startpage has created two custom search URLs for us. They want us to use https://startpage.com/rto/search for the omnibox, and https://startpage.com/rth/search for the homepage.

Supposedly both URLs default to unfiltered.

comment:15 Changed 7 months ago by mikeperry

  • Points set to 2
  • Resolution set to fixed
  • Status changed from new to closed

I've updated the patchsets for both 2.4.17-beta-2 series and 3.0alpha4 to use those two links.

comment:16 Changed 7 months ago by cypherpunks

  • Resolution fixed deleted
  • Status changed from closed to reopened

Looks like you forgot the omnibox link:

pref("keyword.URL", "https://startpage.com/do/search?q=");

Note: See TracTickets for help on using tickets.