Opened 5 years ago

Closed 5 years ago

#6151 closed defect (not a bug)

Startpage is blocking searches

Reported by: cypherpunks Owned by: rransom
Priority: Immediate Milestone:
Component: TorBrowserButton Version:
Severity: Keywords:
Cc: phobos, ioerror, arma, tails@… Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

I have been using Startpage for months.

But since today, I started to see this error message several times:

We have recently received a large number of searches coming from your computer or others on your local network in a very short time frame. In order to protect our service against automated "screen scraping" software programs, your access to Startpage's search has been paused for approximately one hour.

If you were using Startpage normally, we apologize for the inconvenience and will be able to lift this pause if you phone us at (212) 447-1100 (USA). Alternately, if you were operating a "screen scraping" program, you may phone us to work out an arrangement. You can also contact us at: autoquery @ startpage.com

Child Tickets

Attachments (2)

182lhy.png (66.5 KB) - added by rransom 5 years ago.
389vw.png (166.8 KB) - added by rransom 5 years ago.

Download all attachments as: .zip

Change History (21)

comment:1 Changed 5 years ago by cypherpunks

A reason is probably my current exit node. It has the 3rd biggest bandwidth rate currently, so more people should be using it now

comment:2 Changed 5 years ago by cypherpunks

Priority: normalmajor

Startpage just added a special Tor page:

http://i45.tinypic.com/182lhy.png

Can we do anything about this problem?

What about a system to send them the number of users of the exit nodes so that we or they can calculate a good/natural "search per user rate" using all statistics so they won't just look at the total connections especially for the biggest exit nodes.

comment:3 Changed 5 years ago by mikeperry

Cc: phobos added

We're trying to get in direct contact with them. Our best long-term plan for this type of issue is #4666. In the meantime, there's probably a few tricks that can be done to at least make it so that they're sure the thing that connects to them is a real Tor Browser and not a scraper script.

However, that proxy settings tweak they suggest needs to go away right speedy quick...

Changed 5 years ago by rransom

Attachment: 182lhy.png added

comment:4 Changed 5 years ago by rransom

Priority: majorblocker

Raising the priority of this bug because Startpage is telling Firefox-with-Torbutton users to backdoor their browsers (the patch for #5741 would break those instructions in TBB; that's probably the only reason They don't give the same bad advice to TBB users).

comment:5 Changed 5 years ago by phobos

Here's there new error message in text form:

Are you using Tor?

Welcome to Startpage, the world's most private search engine.

    We do not store anything about you — not your IP address, not your search queries
    We encrypt all traffic — using HTTPS, so even your ISP can’t snoop on your searches
    Our privacy protections are third-party certified — we are the only search engine with third-party certification
    The E.U.'s privacy organization has certified us — no other search engine offers EuroPrise-certified privacy, and our systems are regularly audited to confirm this compliance 

Occasionally, Startpage may not return results with Tor.
This happens when many Tor users are searching through the same end node, and Startpage cannot determine whether the searches are originating from automated screen-scraping programs.

Here's what to do when this happens:
If you use the Tor Browser Bundle:

    Switching to a new Tor identity is easy and fast: click the green onion icon next to your address bar, then click "New Identity", and try your search again. In some cases, you may have to switch identities more than once for this method to work. 

Or, if you use Firefox and the Torbutton:

    For a more permanent solution, you can instruct Firefox to use Startpage without its proxying. This is safe because Startpage encrypts your searches and does not store any personal data. To do this, go into the Firefox Options, click the "Advanced" tab, click "Network," then click "Settings". Then just add ".startpage.com" to the "No Proxy for" field and click "Ok". You should now have no problem using Startpage with Firefox and the Torbutton! 

We apologize for the inconvenience! We support the Tor project's privacy mission and have dedicated ourselves to delivering a private and secure search experience. We'd love to hear from you; Thanks for supporting Startpage and Tor's mission of a search engine and Internet that honor your privacy.

comment:6 Changed 5 years ago by phobos

too tired to use the correct "their"

also, no response from starpage as of yet.

comment:7 Changed 5 years ago by mikeperry

Cc: ioerror arma added
Priority: blockermajor

rransom: While I agree that this is Serious Business, this is only a blocker for the Torbutton component. For TorBrowserButton/Firefox Patch Issues, it's at worst a major annoyance.

I would fully endorse creating a Torbutton ticket, assigning it to cypherpunks as a blocker, and encouraging everyone to write up this amusing failure of privacy by policy thinking far and wide. But I think we should be merciful and give them a day or so to come to their senses and remove the offending paragraph, first.

comment:8 Changed 5 years ago by phobos

Startpage responds. And they are now interested in working with us to some degree.

comment:9 Changed 5 years ago by runa

Define "to some degree"? Can we switch to DDG as the default in TBB?

comment:10 Changed 5 years ago by rransom

Priority: majorblocker

The Tor Project has effectively endorsed Startpage by choosing their search engine as the default in TBB. The last sentence of Startpage's blocking page attempts to pass off their advice as supported by The Tor Project. That is unacceptable.

The Startpage search engine file must be removed from the Tor Browser Bundle, as soon as possible.

comment:11 Changed 5 years ago by arma

I'm inclined to agree.

I'm sure we dropped the ball a little bit by not immediately presenting them with a team of engineers and copy editors to help them clean up their act. But despite that, they're pretty clearly demonstrating their lack of understanding about how privacy and anonymity _technologies_ work. Endorsing companies that think privacy is a policy measure is going to undermine a lot of our user education efforts.

comment:12 Changed 5 years ago by arma

That said, who are we waiting on to actually make a decision here? phobos or me because we're "in charge"? Mike or Erinn because they'll be writing the commit? Some other notion of consensus?

comment:13 Changed 5 years ago by proper

Have they responded on removing the "bypass Tor" paragraph?

comment:14 Changed 5 years ago by cypherpunks

I noticed a new message today. Looks like they've removed the bit that instructed users to backdoor their browser. Screenshot: http://i.imgur.com/389vw.png

Changed 5 years ago by rransom

Attachment: 389vw.png added

comment:15 Changed 5 years ago by proper

Are you still adamant about your previous opinions since they removed the offending paragraph?

Or do you think, now that they removed it, leave everything as it is and close this one?

comment:16 Changed 5 years ago by T(A)ILS developers

Cc: tails@… added

comment:17 Changed 5 years ago by cypherpunks

I don't know if Startpage has done anything about this, but I haven't seen that screen in the past several days.

comment:18 Changed 5 years ago by mikeperry

Owner: changed from mikeperry to rransom
Status: newassigned

I dunno guys. I never did get exactly what this bug was "blocking" on our side. Reassigning to rransom so he can let it block some more stuff that he's doing.

Glad the notice seems to have gone away, though!

comment:19 Changed 5 years ago by mikeperry

Resolution: not a bug
Status: assignedclosed

Seems to have stopped happening.

Note: See TracTickets for help on using tickets.