Unable to run ooniprobe against Vodafone mobile broadband (UK)
When running ooniprobe on the Vodafone mobile broadband network, the probe encounters an error during the collection of the non-tor response.
The target URL was reddit.com/r/nsfw (which is blocked on Vodafone UK)
The error from the report file is:
experiment_failure: unknown_failure [<twisted.python.failure.Failure <type 'exceptions.ValueError'>>]
The same error is reported when using a simple test script that contains a twisted Agent retrieving the target page.
I think this may be because the 302 redirect response that Vodafone uses to send people to the content control advisory page does not have a content-length or transfer encoding, and this is breaking twisted's Agent.
Here's the curl response:
$ curl -v http://reddit.com/r/nsfw
* About to connect() to reddit.com port 80 (#0)
* Trying 198.41.209.140...
* connected
* Connected to reddit.com (198.41.209.140) port 80 (#0)
> GET /r/nsfw HTTP/1.1
> User-Agent: curl/7.26.0
> Host: reddit.com
> Accept: */*
>
* additional stuff not fine transfer.c:1037: 0 0
* HTTP 1.1 or later with persistent connection, pipelining supported
< HTTP/1.1 302 Moved Temporarily
< Location: http://online.vodafone.co.uk/dispatch/Portal/ContentControlServlet?type=restricted
< Content-Type: text/html
< charset=iso-8859-1
< Proxy-Connection: close
* no chunk, no close, no size. Assume close to signal end
<
* nread <= 0, server closed connection, bailing
* Closing connection #0
<html><body><p>The document has moved</p></body></html>