wiki:doc/OONI/CensorshipDetectionTools/Herdict

General

HerdictWeb is a tool developed by the Berkman Center for Internet & Society. It allows users to report on the report on websites inaccessibility from places around the world.

It offers two modes of operation: Herdict Reporter (a web application) and Herdict Add-On an in browser addon.

Herdict Reporter

The reporter web application is available here: http://www.herdict.org/participate/reporter

Through this system the user is displayed a series of websites and they are able to select what category they belong to and if it's accessible or not.

The system automatically detects the users ISP.

The sites are visualized inside of an iframe.

On Google Chrome the application does not run cleanly and it issues a large amount of errors to the debug console:

8 event.layerX and event.layerY are broken and deprecated in WebKit. They will be removed from the engine in the near future.

57 Unsafe JavaScript attempt to access frame with URL http://www.herdict.org/participate/reporter from frame with URL http://www.fun1001.com/. Domains, protocols and ports must match.

Unsafe JavaScript attempt to access frame with URL http://www.herdict.org/participate/reporter from frame with URL http://googleads.g.doubleclick.net/pagead/ads?client=ca-pub-0069032636029294&output=html&h=90&slotname=2982632639&w=728&lmt=1339860709&flash=11.1.102&url=http%3A%2F%2Fwww.fun1001.com%2F&dt=1339867909445&bpp=5&shv=r20120606&jsv=r20110914&correlator=1339867909493&frm=22&adk=809099291&ga_vid=1662442146.1339867910&ga_sid=1339867910&ga_hid=2017639099&ga_fc=1&ga_wpids=UA-8566103-2&u_tz=120&u_his=24&u_java=1&u_h=1200&u_w=1920&u_ah=1174&u_aw=1920&u_cd=24&u_nplug=6&u_nmime=95&dff=tahoma&dfs=11&adx=100&ady=80&biw=-12245933&bih=-12245933&isw=927&ish=500&ifk=2586351283&oid=3&ref=http%3A%2F%2Fwww.herdict.org%2Fparticipate%2Freporter&fu=0&ifi=1&dtd=200&xpc=0famS6jaU9&p=http%3A//www.fun1001.com. Domains, protocols and ports must match.

apis.google.com/_/apps-static/_/js/gapi/plusone/rt=j/ver=AzuZKIGCwek.it./sv=1/am=!rFmBCPi40VqIDfp2cA/d=1/rs=AItRSTMsfc8rHyaoY8Eg5sABeeWW-aLc6Q/cb=gapi.loaded_0:117No relay set (used as window.postMessage targetOrigin), cannot send cross-domain message

66 Unsafe JavaScript attempt to access frame with URL http://www.herdict.org/participate/reporter from frame with URL http://www.fun1001.com/. Domains, protocols and ports must match.

38 event.layerX and event.layerY are broken and deprecated in WebKit. They will be removed from the engine in the near future

They appear to be trying to violate SOP with requests from inside the IFRAME. They should probably be using CORS: http://en.wikipedia.org/wiki/Cross-origin_resource_sharing.

Herdict Web Browser Add-on

It is also possible to download an add-on here: http://www.herdict.org/participate/download.

The add-on is available fro Google Chrome, Firefox and Internet Explorer.

The add-on installs a toolbar that asks herdict for the profile of every site the user accesses. If a site that is being visited has been reported blocked from the users country the icon is either yellow or red. The user can report the reachability of the site by clicking on the icon and filling in the information similar to how is done with Herdict Reporter.

Checklist

Is the tool Open Source?

The source is not explicitly released, but it's a web application so the client side part can be accessed. The core of the Reporter web application can be found here: http://www.herdict.org/includes/js/reporter.js

Is the data collected made public?

The data is accessible publicly and is viewable from the web site web application. However it is not possible to download more than 500 records per time.

https://www.herdict.org/explore/data?fs=2245#fs=

Is the data format that is used for publication easy to interact with?

The raw data is available in .csv. The format of the csv file is:

Date,URL,Type,Country,isp,Location,Comments.

What license is used for releasing the data

Licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License

Are the methodologies explained?

Yes.

Is the tool to be used by the general public?

Yes.

If so, is the user warned of possible risks that he may incur when running the tool?

Ni.

Does the data collected by the tool include potentially sensitive information?

Yes.

More broad questions that should be answered when evaluating tools are:

What kind of tests does the tool perform?

The tool relies only on user feedback so it does not perform any test in itself. What Herdict Reporter does is visualize in random order a set of websites.

How accurate are the tests?

Since it relies on user feedback the accuracy of the tool may vary as the user may be reporting for blocked something that is not in fact a sign of blockage.

What claims does the tool make?

To crowd source reporting of site inaccessibility.

Are the claims satisfied?

Yes.

How does the reporting system work?

The reports are done by issuing a GET Request to an API that is provided by the backend herdict website.

The reports for the Herdict Reporter are different than those of Herdict Web. The POST requests being done by Herdict Reporter do not appear to be made over HTTPs, but are done in cleartext to this address:

Method: POST http://www.herdict.org/participate/reporter/1

siteInaccessibleAjax:
testCountry:IT
closeWindow:false
defaultISPName:FREE INTERNET DIAL-UP SERVICES
defaultCountryShortName:IT
returnInSameWindow:false
returnPage:
report.url:googleusercontent.com
report.country.shortName:IT
report.ispName:FREE INTERNET DIAL-UP SERVICES
honey:
report.location:
report.tag:
alternateTag:
report.comments:
_sourcePage:t6w40Ricm2iK0UZ4U8kCl4L43kbS7Rsb2rHKBHOWRsKs9N-SMZviYRK3g32KYH2E
__fp:3-bxLZNZ_-ZErfCjTBA60RDg096X3wIjQRddM1U4tBdTxVG4QtABQUTPbxOCNMy_CyX0SMaPGRfbKVaAN2ZBUQ==

For Herdict Add-on reporter on Firefox the requests are done over HTTPS via GET to this address:

http://www.herdict.org/web/action/ajax/plugin/report
                  + "&report.url=" + encodeURIComponent(this._rot13(document.getElementById("url").value))
                  + "&report.country.shortName=" + document.getElementById("country").selectedItem.value
                  + "&report.ispName=" + encodeURIComponent(document.getElementById("isp").value)
                  + "&report.location=" + document.getElementById("location").selectedItem.value
                  + "&report.interest=" + document.getElementById("interest").selectedItem.value
                  + "&report.reason=" + document.getElementById("reason").selectedItem.value
                  + "&report.sourceId=1"
                  + "&report.tag=" + (("tag.other" == ddlTag.selectedItem.value) ? document.getElementById("categoryOther").value : ddlTag.selectedItem.value)
                  + "&report.comments=" + encodeURIComponent(document.getElementById("comments").value)
                  + "&defaultCountryCode=" + encodeURIComponent(this.country)
                  + "&defaultISPName=" + encodeURIComponent(this.isp)
                  + "&encoding=" + "ROT13";

Is confidentiality and integrity of data being reported maintained?

The data being transmitted to the backend system in the Firefox add-on is encrypted end to end.

On the website no encryption is enforced.

Even when the data is encrypted it does not enforce PFS. It allows the client to choose MD5 as a hash algorithm.

This is the output of sslscan:

$ sslscan herdict.org
                   _
           ___ ___| |___  ___ __ _ _ __
          / __/ __| / __|/ __/ _` | '_ \
          \__ \__ \ \__ \ (_| (_| | | | |
          |___/___/_|___/\___\__,_|_| |_|

                  Version 1.8.0
             http://www.titania.co.uk
        Copyright Ian Ventura-Whiting 2009

Testing SSL server herdict.org on port 443

  Supported Server Cipher(s):
    Accepted  SSLv2  168 bits  DES-CBC3-MD5
    Accepted  SSLv2  56 bits   DES-CBC-MD5
    Accepted  SSLv2  40 bits   EXP-RC2-CBC-MD5
    Accepted  SSLv2  128 bits  RC2-CBC-MD5
    Accepted  SSLv2  40 bits   EXP-RC4-MD5
    Accepted  SSLv2  128 bits  RC4-MD5
    Rejected  N/A              SSLv3  128 bits  ADH-SEED-SHA
    Rejected  N/A              SSLv3  128 bits  DHE-RSA-SEED-SHA
    Rejected  N/A              SSLv3  128 bits  DHE-DSS-SEED-SHA
    Rejected  N/A              SSLv3  128 bits  SEED-SHA
    Rejected  N/A              SSLv3  256 bits  ADH-AES256-SHA
    Accepted  SSLv3  256 bits  DHE-RSA-AES256-SHA
    Rejected  N/A              SSLv3  256 bits  DHE-DSS-AES256-SHA
    Accepted  SSLv3  256 bits  AES256-SHA
    Rejected  N/A              SSLv3  128 bits  ADH-AES128-SHA
    Accepted  SSLv3  128 bits  DHE-RSA-AES128-SHA
    Rejected  N/A              SSLv3  128 bits  DHE-DSS-AES128-SHA
    Accepted  SSLv3  128 bits  AES128-SHA
    Rejected  N/A              SSLv3  168 bits  ADH-DES-CBC3-SHA
    Rejected  N/A              SSLv3  56 bits   ADH-DES-CBC-SHA
    Rejected  N/A              SSLv3  40 bits   EXP-ADH-DES-CBC-SHA
    Rejected  N/A              SSLv3  128 bits  ADH-RC4-MD5
    Rejected  N/A              SSLv3  40 bits   EXP-ADH-RC4-MD5
    Accepted  SSLv3  168 bits  EDH-RSA-DES-CBC3-SHA
    Accepted  SSLv3  56 bits   EDH-RSA-DES-CBC-SHA
    Accepted  SSLv3  40 bits   EXP-EDH-RSA-DES-CBC-SHA
    Rejected  N/A              SSLv3  168 bits  EDH-DSS-DES-CBC3-SHA
    Rejected  N/A              SSLv3  56 bits   EDH-DSS-DES-CBC-SHA
    Rejected  N/A              SSLv3  40 bits   EXP-EDH-DSS-DES-CBC-SHA
    Accepted  SSLv3  168 bits  DES-CBC3-SHA
    Accepted  SSLv3  56 bits   DES-CBC-SHA
    Accepted  SSLv3  40 bits   EXP-DES-CBC-SHA
    Accepted  SSLv3  40 bits   EXP-RC2-CBC-MD5
    Accepted  SSLv3  128 bits  RC4-SHA
    Accepted  SSLv3  128 bits  RC4-MD5
    Accepted  SSLv3  40 bits   EXP-RC4-MD5
    Rejected  N/A              SSLv3  0 bits    NULL-SHA
    Rejected  N/A              SSLv3  0 bits    NULL-MD5
    Rejected  N/A              TLSv1  128 bits  ADH-SEED-SHA
    Rejected  N/A              TLSv1  128 bits  DHE-RSA-SEED-SHA
    Rejected  N/A              TLSv1  128 bits  DHE-DSS-SEED-SHA
    Rejected  N/A              TLSv1  128 bits  SEED-SHA
    Rejected  N/A              TLSv1  256 bits  ADH-AES256-SHA
    Accepted  TLSv1  256 bits  DHE-RSA-AES256-SHA
    Rejected  N/A              TLSv1  256 bits  DHE-DSS-AES256-SHA
    Accepted  TLSv1  256 bits  AES256-SHA
    Rejected  N/A              TLSv1  128 bits  ADH-AES128-SHA
    Accepted  TLSv1  128 bits  DHE-RSA-AES128-SHA
    Rejected  N/A              TLSv1  128 bits  DHE-DSS-AES128-SHA
    Accepted  TLSv1  128 bits  AES128-SHA
    Rejected  N/A              TLSv1  168 bits  ADH-DES-CBC3-SHA
    Rejected  N/A              TLSv1  56 bits   ADH-DES-CBC-SHA
    Rejected  N/A              TLSv1  40 bits   EXP-ADH-DES-CBC-SHA
    Rejected  N/A              TLSv1  128 bits  ADH-RC4-MD5
    Rejected  N/A              TLSv1  40 bits   EXP-ADH-RC4-MD5
    Accepted  TLSv1  168 bits  EDH-RSA-DES-CBC3-SHA
    Accepted  TLSv1  56 bits   EDH-RSA-DES-CBC-SHA
    Accepted  TLSv1  40 bits   EXP-EDH-RSA-DES-CBC-SHA
    Rejected  N/A              TLSv1  168 bits  EDH-DSS-DES-CBC3-SHA
    Rejected  N/A              TLSv1  56 bits   EDH-DSS-DES-CBC-SHA
    Rejected  N/A              TLSv1  40 bits   EXP-EDH-DSS-DES-CBC-SHA
    Accepted  TLSv1  168 bits  DES-CBC3-SHA
    Accepted  TLSv1  56 bits   DES-CBC-SHA
    Accepted  TLSv1  40 bits   EXP-DES-CBC-SHA
    Accepted  TLSv1  40 bits   EXP-RC2-CBC-MD5
    Accepted  TLSv1  128 bits  RC4-SHA
    Accepted  TLSv1  128 bits  RC4-MD5
    Accepted  TLSv1  40 bits   EXP-RC4-MD5
    Rejected  N/A              TLSv1  0 bits    NULL-SHA
    Rejected  N/A              TLSv1  0 bits    NULL-MD5

  Prefered Server Cipher(s):
    SSLv2  168 bits  DES-CBC3-MD5
    SSLv3  256 bits  DHE-RSA-AES256-SHA
    TLSv1  256 bits  DHE-RSA-AES256-SHA

  SSL Certificate:
    Version: 2
    Serial Number: 23991
    Signature Algorithm: sha1WithRSAEncryption
    Issuer: /C=US/O=GeoTrust, Inc./CN=GeoTrust SSL CA
    Not valid before: Jan 26 11:48:09 2011 GMT
    Not valid after: Mar 21 12:05:53 2013 GMT
    Subject: /serialNumber=RtseYs58TwL7oDpzgzF8SPOLnDat3n4-/C=US/ST=Massachusetts/L=Cambridge/O=Berkman Center for Internet & Society/OU=IT/Systems Group/CN=adam.law.harvard.edu
    Public Key Algorithm: rsaEncryption
    RSA Public Key: (2048 bit)
      Modulus (2048 bit):
          00:c0:cb:e1:7e:a4:a3:ea:86:56:98:8b:42:7d:08:
          67:a2:fe:b4:42:1d:1f:ce:3c:d9:c7:30:04:7d:3c:
          10:b7:ce:07:54:07:50:b5:89:b8:c9:c4:40:ab:05:
          95:a9:41:28:12:80:8a:de:e4:6a:2a:af:e6:62:60:
          dc:71:18:c2:b5:14:fe:02:ac:09:6e:5d:72:1b:ab:
          8b:ea:ca:dc:54:e3:83:16:b1:96:f3:e4:9a:56:79:
          55:3a:87:b4:26:33:e6:62:45:55:12:e4:97:50:e8:
          63:0f:98:26:0d:0e:31:d6:62:96:28:2c:d0:28:93:
          72:8b:11:db:16:79:bb:bf:1b:df:c1:25:fa:4f:93:
          2c:6e:43:c5:0f:f5:83:e6:82:f4:55:11:02:31:27:
          c3:07:74:c4:63:3a:43:f4:8a:cb:83:d0:73:47:56:
          23:aa:19:1a:f7:ec:69:6c:fd:3d:c0:b6:4b:7d:98:
          10:a8:66:73:eb:c3:15:e1:fb:8c:5a:18:6e:18:8c:
          80:bb:02:a4:30:30:00:e5:b9:25:32:58:ae:af:76:
          c2:c1:63:55:cb:76:20:19:8b:20:f3:5a:5f:76:50:
          91:9e:c7:6d:1f:be:2d:55:74:80:00:a9:49:9d:4c:
          a3:f5:42:e6:9a:24:5c:67:c1:82:73:d2:d5:7c:da:
          89:67
      Exponent: 65537 (0x10001)
    X509v3 Extensions:
      X509v3 Authority Key Identifier: 
        keyid:42:79:54:1B:61:CD:55:2B:3E:63:D5:3C:48:57:F5:9F:FB:45:CE:4A

      X509v3 Key Usage: critical
        Digital Signature, Key Encipherment
      X509v3 Extended Key Usage: 
        TLS Web Server Authentication, TLS Web Client Authentication
      X509v3 Subject Alternative Name: 
        DNS:cyber.law.harvard.edu, DNS:www.berkman.harvard.edu, DNS:www.herdict.org, DNS:dev.herdict.org, DNS:www.nardikt.ru, DNS:dev.nardikt.ru, DNS:www.citmedialaw.org, DNS:www.omln.org, DNS:www.chillingeffects.org, DNS:images.chillingeffects.org, DNS:adam.law.harvard.edu
      X509v3 CRL Distribution Points: 
        URI:http://gtssl-crl.geotrust.com/crls/gtssl.crl

      X509v3 Subject Key Identifier: 
        82:A7:2F:ED:A8:85:18:FE:CE:62:C6:94:30:0A:E2:FE:63:0C:83:F6
      X509v3 Basic Constraints: critical
        CA:FALSE
      Authority Information Access: 
        CA Issuers - URI:http://gtssl-aia.geotrust.com/gtssl.crt

  Verify Certificate:
    Certificate passed verification

What are it's strengths

  • Censorship data can be easily collected from various parts of the planet. The user wishing to contribute is not require to install special software and can run everything from inside of a web browser.
  • Pretty UI

What are it's weaknesses

  • Encryption is not enforced on the website and when encryption is used it allows weak cipher suites.
  • Potentially inaccurate data collected from users.

Bottom line

As they state in their about page: "Whereas OpenNet views Internet filtering through an academic lens, Herdict uses crowdsourcing to learn about and present a real time view of the experiences of users around the globe", so the data collected by Herdict should be taken with the right amount of caution, but it can be very valuable to have data in real time in places where there would be none.

Last modified 5 years ago Last modified on Jun 18, 2012, 8:54:06 AM