Opened 6 months ago

Closed 4 months ago

#28741 closed defect (fixed)

sbws should send scanner metadata as part of every HTTP request

Reported by: teor Owned by:
Priority: Medium Milestone: sbws: 1.0.x-final
Component: Core Tor/sbws Version:
Severity: Normal Keywords:
Cc: juga, teor Actual Points:
Parent ID: Points:
Reviewer: dgoulet Sponsor:

Description

We can't find out which torflow instances are using a bandwidth server:
https://lists.torproject.org/pipermail/tor-project/2018-December/002108.html

As far as I can tell, sbws doesn't provide any scanner info in its HTTP requests.

Here's some things we might want:

  • software-name: sbws
  • software-version
  • scanner-nickname
  • scanner-IP-address? (pro: doscover users who haven't set nickname, con: discover users)

Non-standard HTTP headers start with "X-".

Assigning to 1.0, because this is vital debugging info.

Child Tickets

Change History (11)

comment:1 Changed 6 months ago by iang

Non-standard HTTP headers start with "X-".

I thought RFC 6648 deprecated that convention?

comment:2 in reply to:  1 ; Changed 6 months ago by teor

Replying to iang:

Non-standard HTTP headers start with "X-".

I thought RFC 6648 deprecated that convention?

Thanks for letting us know.

Replying to teor:

Here's some things we might want:

  • software-name: sbws
  • software-version

These might be user-agent, unless requests sets its own user agent.

  • scanner-nickname

I'm not sure if there is a generic HTTP header for a nickname or other client identifier.

  • scanner-IP-address? (pro: discover users who haven't set nickname, con: discover users)

We should look for a generic HTTP header for the client IP address.
sbws doesn't guarantee any anonymity, and discovering rogue scanners is more important than the risk of malicious servers using the IP address.

Assigning to 1.0, because this is vital debugging info.

comment:3 in reply to:  2 ; Changed 6 months ago by juga

Replying to teor:

Here's some things we might want:

  • software-name: sbws
  • software-version

These might be user-agent, unless requests sets its own user agent.

Python Requests allows to setup custom User-Agent (http://docs.python-requests.org/en/master/community/faq/#custom-user-agents)

So, this would be: User-Agent: sbws/x.y.z

  • scanner-nickname

I'm not sure if there is a generic HTTP header for a nickname or other client identifier.

Can't find any in https://en.wikipedia.org/wiki/List_of_HTTP_header_fields#Standard_request_fields

Following https://tools.ietf.org/html/rfc6648#appendix-B ("incorporate the organization's name"), this could be: Tor-bwauth-Nickname:

  • scanner-IP-address? (pro: discover users who haven't set nickname, con: discover users)

We should look for a generic HTTP header for the client IP address.
sbws doesn't guarantee any anonymity, and discovering rogue scanners is more important than the risk of malicious servers using the IP address.

I also can't find any. It could be: Tor-bwauth-Address:

comment:4 in reply to:  3 Changed 5 months ago by teor

Replying to juga:

Replying to teor:

Here's some things we might want:

  • software-name: sbws
  • software-version

These might be user-agent, unless requests sets its own user agent.

Python Requests allows to setup custom User-Agent (http://docs.python-requests.org/en/master/community/faq/#custom-user-agents)

So, this would be: User-Agent: sbws/x.y.z

Ok.

  • scanner-nickname

I'm not sure if there is a generic HTTP header for a nickname or other client identifier.

Can't find any in https://en.wikipedia.org/wiki/List_of_HTTP_header_fields#Standard_request_fields

Following https://tools.ietf.org/html/rfc6648#appendix-B ("incorporate the organization's name"), this could be: Tor-bwauth-Nickname:

Some tweaks:

Words are capitalised; Abbreviations are rarely used; It's a bandwidth scanner:

Tor-Bandwidth-Scanner-Nickname: IDidntEditTheConfig

  • scanner-IP-address? (pro: discover users who haven't set nickname, con: discover users)

We should look for a generic HTTP header for the client IP address.
sbws doesn't guarantee any anonymity, and discovering rogue scanners is more important than the risk of malicious servers using the IP address.

I also can't find any. It could be: Tor-bwauth-Address:

The standard proxy client address header is:
Forwarded: for=192.0.2.1

comment:5 in reply to:  2 Changed 5 months ago by juga

Replying to teor:

sbws doesn't guarantee any anonymity, and discovering rogue scanners is more important than the risk of malicious servers using the IP address.

Thinking more this, i'm not totally convinced. If we find rogue scanners, are we going to try to contact them?.
I'm also not sure if that might be a problem on the server side due data privacy laws.
What about creating a random unique identifier the first time that it's saved locally so the scanner can always send the same?.

comment:6 Changed 5 months ago by juga

I thought it would be useful to have Tor version and library version in the user-agent too.
I've added a uuid field for now. I can change it to IP address.
The header would looks like:
'Tor-Bandwidth-Scanner-UUID': '4c0cf773-46d9-4633-9c3e-57a26b250be1',
'User-Agent': 'sbws/1.0.3-dev0 (kernel-version-platform-os) Python/x.y.z Requests/2.19.1 Stem/1.7.0 Tor/0.x.y.z (git-sha)'
'Tor-Bandwidth-Scanner-Nickname': 'foo'

https://github.com/torproject/sbws/pull/315

comment:7 Changed 5 months ago by teor

Ok, then we can block scanners if we need to.

comment:8 Changed 5 months ago by juga

Status: newneeds_review

comment:9 Changed 4 months ago by dgoulet

Reviewer: dgoulet

comment:10 Changed 4 months ago by dgoulet

Status: needs_reviewmerge_ready

lgtm;

comment:11 Changed 4 months ago by juga

Resolution: fixed
Status: merge_readyclosed

Merged

Note: See TracTickets for help on using tickets.