Opened 6 months ago

Last modified 4 months ago

#34129 needs_review enhancement

Use STUN to determine NAT behaviour of peers

Reported by: cohosh Owned by: cohosh
Priority: Medium Milestone:
Component: Circumvention/Snowflake Version:
Severity: Normal Keywords:
Cc: cohosh, phw, arlolra, dcf Actual Points:
Parent ID: Points:
Reviewer: Sponsor: Sponsor28

Description (last modified by cohosh)

In investigating high proxy failure rates at clients (#33666) and the logistics of running our own STUN server (#25591), I came across RFC5780, which outlines steps to identify NATs with "endpoint independent mapping and filtering".

Section 4.3 outlines how a client can use a STUN server with an alternate IP address (returned in the first STUN binding request response) to determine how restrictive their NAT is.

This would be useful to match up clients with snowflake proxies that have compatible NATs. We still have the following questions:

  • are there public STUN servers that support this feature? Yes there are several candidates.
  • does the pion/stun library we use support this feature for STUN clients? Not yet but we can implement the feature.
  • If we're able to implement our own STUN server behind a domain-fronted connection (#25591), how can we implement this functionality?

I see at least one open source STUN server implementation that claims to support this (written in C): https://github.com/coturn/coturn

Child Tickets

Attachments (1)

main.go (6.0 KB) - added by cohosh 5 months ago.

Download all attachments as: .zip

Change History (15)

comment:1 Changed 6 months ago by cohosh

Description: modified (diff)

comment:2 Changed 5 months ago by cohosh

Owner: set to cohosh
Status: newassigned

comment:3 in reply to:  description ; Changed 5 months ago by cohosh

Replying to cohosh:

  • are there public STUN servers that support this feature?

In starting to answer this question, I first set up my own STUN server using coturn (linked above). I'm running this just temporarily on a digital ocean droplet to test out the feature. The droplet has two IP addresses: one that was assigned to it and one floating IP that I've attached to the droplet.

It didn't take much to set up, i just installed the debian package:

apt-get install coturn

and then edited the configuration file that's created at /etc/turnserver.conf to set the two listening IP addresses. This appears to be all that's required to add support for RFC 5780. I also disabled TURN so it's a STUN-only server.

Because of how floating IP addresses with digital ocean, we'll have to use the floating IP as the main STUN address and leave the allocated IP as the additional address. I sent a STUN binding request to 174.138.112.125:3478 and got back a reply with the OTHER-ADDRESS attribute in the response (in addition to the usual XOR-MAPPED-ADDRESS) which shows that it supports the NAT behaviour discovery feature.

Feel free to try it out.. I won't leave it up and running forever, just as long as we need to figure out how it works, find public STUN servers that have it, or work it into the broker for #25591.

My next step will be to look at the pion STUN library and see whether they support this feature/how to use it for our purposes.

comment:4 in reply to:  3 Changed 5 months ago by cohosh

Replying to cohosh:

My next step will be to look at the pion STUN library and see whether they support this feature/how to use it for our purposes.

Looks like this is unsupported at the moment, but there is an open issue for it and this RFC is referenced in their README. It wouldn't be an unreasonable task for us to implement this and submit a PR. We've had these PRs merged very quickly in the past.

comment:5 in reply to:  description Changed 5 months ago by cohosh

Replying to cohosh:

  • are there public STUN servers that support this feature?

I just modified the pion/stun library to check for this attribute tested it against this list of public STUN servers. 17 of them support RFC 5780, including stun.voip.blackberry.com and stun.voipgate.com which judging by their names might correspond to existing VoIP applications and so might be good candidates for #30579 anyway.

comment:6 Changed 5 months ago by cohosh

Description: modified (diff)

comment:7 Changed 5 months ago by dcf

I did apt install coturn to use the turnutils_stunclient program. I ran it and got the following output. I changed my actual IP address to 192.0.2.3.

$ turnutils_stunclient -f 174.138.112.125

========================================
RFC 5780 response 1
0: IPv4. Response origin: : 10.20.0.7:3478
0: IPv4. Other addr: : 68.183.200.83:3479
0: IPv4. UDP reflexive addr: 192.0.2.3:32960

turnutils_stunclient then hangs until I ctrl-C it.

Looking at a packet capture, there are 2 outgoing packets and 1 incoming packet.

Internet Protocol Version 4, Src: 192.168.0.2, Dst: 174.138.112.125
User Datagram Protocol, Src Port: 32960 (32960), Dst Port: stun (3478)
Session Traversal Utilities for NAT
    Message Type: 0x0001 (Binding Request)
        .... ...0 ...0 .... = Message Class: 0x00 Request (0)
        ..00 000. 000. 0001 = Message Method: 0x0001 Binding (0x001)
        ..0. .... .... .... = Message Method Assignment: IETF Review (0x0)
    Message Length: 0
    Message Cookie: 2112a442
    Message Transaction ID: c7766860ae3304ad85de6440

Internet Protocol Version 4, Src: 174.138.112.125, Dst: 192.168.0.2
User Datagram Protocol, Src Port: stun (3478), Dst Port: 32960 (32960)
Session Traversal Utilities for NAT
    [Request In: 1]
    [Time: 0.123115000 seconds]
    Message Type: 0x0101 (Binding Success Response)
        .... ...1 ...0 .... = Message Class: 0x10 Success Response (2)
        ..00 000. 000. 0001 = Message Method: 0x0001 Binding (0x001)
        ..0. .... .... .... = Message Method Assignment: IETF Review (0x0)
    Message Length: 56
    Message Cookie: 2112a442
    Message Transaction ID: c7766860ae3304ad85de6440
    Attributes
        XOR-MAPPED-ADDRESS: 192.0.2.3:32960
            Attribute Type: XOR-MAPPED-ADDRESS (0x0020)
                0... .... .... .... = Attribute Type Comprehension: Required (0x0)
                .0.. .... .... .... = Attribute Type Assignment: IETF Review (0x0)
            Attribute Length: 8
            Reserved: 00
            Protocol Family: IPv4 (0x01)
            Port (XOR-d): a1d2
            [Port: 32960]
            IP (XOR-d): e112a641
            [IP: 192.0.2.3]
        MAPPED-ADDRESS: 192.0.2.3:32960
            Attribute Type: MAPPED-ADDRESS (0x0001)
                0... .... .... .... = Attribute Type Comprehension: Required (0x0)
                .0.. .... .... .... = Attribute Type Assignment: IETF Review (0x0)
            Attribute Length: 8
            Reserved: 00
            Protocol Family: IPv4 (0x01)
            Port: 32960
            IP: 192.0.2.3
        RESPONSE-ORIGIN: 10.20.0.7:3478
            Attribute Type: RESPONSE-ORIGIN (0x802b)
                1... .... .... .... = Attribute Type Comprehension: Optional (0x1)
                .0.. .... .... .... = Attribute Type Assignment: IETF Review (0x0)
            Attribute Length: 8
            Reserved: 00
            Protocol Family: IPv4 (0x01)
            Port: 3478
            IP: 10.20.0.7
        OTHER-ADDRESS: 68.183.200.83:3479
            Attribute Type: OTHER-ADDRESS (0x802c)
                1... .... .... .... = Attribute Type Comprehension: Optional (0x1)
                .0.. .... .... .... = Attribute Type Assignment: IETF Review (0x0)
            Attribute Length: 8
            Reserved: 00
            Protocol Family: IPv4 (0x01)
            Port: 3479
            IP: 68.183.200.83
        SOFTWARE
            Attribute Type: SOFTWARE (0x8022)
                1... .... .... .... = Attribute Type Comprehension: Optional (0x1)
                .0.. .... .... .... = Attribute Type Assignment: IETF Review (0x0)
            Attribute Length: 4
            Software: None

Internet Protocol Version 4, Src: 192.168.0.2, Dst: 174.138.112.125
User Datagram Protocol, Src Port: 32960 (32960), Dst Port: stun (3478)
Session Traversal Utilities for NAT
    Message Type: 0x0001 (Binding Request)
        .... ...0 ...0 .... = Message Class: 0x00 Request (0)
        ..00 000. 000. 0001 = Message Method: 0x0001 Binding (0x001)
        ..0. .... .... .... = Message Method Assignment: IETF Review (0x0)
    Message Length: 16
    Message Cookie: 2112a442
    Message Transaction ID: 71532acd2efaf39f4cabcb5b
    Attributes
        XOR-RESPONSE-TARGET
            Attribute Type: XOR-RESPONSE-TARGET (0x0027)
                0... .... .... .... = Attribute Type Comprehension: Required (0x0)
                .0.. .... .... .... = Attribute Type Assignment: IETF Review (0x0)
            Attribute Length: 4
            Reserved: 80
            Protocol Family: Unknown (0xc1)
            Port (XOR-d): 0000
            [Port: 8466]
        CHANGE_REQUEST
            Attribute Type: CHANGE_REQUEST (0x0003)
                0... .... .... .... = Attribute Type Comprehension: Required (0x0)
                .0.. .... .... .... = Attribute Type Assignment: IETF Review (0x0)
            Attribute Length: 4
            .... .... .... .1.. = Change IP: Set
            .... .... .... ..1. = Change Port: Set

comment:8 in reply to:  7 ; Changed 5 months ago by cohosh

Replying to dcf:

I did apt install coturn to use the turnutils_stunclient program. I ran it and got the following output. I changed my actual IP address to 192.0.2.3.

$ turnutils_stunclient -f 174.138.112.125

========================================
RFC 5780 response 1
0: IPv4. Response origin: : 10.20.0.7:3478
0: IPv4. Other addr: : 68.183.200.83:3479
0: IPv4. UDP reflexive addr: 192.0.2.3:32960

turnutils_stunclient then hangs until I ctrl-C it.

Looking at a packet capture, there are 2 outgoing packets and 1 incoming packet.

I just tried this myself and got the same thing. It also when I tried other STUN servers that advertize the OTHER-ADDRESS attribute. I wonder if this is a bug in the coturn server (or at the very least the set up instructions).

From reading RFC 5780, the CHANGE-REQUEST attribute is used to determine the type of NAT filtering, and is not needed for determining the type of NAT mapping, but of course both are important to know.

Also, now that I'm looking at this. It seems there's a typo here in the advertized attribute from the client. I think it should be CHANGE-REQUEST, not CHANGE_REQUEST. That might be causing problems.

comment:9 in reply to:  8 Changed 5 months ago by cohosh

Replying to cohosh:

Also, now that I'm looking at this. It seems there's a typo here in the advertized attribute from the client. I think it should be CHANGE-REQUEST, not CHANGE_REQUEST. That might be causing problems.

Lol nevermind the code is correct.

Anyway, I'm writing up some client-side demo code now I'll post it later tonight and see if I can debug the issues with CHANGE-REQUEST.

Changed 5 months ago by cohosh

Attachment: main.go added

comment:10 Changed 5 months ago by cohosh

The attached go code demonstrates the use of RFC 5780 on the client side. It required only a few modifications to the pion/stun library: https://github.com/cohosh/stun

To summarize, it checks both the NAT mapping behaviour of the client and its NAT filtering behaviour. Each of these can be classified as:

  • address-independent (the least restrictive NATs)
  • address-dependent (ports are assigned/connections filtered based on the server address)
  • address and port-dependent (ports are assigned/connections filtered based on the server address and port)

Note: I'm only checking the mapping behaviour for address independence, not port independence just yet because it requires an additional configuration change to my coturn server.

You can test it out by doing the following:

go build main.go
./main --server 174.138.112.125:3478

Note that the filtering test relies on a time out and so will take up to a minute to complete (it won't hang forever though). I suspect this is the problem we ran into with the coturn utils. Maybe too long a time out or a buggy client that waits forever for a filtered response.

Last edited 5 months ago by cohosh (previous) (diff)

comment:11 Changed 5 months ago by cohosh

I submitted a PR to upstream the changes to pion/stun: https://github.com/pion/stun/pull/33

There are a couple ways to move forward with this. I'm suggesting the following steps:

  • Do NAT discovery at the proxy and use that to decide how often they poll

This is actually more useful for webextension users to do than standalone go proxies since we have way more of them. There's no functionality for this in the webrtc library we're using, but the stun package claims to have partial support for RFC 5780, and lists the attributes we need.

This basically replaces our datachannel failure heuristic with a NAT type heuristic. We can do both but should make sure they interact correctly.

  • Do NAT discovery at the proxy and client and send that information to the broker to match them up in a smarter way.

I'd like some feedback on this before moving forward since it will take some effort and be a substantial change to the way the broker works. I'm also hesitant to make decisions that prioritize some proxies over others that rely on proxy honestly since it increases the ability of a malicious party to DoS Snowflake with bad proxies. If they can falsely report a value to get their bad proxies prioritized over others, we'll be in a worse situation w.r.t. DoS than we are now.

comment:12 Changed 5 months ago by cohosh

I'm moving forward with implementing NAT discovery at the peers and noticed that we're actually using the web browser APIs, and not node dependencies at the end of the day. There's no corresponding mozilla API for the node 'stun' package so I guess we're going to have to bundle that dependency with our source somehow.

comment:13 Changed 5 months ago by cohosh

As mentioned in #34358, we don't have the libraries we need to implement this for our web-based proxies at the moment. Given that these far outnumber our standalone Go proxies, we might have to abandon this effort for the time being and rely on success-based heuristics to determine the proxy poll rate.

We could still do some smarter matching here if we have the client determine their NAT type and have the proxies report to the broker their success rate. If the client has a restrictive NAT, the broker could match them up with a high-success-rate proxy.

comment:14 Changed 4 months ago by cohosh

Status: assignedneeds_review

An update on this: users with symmetric NATs still have a very poor experience with snowflake. I'm working towards a solution that will have the broker match clients with restricted NATs with proxies on unrestricted NATs. Here's a start to that, with a common NAT discovery utility and an example of integration at the client:

https://github.com/cohosh/snowflake/pull/31

My next steps will be to:

  • implement NAT discovery on standalone proxies
  • modify the heuristics on web-based proxies
  • modify the broker to perform the updated proxy-matching
Note: See TracTickets for help on using tickets.