As you can see, I have a vnet0. It has the link-local address that is specified as TransPort.
Now let's start tor:
$ sudo torOct 10 21:34:28.384 [notice] Tor 0.2.9.11 (git-aa8950022562be76) running on Linux with Libevent 2.0.21-stable, OpenSSL 1.0.2g and Zlib 1.2.8.
(...)
Oct 10 21:34:28.385 [notice] You configured a non-loopback address '[fe80::1c9a:c3ff:fec8:7768]:9040' for TransPort. This allows everybody on your local network to use your machine as a proxy. Make sure this is what you wanted.
(...)
Oct 10 21:34:28.386 [notice] Opening Transparent pf/netfilter listener on [fe80::1c9a:c3ff:fec8:7768]:9040Oct 10 21:34:28.386 [warn] Could not bind to fe80::1[c9a:c3ff:fec8:7768:9040 c9a:c3ff:fec8:7768:9040]: Invalid argument
As you can see, it is correctly striping the %vnet0 and reading my link-local address from the /etc/tor/torrc
It then tries to open the "pf/netfilter" and fails to bind, and says "invalid argument"!
Indeed, binding a link-local ipv6 address needs one more argument in the syscall to bind: the interface!
Other tests:
Trying with fancy notations like
TransPort [fe80::1c9a:c3ff:fec8:7768]%vnet0:9040
fails at parsing.
Trying with a global address (with ipV6 you can just add addresses to the interface) works but opens other headaches such as having to advertise a different router address to the clients.
Conclusion, this is either:
(bug) the implementation of the "interface" parameter when binding link-local addresses is missing or failing.
or
(documentation) it works and it is a documentation defect since nowhere we can find how to bind a link-local ipv6 address or even a working example.
Additional: there could be the exact same bug/missing documentation in other places where you can specify an ipv6 address.
Trac: Username: Zakhar
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information
Child items
0
Show closed items
No child items are currently assigned. Use child items to break down this issue into smaller parts.
Linked items
0
Link issues together to show that they're related.
Learn more.
Tor documents that IPv6 address syntax that is does support: [xxxx::xxxx:xxxx].
It does not support additional IPv6 syntax, including specifying interfaces using the % syntax.
The parsing of these kinds of addresses also looks like a bug - it should not parse fe80::1 as a separate part of the address:
Oct 10 21:34:28.386 [warn] Could not bind to fe80::1[c9a:c3ff:fec8:7768:9040 c9a:c3ff:fec8:7768:9040]: Invalid argument
Trac: Version: Tor: 0.2.9.11 to Tor: unspecified Summary: Tor doesn't bind to link-local (ipv6) addresses to Support IPv6 link-local interface addresses Type: defect to enhancement Milestone: N/Ato Tor: unspecified
Function names and lines below from the source downloaded from repo:apt-get source tor(Running Ubuntu 16.04x64)
The bug happens in the file: src/common/address.c
function: tor_addr_lookup() [line 239]
From there we go down #ifdef HAVE_GETADDRINFO
...since we are in Linux an we have that function.
[line 273] getaddrinfo (in fact sandbox_getaddrinfo) correctly parses the ipv6 address** including the additional interface** with the standard notation.
The field: sin6_scope_id of the struct sockaddr_in6 is correctly filled with this call.
In my case (debugged through nemiver), it happens this 32bits value is 3.
After that, the function is happy to have gotten an ipv6 address and calls tor_addr_from_in6 [line 298]... which (BUG) saves only the 16 bits of the ipv6 address and completely drops the field sin6_scope_id from the sockaddr structure.
Since the structure returned by getaddrinfo is correctly freed at [line 302] by freeaddrinfo, we have now lost definitively this essential sin6_scope_id!
A little bit later during the initialisation process, we will try to bind what we have read from the configuration file.
That happens in the file: src/or/connection.c in the function connection_listener_new
In this function at line [line 1214] we are now binding.
When we try binding our link-local interface, since we haven't saved the sin6_scope_id that we correctly got before, obviously the bind fails.
I have proven that, again with nemiver, I set a breakpoint on this line 1214 where we bind, and when we were binding our ipv6 link-local, I just added the value of the sin6_scope_id [value =3] by changing the memory buffer directly from nemiver.
Surprise... it works perfectly!
So the next thing now is to check on you last version (from your git) if that was fixed or if the bug is still there.
If the bug was fixed, I'll just upgrade my tor version to the first new version that fixes this use-case.
If the bug is still there, I'll propose on patch against 0.2.9-11 ... which hopefully will work on the last version.
_Sorry for the initial post, my addresses are all correct, it is only the syntax of this wiki that tricked me into mixing things up! _
Additional note: as you can see in src/common/compat.h
[line 559] we have:
/* Implementation of struct sockaddr_in6 on platforms that do not have * it. See notes on struct in6_addr. */#if !defined(HAVE_STRUCT_SOCKADDR_IN6)struct sockaddr_in6 { sa_family_t sin6_family; uint16_t sin6_port; // uint32_t sin6_flowinfo; struct in6_addr sin6_addr; // uint32_t sin6_scope_id;};#endif
So indeed someone copied this structure from systems that have it... and wrongfully removed the last essential element!
Ok, I have checked that it is unchanged in the last version, and I am proposing a "patch" against version 0.2.9.11
But before that, for the record, I'll detail the "rationale" of this "enhancement request"
Rationale
Use cases and addressing
Tor can be used in different way on the client machine.
Most commonly, people would use software bundles as "Tor Browser". For this kind of use, the Tor daemon needs only listen on the loopback interface since we are using it only from a single host.
Although less common, tor can be configured on a machine so that this machine behaves as a router redirecting traffic to tor for other machines on the LAN or even for VMs on the router itself.
For this usage, we need to make the tor daemon listen some ports on the relevant local interface of our router.
When using IPV4, the daemon would typically bind to addresses like 192.168.0.17 or 172.30.20.10 or the like. Those are "reserved" in the V4 addressing space and cannot be routed to the Internet.
When using IPV6, the "best practice" for routing inside a local network is to use "link-local" addresses. This is a new concept that comes with IPV6. These addresses are local and unique in the scope of a link. What this means, is that since we are configuring a router, this router has several interface, and you can validly assign the same "link local" address, for example ![fe80::1] to all your interfaces in your router. Thus, for routing purpose, you must add the interface to the address for disambiguation.
The RFC clearly says (last sentence first paragraph of section 4)
The scope of an address is encoded as part of the address
The tor daemon is also used for tor nodes: entry, relay, exit, etc... I won't go into that use case here.
Now, of course, the tor daemon running on the client/router machine also uses other addresses, it is the addresses of the sites you are visiting.
Those addresses should normally be "on the internet" meaning outside of your local machine (unless you are performing tests a locally installed tor network which is probably a super rare use case).
For ipV6 we call that "global" scope. "Global" scope addresses don't need the "zone index"... well since there are global as they name says, there is no ambiguity. You then transparently use "default zone index" (precisely you don't need to specify it as it is explained later in RFC4007).
Summary of what tor daemon supports
Let's forget the ultra rare case of testing a local tor network with relays, entry guards, etc... and focus of what the tor daemon can do on your local machine apart from routing packets to the Internet on global addresses (which is the main purpose and work well!)
"global" address? (not tested but should work since there are some entry/exits having ipV6!)
So basically, you CANNOT (in the respect of good practice) build an ipv6 "tor router" at the moment.
This is an acknowledged fact and a limitation.
Some might argue that it is indeed a "bug" to be not compliant with RFC4007, limiting the ipv6 address to 16 bytes and omitting the "zone index"... but I'm fine with "acknowledged limitation"!
(NOT RECOMMENDED) Workaround
Since ipv6 is much more easy than ipv4, one could also assign a global scope address to the interface and work with that.
But in doing so, you are opening a lot of other issues. First you would have to advertise this address as the route for SLAAC configured machines on you "tor router" LAN, which is not the default behaviour as you are supposed to use link local addresses.
Link local addresses are best practice, because should you have misconfigured some parts of your router, there is always a risk that your global addressable ipv6 be leaked. That is your "real" address and in a "tor" situation you absolutely do NOT want that!
You could try with ULA (Unique Local Address) with some more hedaches on the softwares to make that unique, but indeed avoid the risk of global addresses.
Patch
So to be helpful (hopefully) I am attaching a patch against 0.2.9.11
This patch touches only 2 files under src:
common/address.c
common/address.h
It is 8 lines of code.
Limitations of the patch
It works on a reasonably recent Linux (must have getaddrinfo).
I have no time nor the resource to test on other O.S.es
It is not a "correct architecture" correction, I consider it a "temporary patch".
The correct re-architecture would be to use, on each instance where an ipv6 address is used, a "full" ipv6 address as specified by RFC4007, without omitting the "zone index" (which on Linux is simplified as the 'scope_id', and that is explained also in the RFC).
Instead, I added in6_full to the tor_addr_t union, without touching in6_addr which would have had a lot of impacts everywhere.
in6_full is then used only in address.c where we need it, that is after getaddrinfo to save the 'scope_id' that is returned, and when we translate back a tor_add_t into the structure to open the socket, in order to restore the captured 'scope_id' so that the subsequent bind works.
Last limitation, I have not done extensive tests so far.
Applicability to the main branch (0.3.x)
Although the lines have changed, address.c looks the same around where I patched. So the patch should be applicable from 0.2.9.11 upwards.
AOB
@For the TOR team... do you need any other input (on your GIT, or else) from me to progress this "enhancement"?
This adds 4 bytes to every address in Tor, just for local addresses.
Maybe there's a smarter way of doing this?
We use address types AF_INET and AF_INET6, but there's no type or flag for full IPv6 addresses with scopes. We should add a flag or type, so that we don't confuse these different types of addresses. (Maybe it's sufficient to check if sin6_scope_id is non-zero?)
Documentation:
Tor patches need a changes file that contains a short changelog entry for the change (see tor/changes/ and make check-changes).
We also need to document this in the manual page under the appropriate options.
Code style:
We typically put spaces on both sides of the assignment operator:
There is probably a possibility to do the change with a local structure since the patch is limited to address.c/address.h, but then, should you need link-local ipv6 addresses elsewhere, you would have to patch again.
So yes, it adds 4 bytes to every address, but that is compliant with RFC4007 Section 4 that speaks about ipv6 addresses. Since ipv6 addresses are in fact 20 bytes long (if you don't want to rule out link-local addresses) I though it could be a more durable patch to go directly to the target 20 bytes.
Should you prefer a local patch for address.c/address.h, I can look into that.
I'll also look into the coding style (spaces and line length) and how to add the documentation in the man.
A last word about a limitation I forgot: I didn't change the printing. I mean now it accepts a parameter like:
TransPort [fe80::12:34%eth0]:9040
But you still get printed out that it successfully binds to [fe80::12:34], that is the %eth0 is stripped out.
That was to keep the patch as short as possible, but to go into details, we should also keep that bit of the address and print it out nicely.
I hope it is more compliant with coding standards.
Here is what I did:
removed a tab (spotted by make check-spaces)
split the assignment in 2 lines to stick to max line length
(Note since nothing is said in the "coding style" if the assignment operator is to be at the end of the first line, or beginning of the second, I tried to find other examples and followed them: assignment operator at the end of first line, just before LF)
provided a change description, categorized as "Minor feature".
The change ticket describes itself as "Initial support"... because we know there is more work to do on link-local addresses!
What I didn't do:
Documentation: I didn't change the man/documentation. Rationale: in the man, I couldn't find any explanation on how to specify ipv6 addresses: enclosing them with square brackets, semi colons, abbreviations, etc... That is because it is "the standard way" to do it, hence it does not need documentation. Since we follow here "the standard way" to specify a link-local address, I didn't feel the urge to add anything in the documentation.
Code: I left the in6_full as my original patch for the moment (more below)
Indeed link-local addresses are tricky! Probably, it would be best to open a discussion on the mailing list about the correct way to handle them.
My understanding is that they only have a meaning locally to a host. There is no provision in the ipv6 header to ever route a link-local ipv6 address (and RFC4291 explicitly says that link-local addresses MUST NOT be routed), you have to use a ULA or global address. But, ipv6 addresses used in a "local/host" context (such as when binding) can require this scope_id. On the other hand, ipv6 addresses used for routing traffic do never need that and logically you can't put the scope_id in the ipv6 header!
This distinction was not needed with ipv4, thus is not coded!
To my opinion, the proper modification would probably be to create a new structure instead of tor_addr_t, like: tor_local_addr_t, when we use an address locally (binding for instance). This tor_local_addr_t will definitely needs 20 bytes for ipv6, whereas the previous tor_addr_t do not need that since we are routing.
Doing that properly would probably be a Major architecture change in tor: distinguishing between local stuff, and routed stuff, and assigning each to their own addresses.
The question you asked in your code review about knowing whether it is a AF_INET6/full or AF_INET6/partial is only for local stuff since for routing there is no scope_id. For local addresses, the simpler is to consider they always have a scope, be it zero when not needed.
Also note, about this question, that you could always use in6_full when the family is AF_INET6, in6_full includes in6_addr, and in6_addr is at he same offset in the union. Thus, the other solution is to drop in6_addr and use in6_full instead everywhere. Doing so, the family can serve as a flag again, as it is best practice for unions to have a flag.
Alternate hack: if bumping the tor_addr_t to 20 bytes is too much, and you don't want to undertake any Major architecture change, BSD/OSX is using a (horrible) hack to specify link_local address. Instead of having fe80::3%17 (17 as an example, being the interface number as seen by the local system), they use: fe80:17::3. The second 16bits word is used to store the scope_id.
This is definitely a (horrible) hack with possible severe implications in the future. It produces an illegal link-local address since https://tools.ietf.org/html/rfc4291#section-2.5.6 defines the remaining 54 bits of the fe80/10 subnet to be 0. Should the norm change, this hack will fail.
There is also a limitation since the scope_id is 32bits and we use 16bits only here. We don't know the kernel's algorithm to allocate interface numbers, and 16bits could fail.
Because there are only 2 places where I touched address.c: one to save the scope_id, one to restore the scope_id before we bind, I could produce a patch using the BSD alternate method of specifying scope_id.
The save part would store in the second 16bits of the address (failing if scope_id >65535)
The restore part would test if we have fe80/10 and if so, restore the scope_id from the second 16bits and zero it in the copy.
We would not need in6_full (since it is the purpose of the hack)
There are potential unknown impacts if a function I didn't touch uses this "illegal" ipv6 link-local address. Most probably, for example, you would see it printed out when binding [fe80::3%eth1], that tor successfully binded to [fe80:17::3] (minor annoyance!). More testing to plan!
What is left to be done
Print the full ipv6 link-local address including the interface in the message to report the status of the bind operation (currently the patch does not address that, and the message is printed out without the interface part)
Test on systems other than Linux! Since getaddrinfo conforms to POSIX.1-2001 and POSIX.1-2008, the patch should work also on BSD and OSX. The Win32 documentation shows an API with the same parameters and the same ipv6 structure including ip6 scope_id, so I presume it might work, but I have no way to test that.
Questions: (the way forward)
Do you wish I deliver also the alternate hack patch so that you can choose/vote the patch you prefer?
Do you wish I open another 'low priority' ticket for what is still not implemented with link-local (such as printing nicely the interface part of the address)?
I hope it is more compliant with coding standards.
== Here is what I did: ==
removed a tab (spotted by make check-spaces)
split the assignment in 2 lines to stick to max line length
(Note since nothing is said in the "coding style" if the assignment operator is to be at the end of the first line, or beginning of the second, I tried to find other examples and followed them: assignment operator at the end of first line, just before LF)
provided a change description, categorized as "Minor feature".
The change ticket describes itself as "Initial support"... because we know there is more work to do on link-local addresses!
Thanks!
== What I didn't do: ==
Documentation: I didn't change the man/documentation. Rationale: in the man, I couldn't find any explanation on how to specify ipv6 addresses: enclosing them with square brackets, semi colons, abbreviations, etc... That is because it is "the standard way" to do it, hence it does not need documentation. Since we follow here "the standard way" to specify a link-local address, I didn't feel the urge to add anything in the documentation.
We recently documented the format we expect for IPv6:
Please rebase this patch to master, then document the changes to the IPv6 format.
(We won't accept new features in 0.2.9, we are only accepting security bugfixes.)
Code: I left the in6_full as my original patch for the moment (more below)
Indeed link-local addresses are tricky! Probably, it would be best to open a discussion on the mailing list about the correct way to handle them.
To my opinion, the proper modification would probably be to create a new structure instead of tor_addr_t, like: tor_local_addr_t, when we use an address locally (binding for instance). This tor_local_addr_t will definitely needs 20 bytes for ipv6, whereas the previous tor_addr_t do not need that since we are routing.
Doing that properly would probably be a Major architecture change in tor: distinguishing between local stuff, and routed stuff, and assigning each to their own addresses.
The question you asked in your code review about knowing whether it is a AF_INET6/full or AF_INET6/partial is only for local stuff since for routing there is no scope_id. For local addresses, the simpler is to consider they always have a scope, be it zero when not needed.
I think using tor_local_addr_t would be a good way of making this change, and it makes sure the address family is specified.
Also note, about this question, that you could always use in6_full when the family is AF_INET6, in6_full includes in6_addr, and in6_addr is at he same offset in the union. Thus, the other solution is to drop in6_addr and use in6_full instead everywhere. Doing so, the family can serve as a flag again, as it is best practice for unions to have a flag.
No, this would waste too much RAM, particularly on clients. (Tor stores a tor_addr_t for each of 7000 relays.)
Alternate hack: if bumping the tor_addr_t to 20 bytes is too much, and you don't want to undertake any Major architecture change, BSD/OSX is using a (horrible) hack to specify link_local address. Instead of having fe80::3%17 (17 as an example, being the interface number as seen by the local system), they use: fe80:17::3. The second 16bits word is used to store the scope_id.
No, we don't do hacks like this. They are hard to maintain, and lead to subtle bugs.
== What is left to be done ==
Print the full ipv6 link-local address including the interface in the message to report the status of the bind operation (currently the patch does not address that, and the message is printed out without the interface part)
This is an important part of the patch, otherwise we can't diagnose errors.
Test on systems other than Linux! Since getaddrinfo conforms to POSIX.1-2001 and POSIX.1-2008, the patch should work also on BSD and OSX. The Win32 documentation shows an API with the same parameters and the same ipv6 structure including ip6 scope_id, so I presume it might work, but I have no way to test that.
I can do this for macOS, and we have some BSD folk around.
Windows testing is hard, but we don't have that many relays or relay testers on Windows, so as long as it doesn't break existing addresses, it should be ok.
== Questions: (the way forward) ==
Do you wish I deliver also the alternate hack patch so that you can choose/vote the patch you prefer?
No, it is very unlikely it would be merged.
Do you wish I open another 'low priority' ticket for what is still not implemented with link-local (such as printing nicely the interface part of the address)?
No, that's an essential part of this ticket, otherwise we can't debug errors.
But you could create a git branch and put it in a separate commit.
So, since I'm apparently the first one looking into binding on ipv6 link-local addresses, and doing so in Ubuntu 16.04 which is non-rolling-release/no-new-feature for what concerns Tor, I'll keep the present incomplete 0.2.9 patch for my own things, until I can provide a full patch on master.
What I'll do next (after some holidays... and since it does not seem a lot of folk need binding ipv6!)
rebase on master (I have already checked the part I patched didn't change, it's only at a different place in address.c)
document on the %interface part (the explanation about "square brackets" was indeed added recently since it was not there in 0.2.9 where I checked)
complete the patch with the "printing" bit that is indeed missing
try to make it tor_local_addr_t or tor_non_routed_addr_t so that we don't propagate the scope_id to addresses that don't need it.
Bullets 1 and 2 are quite trivial.
Bullet 3 might take me some time to figure out from where the printing is done and how it is saving/restoring!
Bullet 4 is all another story... and i didn't imagine you would like un-maintanable hacks like using zones not meant for that to store what we need! I'll look into the "proposal" process, and tor_dev mailing list, people might come with nice suggestions to handle these "local/non-routed" addresses.
So, since I'm apparently the first one looking into binding on ipv6 link-local addresses, and doing so in Ubuntu 16.04 which is non-rolling-release/no-new-feature for what concerns Tor, I'll keep the present incomplete 0.2.9 patch for my own things, until I can provide a full patch on master.
What I'll do next (after some holidays... and since it does not seem a lot of folk need binding ipv6!)
rebase on master (I have already checked the part I patched didn't change, it's only at a different place in address.c)
document on the %interface part (the explanation about "square brackets" was indeed added recently since it was not there in 0.2.9 where I checked)
complete the patch with the "printing" bit that is indeed missing
Sounds good!
try to make it tor_local_addr_t or tor_non_routed_addr_t so that we don't propagate the scope_id to addresses that don't need it.
Bullets 1 and 2 are quite trivial.
Bullet 3 might take me some time to figure out from where the printing is done and how it is saving/restoring!
tor_addr_fmt and similar functions would be helpful here.
Bullet 4 is all another story... and i didn't imagine you would like un-maintanable hacks like using zones not meant for that to store what we need! I'll look into the "proposal" process, and tor_dev mailing list, people might come with nice suggestions to handle these "local/non-routed" addresses.
You could do something like this, and make all listener addresses tor_listener_addr_t:
$ ifconfigenp3s0 Link encap:Ethernet HWaddr cc:cc:cc:cc:cc:cc(...)adr inet6: fe80::329f:52e6:fa18:88e3/64 Scope:Lien$ ping6 -c2 -I enp3s0 ff02::1PING ff02::1(ff02::1) from fe80::329f:52e6:fa18:88e3 enp3s0: 56 data bytes64 bytes from fe80::329f:52e6:fa18:88e3: icmp_seq=1 ttl=64 time=0.085 ms64 bytes from fe80::224:d4ff:feab:9253: icmp_seq=1 ttl=64 time=0.437 ms (DUP!)--- ff02::1 ping statistics ---2 packets transmitted, 2 received, +1 duplicates, 0% packet loss, time 999msrtt min/avg/max/mdev = 0.085/0.203/0.437/0.165 ms$ ip -6 neighfe80::224:d4ff:feab:9253 dev enp3s0 lladdr 00:24:d4:ab:92:53 router DELAY$ curl -g -6 http://[fe80::224:d4ff:feab:9253%25enp3s0]/ -vv Trying fe80::224:d4ff:feab:9253...Connected to fe80::224:d4ff:feab:9253 (fe80::224:d4ff:feab:9253) port 80 (#0)GET / HTTP/1.1Host: fe80::224:d4ff:feab:9253User-Agent: curl/7.47.0Accept: */*< HTTP/1.1 200 OK< Server: nginx< Date: Thu, 02 Nov 2017 21:46:07 GMT< Content-Type: text/html; charset=utf-8< Transfer-Encoding: chunked< Connection: keep-alive< Expires: Thu, 02 Nov 2017 21:46:06 GMT< Cache-Control: no-cache< Cache-Control: must-revalidate,no-store< <!DOCTYPE HTML><html><head>(...)
What I am trying to show here, is that you CAN call your server (hence do TCP/IP) on an ipv6-link-local address, provided you add the interface specification.
This is the same as ipv4 calling your server on 192.168.0.254
Should you try to route such an ipv4 through Tor, it would probably fail because the exit server, as a security policy might have blocked 192.168.0.0/16
... but that can still work, as a test case, if the exit server is not blocking it for whatever insane reason.
If Tor would allow the same normally-local-ipv6 packet to be played by the exit server, that would be even worse, we would have to send the whole ipv6-link-local + interface to the exit server. This would also probably be even a bigger security issue than allowing ipv4-LAN-addresses on the exit node.
So let's says this use case for ipv6 is ruled out for now. We can route a normally-LAN address through tor with ipv4 (with little chance it succeeds) but cannot do the same for ipv6.
I guess we can live with that... and therefore to be clear shouldn't we change the ticket name to make it only about binding?
Proposition:
Support binding IPv6 link-local interface addresses
With such a title, it would be clearer that the patch is only about binding/listener, and does not intend to do anything about routing those addresses via Tor.
These needs_revision, tickets, tagged with 034-removed-*, are no longer in-scope for 0.3.4. We can reconsider any of them, if somebody does the necessary revision.
Trac: Milestone: Tor: 0.3.4.x-final to Tor: unspecified