Opened 4 years ago

Closed 4 years ago

Last modified 4 years ago

#17337 closed defect (fixed)

urras is down

Reported by: atagar Owned by: ioerror
Priority: High Milestone:
Component: Core Tor/DirAuth Version:
Severity: Normal Keywords: urras
Cc: Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

Tracking ticket for urras' ongoing outage. Jake is well aware of the issue and working on it.

Child Tickets

Change History (11)

comment:1 Changed 4 years ago by micah

Severity: Normal

When did urras go down exactly?

I'm wondering what the plan is with urras, will it come back up soon?

With it down, it causes timeouts for clients who are connecting to the network, and slow startup times.

comment:2 Changed 4 years ago by atagar

Hi micah, are you still seeing issues? According to DocTor urras has been back for a few days. I was gonna update DocTor and resolve this ticket tomorrow (been on a trip), but if you're seeing ongoing issues I'd be curious about what they are.

comment:3 Changed 4 years ago by micah

Owner: set to ioerror
Status: newassigned

Well urras isn't voting. Maybe doctor just noticed that its pinging?

comment:4 Changed 4 years ago by atagar

Ahh, gotcha. I changed DocTor to stop considering urras an authority due to excessive notices so all it notices right now is that it's back with the authority flag.

Jake, any notion of what's up with urras?

comment:5 Changed 4 years ago by atagar

Jake is working on this now, and I've resumed DocTor notifications for the authority.

comment:6 Changed 4 years ago by ioerror

Resolution: fixed
Status: assignedclosed

OK - so good news - Urras is back up. Better news: I've renewed the certificate for a year. Even better news: the cert is now 2048bits! rather than 1024bits!

Hooray.

Now, I expect some other horror to show up but let us prematurely close this ticket and call it finished.

comment:7 Changed 4 years ago by ioerror

I just got some errors from DocTor:


WARNING: The consensuses published by the following directory authorities are more than one hour old and therefore not fresh anymore: urras
WARNING: The certificate of the following directory authority expires within the next two weeks: maatuska (2015-11-17 20-57-57)
NOTICE: Consensus fetched from maatuska was missing the following authority signatures: urras
NOTICE: Consensus fetched from tor26 was missing the following authority signatures: urras
NOTICE: Consensus fetched from longclaw was missing the following authority signatures: urras
NOTICE: Consensus fetched from dizum was missing the following authority signatures: urras
NOTICE: Consensus fetched from gabelmoo was missing the following authority signatures: urras
NOTICE: Consensus fetched from moria1 was missing the following authority signatures: urras
NOTICE: Consensus fetched from dannenberg was missing the following authority signatures: urras
NOTICE: Consensus fetched from Faravahar was missing the following authority signatures: urras

Unsure why it is missing signatures but I'm gonna let it sit for an hour and then go from there...

comment:8 Changed 4 years ago by ioerror

Urras appears to have upgraded to a 2048 bit key and this is reflected in the consensus health ( https://consensus-health.torproject.org/ ). However, it also reports:

Missing Signature! Valid-after time of auth's displayed consensus: 2015-11-10 22:00:00

Upon looking at the logs for Urras I see:

Nov 10 23:50:01.000 [notice] Time to vote.
Nov 10 23:50:04.000 [notice] Choosing valid-after time in vote as 2015-11-11 00:00:00: consensus_set=1, last_interval=3600
Nov 10 23:50:05.000 [notice] Vote posted.
Nov 10 23:50:06.000 [notice] Uploaded a vote to dirserver 199.254.238.52:80
Nov 10 23:50:08.000 [notice] Uploaded a vote to dirserver 154.35.175.225:80
Nov 10 23:50:09.000 [notice] Uploaded a vote to dirserver 128.31.0.39:9131
Nov 10 23:50:15.000 [notice] Uploaded a vote to dirserver 194.109.206.212:80
Nov 10 23:50:16.000 [notice] Uploaded a vote to dirserver 86.59.21.38:80
Nov 10 23:50:16.000 [notice] Uploaded a vote to dirserver 131.188.40.189:80
Nov 10 23:50:16.000 [notice] Uploaded a vote to dirserver 193.23.244.244:80
Nov 10 23:50:23.000 [notice] Uploaded a vote to dirserver 171.25.193.9:443
Nov 10 23:52:31.000 [notice] Time to fetch any votes that we're missing.
Nov 10 23:52:31.000 [notice] We're missing votes from 8 authorities (D586D18309DED4CD6D57C18FDB97EFA96D330566 14C131DFC5C6F93646BE72FA1401C02A8DF2E8B4 E8A9C45EDE6D711294FADF8E7951F4DE6CA56B58 ED03BB616EB2F60BEC80151114BB25CEF515B226 585769C78764D58426B8B52B6651A5A71137189A 49015F787433103580E3B66A1707A00E60F2D15B EFCBE720AB3A82B99F9E953CD5BF50F7EEFC7B97 23D15D965BC35114467363C165C4F724B64B4F66). Asking every other authority for a copy.
Nov 10 23:52:31.000 [warn] Received http status code 404 ("Not found") from server '199.254.238.52:80' while fetching "/tor/status-vote/next/D586D18309DED4CD6D57C18FDB97EFA96D330566+14C131DFC5C6F93646BE72FA1401C02A8DF2E8B4+E8A9C45EDE6D711294FADF8E7951F4DE6CA56B58+ED03BB616EB2F60BEC80151114BB25CEF515B226+585769C78764D58426B8B52B6651A5A71137189A+49015F787433103580E3B66A1707A00E60F2D15B+EFCBE720AB3A82B99F9E953CD5BF50F7EEFC7B97+23D15D965BC35114467363C165C4F724B64B4F66.z".
Nov 10 23:52:31.000 [warn] Received http status code 404 ("Not found") from server '154.35.175.225:80' while fetching "/tor/status-vote/next/D586D18309DED4CD6D57C18FDB97EFA96D330566+14C131DFC5C6F93646BE72FA1401C02A8DF2E8B4+E8A9C45EDE6D711294FADF8E7951F4DE6CA56B58+ED03BB616EB2F60BEC80151114BB25CEF515B226+585769C78764D58426B8B52B6651A5A71137189A+49015F787433103580E3B66A1707A00E60F2D15B+EFCBE720AB3A82B99F9E953CD5BF50F7EEFC7B97+23D15D965BC35114467363C165C4F724B64B4F66.z".
Nov 10 23:52:31.000 [warn] Received http status code 404 ("Not found") from server '128.31.0.39:9131' while fetching "/tor/status-vote/next/D586D18309DED4CD6D57C18FDB97EFA96D330566+14C131DFC5C6F93646BE72FA1401C02A8DF2E8B4+E8A9C45EDE6D711294FADF8E7951F4DE6CA56B58+ED03BB616EB2F60BEC80151114BB25CEF515B226+585769C78764D58426B8B52B6651A5A71137189A+49015F787433103580E3B66A1707A00E60F2D15B+EFCBE720AB3A82B99F9E953CD5BF50F7EEFC7B97+23D15D965BC35114467363C165C4F724B64B4F66.z".
Nov 10 23:52:31.000 [warn] Received http status code 404 ("Not found") from server '194.109.206.212:80' while fetching "/tor/status-vote/next/D586D18309DED4CD6D57C18FDB97EFA96D330566+14C131DFC5C6F93646BE72FA1401C02A8DF2E8B4+E8A9C45EDE6D711294FADF8E7951F4DE6CA56B58+ED03BB616EB2F60BEC80151114BB25CEF515B226+585769C78764D58426B8B52B6651A5A71137189A+49015F787433103580E3B66A1707A00E60F2D15B+EFCBE720AB3A82B99F9E953CD5BF50F7EEFC7B97+23D15D965BC35114467363C165C4F724B64B4F66.z".
Nov 10 23:52:31.000 [warn] Received http status code 404 ("Not found") from server '193.23.244.244:80' while fetching "/tor/status-vote/next/D586D18309DED4CD6D57C18FDB97EFA96D330566+14C131DFC5C6F93646BE72FA1401C02A8DF2E8B4+E8A9C45EDE6D711294FADF8E7951F4DE6CA56B58+ED03BB616EB2F60BEC80151114BB25CEF515B226+585769C78764D58426B8B52B6651A5A71137189A+49015F787433103580E3B66A1707A00E60F2D15B+EFCBE720AB3A82B99F9E953CD5BF50F7EEFC7B97+23D15D965BC35114467363C165C4F724B64B4F66.z".
Nov 10 23:52:31.000 [warn] Received http status code 404 ("Not found") from server '171.25.193.9:443' while fetching "/tor/status-vote/next/D586D18309DED4CD6D57C18FDB97EFA96D330566+14C131DFC5C6F93646BE72FA1401C02A8DF2E8B4+E8A9C45EDE6D711294FADF8E7951F4DE6CA56B58+ED03BB616EB2F60BEC80151114BB25CEF515B226+585769C78764D58426B8B52B6651A5A71137189A+49015F787433103580E3B66A1707A00E60F2D15B+EFCBE720AB3A82B99F9E953CD5BF50F7EEFC7B97+23D15D965BC35114467363C165C4F724B64B4F66.z".
Nov 10 23:52:31.000 [warn] Received http status code 404 ("Not found") from server '131.188.40.189:80' while fetching "/tor/status-vote/next/D586D18309DED4CD6D57C18FDB97EFA96D330566+14C131DFC5C6F93646BE72FA1401C02A8DF2E8B4+E8A9C45EDE6D711294FADF8E7951F4DE6CA56B58+ED03BB616EB2F60BEC80151114BB25CEF515B226+585769C78764D58426B8B52B6651A5A71137189A+49015F787433103580E3B66A1707A00E60F2D15B+EFCBE720AB3A82B99F9E953CD5BF50F7EEFC7B97+23D15D965BC35114467363C165C4F724B64B4F66.z".
Nov 10 23:52:31.000 [warn] Received http status code 404 ("Not found") from server '86.59.21.38:80' while fetching "/tor/status-vote/next/D586D18309DED4CD6D57C18FDB97EFA96D330566+14C131DFC5C6F93646BE72FA1401C02A8DF2E8B4+E8A9C45EDE6D711294FADF8E7951F4DE6CA56B58+ED03BB616EB2F60BEC80151114BB25CEF515B226+585769C78764D58426B8B52B6651A5A71137189A+49015F787433103580E3B66A1707A00E60F2D15B+EFCBE720AB3A82B99F9E953CD5BF50F7EEFC7B97+23D15D965BC35114467363C165C4F724B64B4F66.z".



Nov 10 23:55:01.000 [notice] Time to compute a consensus.
Nov 10 23:55:01.000 [warn] We don't have enough votes to generate a consensus: 1 of 5
Nov 10 23:56:54.000 [warn] Router $A07FF746D9BA56C3F916BBD404307396BFA862E0~chaoscomputerclub33 at 217.115.10.133 uploaded a descriptor with a Ed25519 key but the <rsa,ed25519> keys don't match what they were before.
Nov 10 23:56:54.000 [warn] Router $A69221A7EC7498D2F88A0FB795261013FA36CAAE~Truie at 198.50.156.78 uploaded a descriptor with a Ed25519 key but the <rsa,ed25519> keys don't match what they were before.

I guess that somehow I have something strange happening again. Argh.

Last edited 4 years ago by nickm (previous) (diff)

comment:9 Changed 4 years ago by ioerror

The clock may have been off by just a few minutes. We'll know in an hour if that fixes it.

comment:10 Changed 4 years ago by ioerror

Darn. Nope. I'm out of ideas for tonight:

Nov 11 00:50:04.000 [notice] Choosing valid-after time in vote as 2015-11-11 01:00:00: consensus_set=1, last_interval=3600
Nov 11 00:50:05.000 [notice] Vote posted.
Nov 11 00:50:06.000 [notice] Uploaded a vote to dirserver 199.254.238.52:80
Nov 11 00:50:08.000 [notice] Uploaded a vote to dirserver 154.35.175.225:80
Nov 11 00:50:09.000 [notice] Uploaded a vote to dirserver 128.31.0.39:9131
Nov 11 00:50:15.000 [notice] Uploaded a vote to dirserver 194.109.206.212:80
Nov 11 00:50:16.000 [notice] Uploaded a vote to dirserver 131.188.40.189:80
Nov 11 00:50:16.000 [notice] Uploaded a vote to dirserver 86.59.21.38:80
Nov 11 00:50:16.000 [notice] Uploaded a vote to dirserver 193.23.244.244:80
Nov 11 00:50:25.000 [notice] Uploaded a vote to dirserver 171.25.193.9:443


Nov 11 00:52:31.000 [notice] Time to fetch any votes that we're missing.
Nov 11 00:52:31.000 [notice] We're missing votes from 8 authorities (D586D18309DED4CD6D57C18FDB97EFA96D330566 14C131DFC5C6F93646BE72FA1401C02A8DF2E8B4 E8A9C45EDE6D711294FADF8E7951F4DE6CA56B58 ED03BB616EB2F60BEC80151114BB25CEF515B226 585769C78764D58426B8B52B6651A5A71137189A 49015F787433103580E3B66A1707A00E60F2D15B EFCBE720AB3A82B99F9E953CD5BF50F7EEFC7B97 23D15D965BC35114467363C165C4F724B64B4F66). Asking every other authority for a copy.
Nov 11 00:52:31.000 [warn] Received http status code 404 ("Not found") from server '199.254.238.52:80' while fetching "/tor/status-vote/next/D586D18309DED4CD6D57C18FDB97EFA96D330566+14C131DFC5C6F93646BE72FA1401C02A8DF2E8B4+E8A9C45EDE6D711294FADF8E7951F4DE6CA56B58+ED03BB616EB2F60BEC80151114BB25CEF515B226+585769C78764D58426B8B52B6651A5A71137189A+49015F787433103580E3B66A1707A00E60F2D15B+EFCBE720AB3A82B99F9E953CD5BF50F7EEFC7B97+23D15D965BC35114467363C165C4F724B64B4F66.z".
Nov 11 00:52:31.000 [warn] Received http status code 404 ("Not found") from server '154.35.175.225:80' while fetching "/tor/status-vote/next/D586D18309DED4CD6D57C18FDB97EFA96D330566+14C131DFC5C6F93646BE72FA1401C02A8DF2E8B4+E8A9C45EDE6D711294FADF8E7951F4DE6CA56B58+ED03BB616EB2F60BEC80151114BB25CEF515B226+585769C78764D58426B8B52B6651A5A71137189A+49015F787433103580E3B66A1707A00E60F2D15B+EFCBE720AB3A82B99F9E953CD5BF50F7EEFC7B97+23D15D965BC35114467363C165C4F724B64B4F66.z".
Nov 11 00:52:31.000 [warn] Received http status code 404 ("Not found") from server '128.31.0.39:9131' while fetching "/tor/status-vote/next/D586D18309DED4CD6D57C18FDB97EFA96D330566+14C131DFC5C6F93646BE72FA1401C02A8DF2E8B4+E8A9C45EDE6D711294FADF8E7951F4DE6CA56B58+ED03BB616EB2F60BEC80151114BB25CEF515B226+585769C78764D58426B8B52B6651A5A71137189A+49015F787433103580E3B66A1707A00E60F2D15B+EFCBE720AB3A82B99F9E953CD5BF50F7EEFC7B97+23D15D965BC35114467363C165C4F724B64B4F66.z".
Nov 11 00:52:31.000 [warn] Received http status code 404 ("Not found") from server '194.109.206.212:80' while fetching "/tor/status-vote/next/D586D18309DED4CD6D57C18FDB97EFA96D330566+14C131DFC5C6F93646BE72FA1401C02A8DF2E8B4+E8A9C45EDE6D711294FADF8E7951F4DE6CA56B58+ED03BB616EB2F60BEC80151114BB25CEF515B226+585769C78764D58426B8B52B6651A5A71137189A+49015F787433103580E3B66A1707A00E60F2D15B+EFCBE720AB3A82B99F9E953CD5BF50F7EEFC7B97+23D15D965BC35114467363C165C4F724B64B4F66.z".
Nov 11 00:52:31.000 [warn] Received http status code 404 ("Not found") from server '193.23.244.244:80' while fetching "/tor/status-vote/next/D586D18309DED4CD6D57C18FDB97EFA96D330566+14C131DFC5C6F93646BE72FA1401C02A8DF2E8B4+E8A9C45EDE6D711294FADF8E7951F4DE6CA56B58+ED03BB616EB2F60BEC80151114BB25CEF515B226+585769C78764D58426B8B52B6651A5A71137189A+49015F787433103580E3B66A1707A00E60F2D15B+EFCBE720AB3A82B99F9E953CD5BF50F7EEFC7B97+23D15D965BC35114467363C165C4F724B64B4F66.z".
Nov 11 00:52:31.000 [warn] Received http status code 404 ("Not found") from server '131.188.40.189:80' while fetching "/tor/status-vote/next/D586D18309DED4CD6D57C18FDB97EFA96D330566+14C131DFC5C6F93646BE72FA1401C02A8DF2E8B4+E8A9C45EDE6D711294FADF8E7951F4DE6CA56B58+ED03BB616EB2F60BEC80151114BB25CEF515B226+585769C78764D58426B8B52B6651A5A71137189A+49015F787433103580E3B66A1707A00E60F2D15B+EFCBE720AB3A82B99F9E953CD5BF50F7EEFC7B97+23D15D965BC35114467363C165C4F724B64B4F66.z".
Nov 11 00:52:31.000 [warn] Received http status code 404 ("Not found") from server '86.59.21.38:80' while fetching "/tor/status-vote/next/D586D18309DED4CD6D57C18FDB97EFA96D330566+14C131DFC5C6F93646BE72FA1401C02A8DF2E8B4+E8A9C45EDE6D711294FADF8E7951F4DE6CA56B58+ED03BB616EB2F60BEC80151114BB25CEF515B226+585769C78764D58426B8B52B6651A5A71137189A+49015F787433103580E3B66A1707A00E60F2D15B+EFCBE720AB3A82B99F9E953CD5BF50F7EEFC7B97+23D15D965BC35114467363C165C4F724B64B4F66.z".
Nov 11 00:52:31.000 [warn] Received http status code 404 ("Not found") from server '171.25.193.9:443' while fetching "/tor/status-vote/next/D586D18309DED4CD6D57C18FDB97EFA96D330566+14C131DFC5C6F93646BE72FA1401C02A8DF2E8B4+E8A9C45EDE6D711294FADF8E7951F4DE6CA56B58+ED03BB616EB2F60BEC80151114BB25CEF515B226+585769C78764D58426B8B52B6651A5A71137189A+49015F787433103580E3B66A1707A00E60F2D15B+EFCBE720AB3A82B99F9E953CD5BF50F7EEFC7B97+23D15D965BC35114467363C165C4F724B64B4F66.z".
Nov 11 00:55:01.000 [notice] Time to compute a consensus.
Nov 11 00:55:01.000 [warn] We don't have enough votes to generate a consensus: 1 of 5

comment:11 Changed 4 years ago by arma

In your logs, you asked every other authority for any vote, and every one of them told you it had no votes.

Wondering about your clock is still a fine thought.

Note: See TracTickets for help on using tickets.