Opened 6 months ago

Closed 6 months ago

#30262 closed defect (not a bug)

stem.descriptor.remote not handling 'HTTP/1.0 404 Not found' gracefully

Reported by: starlight Owned by: atagar
Priority: Medium Milestone:
Component: Core Tor/Stem Version:
Severity: Normal Keywords:
Cc: Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

running (commit 4BE6695A)

torsocks download_descriptor.py -t server -f 7FE6E24BF6058EA55717C18D34FCD049307D8D2C --orport 171.25.193.9:80

one receives

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/stem/descriptor/remote.py", line 560, in _download_descriptors
Downloading server descriptor from 171.25.193.9:80...

    self.content, self.reply_headers = _download_from_orport(endpoint, self.compression, self.resource)
  File "/usr/local/lib/python3.7/site-packages/stem/descriptor/remote.py", line 973, in _download_from_orport
    raise stem.ProtocolError("Response should begin with HTTP success, but was '%s'" % str_tools._to_unicode(first_line))
stem.ProtocolError: Response should begin with HTTP success, but was 'HTTP/1.0 404 Not found'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/stem/descriptor/remote.py", line 560, in _download_descriptors
    self.content, self.reply_headers = _download_from_orport(endpoint, self.compression, self.resource)
  File "/usr/local/lib/python3.7/site-packages/stem/descriptor/remote.py", line 973, in _download_from_orport
    raise stem.ProtocolError("Response should begin with HTTP success, but was '%s'" % str_tools._to_unicode(first_line))
stem.ProtocolError: Response should begin with HTTP success, but was 'HTTP/1.0 404 Not found'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/tor/Downloads/../download_descriptor.py", line 137, in <module>
    main()
  File "/home/tor/Downloads/../download_descriptor.py", line 116, in main
    compression = [stem.descriptor.remote.Compression.GZIP]
  File "/usr/local/lib/python3.7/site-packages/stem/descriptor/remote.py", line 484, in run
    return list(self._run(suppress))
  File "/usr/local/lib/python3.7/site-packages/stem/descriptor/remote.py", line 495, in _run
    raise self.error
  File "/usr/local/lib/python3.7/site-packages/stem/descriptor/remote.py", line 560, in _download_descriptors
    self.content, self.reply_headers = _download_from_orport(endpoint, self.compression, self.resource)
  File "/usr/local/lib/python3.7/site-packages/stem/descriptor/remote.py", line 973, in _download_from_orport
    raise stem.ProtocolError("Response should begin with HTTP success, but was '%s'" % str_tools._to_unicode(first_line))
stem.ProtocolError: Response should begin with HTTP success, but was 'HTTP/1.0 404 Not found'

could be better

Child Tickets

Change History (6)

comment:1 Changed 6 months ago by starlight

stem.descriptor.remote also has a problem with empty documents:

NOTE: replaced actual relay fingerprint with randomly generated one from example above

$ curl -sv http://194.109.206.212:443/tor/server/fp/7FE6E24BF6058EA55717C18D34FCD049307D8D2C.z | openssl zlib -d
*   Trying 194.109.206.212...
* Connected to 194.109.206.212 (194.109.206.212) port 443 (#0)
> GET /tor/server/fp/7FE6E24BF6058EA55717C18D34FCD049307D8D2C.z HTTP/1.1
> Host: 194.109.206.212:443
> User-Agent: ...
> Accept: */*
>
* Empty reply from server
* Connection #0 to host 194.109.206.212 left intact
$ download_descriptor.py -t server -f 7FE6E24BF6058EA55717C18D34FCD049307D8D2C --orport 194.109.206.212:443                                                                                                                       
Downloading server descriptor from 194.109.206.212:443...                                                                  

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/stem/descriptor/remote.py", line 560, in _download_descriptors
    self.content, self.reply_headers = _download_from_orport(endpoint, self.compression, self.resource)
  File "/usr/local/lib/python3.7/site-packages/stem/descriptor/remote.py", line 961, in _download_from_orport
    with relay.create_circuit() as circ:
  File "/usr/local/lib/python3.7/site-packages/stem/client/__init__.py", line 270, in create_circuit
    for cell in self._msg(create_fast_cell):
  File "/usr/local/lib/python3.7/site-packages/stem/client/__init__.py", line 225, in _msg
    for received_cell in stem.client.cell.Cell.pop(response, self.link_protocol):
  File "/usr/local/lib/python3.7/site-packages/stem/client/cell.py", line 174, in pop
    circ_id, content = link_protocol.circ_id_size.pop(content)
  File "/usr/local/lib/python3.7/site-packages/stem/client/datatype.py", line 408, in pop
    return self.unpack(to_unpack), remainder
  File "/usr/local/lib/python3.7/site-packages/stem/client/datatype.py", line 401, in unpack
    raise ValueError('%s is the wrong size for a %s field' % (repr(packed), self.name))
ValueError: b'\x00' is the wrong size for a SHORT field

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/stem/descriptor/remote.py", line 560, in _download_descriptors
    self.content, self.reply_headers = _download_from_orport(endpoint, self.compression, self.resource)
  File "/usr/local/lib/python3.7/site-packages/stem/descriptor/remote.py", line 961, in _download_from_orport
    with relay.create_circuit() as circ:
  File "/usr/local/lib/python3.7/site-packages/stem/client/__init__.py", line 270, in create_circuit
    for cell in self._msg(create_fast_cell):
  File "/usr/local/lib/python3.7/site-packages/stem/client/__init__.py", line 225, in _msg
    for received_cell in stem.client.cell.Cell.pop(response, self.link_protocol):
  File "/usr/local/lib/python3.7/site-packages/stem/client/cell.py", line 174, in pop
    circ_id, content = link_protocol.circ_id_size.pop(content)
  File "/usr/local/lib/python3.7/site-packages/stem/client/datatype.py", line 408, in pop
    return self.unpack(to_unpack), remainder
  File "/usr/local/lib/python3.7/site-packages/stem/client/datatype.py", line 401, in unpack
    raise ValueError('%s is the wrong size for a %s field' % (repr(packed), self.name))
ValueError: b'\x00' is the wrong size for a SHORT field

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/tor/Downloads/../download_descriptor.py", line 137, in <module>
    main()
  File "/home/tor/Downloads/../download_descriptor.py", line 116, in main
    compression = [stem.descriptor.remote.Compression.GZIP]
  File "/usr/local/lib/python3.7/site-packages/stem/descriptor/remote.py", line 484, in run
    return list(self._run(suppress))
  File "/usr/local/lib/python3.7/site-packages/stem/descriptor/remote.py", line 495, in _run
    raise self.error
  File "/usr/local/lib/python3.7/site-packages/stem/descriptor/remote.py", line 560, in _download_descriptors
    self.content, self.reply_headers = _download_from_orport(endpoint, self.compression, self.resource)
  File "/usr/local/lib/python3.7/site-packages/stem/descriptor/remote.py", line 961, in _download_from_orport
    with relay.create_circuit() as circ:
  File "/usr/local/lib/python3.7/site-packages/stem/client/__init__.py", line 270, in create_circuit
    for cell in self._msg(create_fast_cell):
  File "/usr/local/lib/python3.7/site-packages/stem/client/__init__.py", line 225, in _msg
    for received_cell in stem.client.cell.Cell.pop(response, self.link_protocol):
  File "/usr/local/lib/python3.7/site-packages/stem/client/cell.py", line 174, in pop
    circ_id, content = link_protocol.circ_id_size.pop(content)
  File "/usr/local/lib/python3.7/site-packages/stem/client/datatype.py", line 408, in pop
    return self.unpack(to_unpack), remainder
  File "/usr/local/lib/python3.7/site-packages/stem/client/datatype.py", line 401, in unpack
    raise ValueError('%s is the wrong size for a %s field' % (repr(packed), self.name))
ValueError: b'\x00' is the wrong size for a SHORT field

comment:2 Changed 6 months ago by atagar

Status: newneeds_information

Hi starlight. Stem's descriptor.remote module propagates its underlying exception. When downloading through a DirPort this is a urllib2.HTTPError...

atagar@morrigan:~/Desktop/stem$ python download_descriptor.py -t server -f 7FE6E24BF6058EA55717C18D34FCD049307D8D2C
Downloading server descriptor from 128.31.0.34:9131...

Traceback (most recent call last):
  File "download_descriptor.py", line 131, in <module>
    main()
  File "download_descriptor.py", line 113, in main
    endpoints = [args.download_from],
  File "/home/atagar/Desktop/stem/stem/descriptor/remote.py", line 487, in run
    return list(self._run(suppress))
  File "/home/atagar/Desktop/stem/stem/descriptor/remote.py", line 498, in _run
    raise self.error
urllib2.HTTPError: HTTP Error 404: Servers unavailable

Whereas with an ORPort it is the same stem.ProtocolError...

atagar@morrigan:~/Desktop/stem$ python download_descriptor.py -t server -f 7FE6E24BF6058EA55717C18D34FCD049307D8D2C --orport 171.25.193.9:80
Downloading server descriptor from 171.25.193.9:80...

Traceback (most recent call last):
  File "download_descriptor.py", line 131, in <module>
    main()
  File "download_descriptor.py", line 113, in main
    endpoints = [args.download_from],
  File "/home/atagar/Desktop/stem/stem/descriptor/remote.py", line 487, in run
    return list(self._run(suppress))
  File "/home/atagar/Desktop/stem/stem/descriptor/remote.py", line 498, in _run
    raise self.error
stem.ProtocolError: Response should begin with HTTP success, but was 'HTTP/1.0 404 Not found'

What are you hoping for Stem to do? I suspect your concern might be the verbosity of exception chaining in which case that is an artifact of python3...

https://trac.torproject.org/projects/tor/ticket/16348

stem.descriptor.remote also has a problem with empty documents

Could you please re-try with the git codebase? I suspect this is already fixed - I get a 404 when I run the command you provided.

comment:3 Changed 6 months ago by starlight

My idea is more that Stem should handle two very common, unexceptional failure cases without throwing an exception, but rather by gracefully returning an error code.

comment:4 Changed 6 months ago by atagar

Hi starlight. Could you please describe from an API perspective what you mean by "gracefully returning an error code"? These methods are documented as returning the requested descriptor, and raising an exception if unable to do so, so I'm unsure what precisely you'd like for this method to do differently.

comment:5 Changed 6 months ago by starlight

Didn't realized that exceptions were the API mechanism for all errors, including known typical errors. Avoid it myself, reserving exception throwing for arrival of the unexpected.

Have to repeat, not graceful. If you are satisfied it's working "to spec," then feel free to close the ticket.

comment:6 Changed 6 months ago by atagar

Resolution: not a bug
Status: needs_informationclosed

Gotcha. Exceptions are not reserved for unexpected issues in python (or most languages). When you provide an invalid input methods should raise a ValueError...

>>> int('hello')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: invalid literal for int() with base 10: 'hello'

When file or network issues arise you should receive an IOError...

>>> open('/no/such/path', 'r')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IOError: [Errno 2] No such file or directory: '/no/such/path'

... and similarly, if the descriptor you request is unavailable or cannot be received you should receive an exception telling you that. In this particular case Stem is being uncommonly hands-off with its exception handling because I don't want to mask underlying network problems.

I could be talked into raising a "NoSuchDescriptor" or some other exception type instead, but it's definitely expected that we should raise something.

Note: See TracTickets for help on using tickets.