Opened 8 years ago

Closed 8 years ago

Last modified 8 years ago

#5209 closed defect (fixed)

ooni-probe enters some endless loop after scanning a few hundred bridges

Reported by: karsten Owned by: hellais
Priority: High Milestone:
Component: Archived/Ooni Version:
Severity: Keywords:
Cc: Actual Points:
Parent ID: #5028 Points:
Reviewer: Sponsor:

Description

We had two cases of ooni-probe not making any progress anymore after scanning ~500 and ~900 bridges. The python process was at 75% CPU though.

Child Tickets

Attachments (1)

sanitized-ooniprobe.log.gz (4.2 KB) - added by karsten 8 years ago.
Sanitized ooniprobe.log file from running into endless loop

Download all attachments as: .zip

Change History (10)

comment:1 Changed 8 years ago by runa

Milestone: Sponsor F: March 15, 2012

comment:2 Changed 8 years ago by runa

I'm setting up a temporary host later this week to test things (#5206), will update this ticket when I have more info.

Changed 8 years ago by karsten

Attachment: sanitized-ooniprobe.log.gz added

Sanitized ooniprobe.log file from running into endless loop

comment:3 Changed 8 years ago by karsten

Milestone: Sponsor F: March 15, 2012

I attached the sanitized ooniprobe.log from last running into the endless loop. This is from running commit e324fd8. I didn't spot what the problem is. The "Address already in use. Is Tor already running?" part is worrisome, but I couldn't confirm it's the problem here.

I'll re-run bridget with a much larger random port range. I hope it doesn't fill the entire disk with logs this time.

(Removing the sponsor milestone, because this ticket is already part of a project ticket that is assigned to that sponsor milestone.)

comment:4 Changed 8 years ago by karsten

With a random port range from 11000 to 65535 the last test completed within three hours. So, it could be that this problem is related to #5229. Once that's fixed we should see if the endless loop thing shows up again. If it doesn't, let's remove the "ANTANI" debug message and close this bug.

comment:5 Changed 8 years ago by hellais

I fixed #5229.

Karsten: You should check to see if this bug still occurs. If not I will remove the debug lines and close also this ticket.

comment:6 Changed 8 years ago by karsten

I just started another scan running 4c68347.

comment:7 Changed 8 years ago by karsten

Scan completed, so 4c68347 may have solved the problem. (Yay!)

I only found a probably harmless traceback at the end of the scan:

ooniprobe   : ERROR    Error in connecting to Tor Control port
Traceback (most recent call last):
  File "/usr/local/lib/python2.6/dist-packages/gevent-0.13.6-py2.6-linux-i686.egg/gevent/greenlet.py", line 390, in run
    result = self._run(*self.args, **self.kwargs)
  File "tests/bridget.py", line 301, in experiment
    return self.connect(bridge)
  File "tests/bridget.py", line 239, in connect
    self.logger.info("Bandwidth: %s" % bandwidth)
UnboundLocalError: local variable 'bandwidth' referenced before assignment
<Greenlet at 0x95d6b6c: <bound method BridgeT.experiment of <bridget.BridgeT instance at 0x94f25cc>>(data='[[address-scrubbed-for-trac-ticket]]')> failed with UnboundLocalError
ooniprobe   : INFO     [[address-scrubbed-for-trac-ticket]] bridge does not work (40 s timeout)

comment:8 in reply to:  7 ; Changed 8 years ago by hellais

Resolution: fixed
Status: newclosed

Replying to karsten:

Scan completed, so 4c68347 may have solved the problem. (Yay!)

Great!

I am going to close this ticket then.

I only found a probably harmless traceback at the end of the scan:

ooniprobe   : ERROR    Error in connecting to Tor Control port
Traceback (most recent call last):
  File "/usr/local/lib/python2.6/dist-packages/gevent-0.13.6-py2.6-linux-i686.egg/gevent/greenlet.py", line 390, in run
    result = self._run(*self.args, **self.kwargs)
  File "tests/bridget.py", line 301, in experiment
    return self.connect(bridge)
  File "tests/bridget.py", line 239, in connect
    self.logger.info("Bandwidth: %s" % bandwidth)
UnboundLocalError: local variable 'bandwidth' referenced before assignment
<Greenlet at 0x95d6b6c: <bound method BridgeT.experiment of <bridget.BridgeT instance at 0x94f25cc>>(data='[[address-scrubbed-for-trac-ticket]]')> failed with UnboundLocalError
ooniprobe   : INFO     [[address-scrubbed-for-trac-ticket]] bridge does not work (40 s timeout)

I am unable to find those lines in my current version of bridget. Maybe you have a slightly outdated version of it?

comment:9 in reply to:  8 Changed 8 years ago by karsten

Replying to hellais:

I am unable to find those lines in my current version of bridget. Maybe you have a slightly outdated version of it?

I was running 4c68347. Looks like the problem was fixed in f26af2d. Great! Leaving as closed.

Note: See TracTickets for help on using tickets.