Detect when sbws stalls and log backtrace

Since it's possible that there will be other bug we haven't found yet, detect when sbws is stalling. Also print backtrace so that operators can open tickets with the backtrace and help to find the bug.

changed milestone to %sbws: 1.0.x-final

Trac:
Parent Ticket: #28663 (moved)

added component::core tor/sbws milestone::sbws: 1.0.x-final parent::28663 priority::medium resolution::fixed reviewer::dgoulet severity::normal status::closed type::defect version::sbws 1.0.2 labels

https://github.com/torproject/sbws/pull/321

Trac:
Status: assigned to needs_review

Trac:
Reviewer: N/A to dgoulet

I think we should make 36 value into some sort of constant because it isn't clear what is that value but also it is repeated twice.

I had to read the loop above to get that it is every 5 seconds and thus 36 is for 3 minutes. Either adding a clearer comment linking the two or making it a constant and comment there.

The other thing is that, don't you want to exit after the bug is detected or it will recover from the mistake?

Feel free to merge after that :).

Trac:
Status: needs_review to needs_revision

Replying to dgoulet:

I think we should make 36 value into some sort of constant because it isn't clear what is that value but also it is repeated twice.

I had to read the loop above to get that it is every 5 seconds and thus 36 is for 3 minutes. Either adding a clearer comment linking the two or making it a constant and comment there.

Added a constant.

The other thing is that, don't you want to exit after the bug is detected or it will recover from the mistake?

No, the error doesn't recover. I changed the code so that it exits unless the debugging level is log (more verbose) so that it continues instanciating a Python Debugger (for developers to debug the error).

Trac:
Status: needs_revision to merge_ready

Merged

Trac:
Status: merge_ready to closed
Resolution: N/A to fixed

closed

moved to tpo/network-health/sbws#28932 (closed)