Opened 3 months ago

Closed 3 months ago

Last modified 2 months ago

#28902 closed defect (fixed)

GETINFO commands with huge outputs slow down interpreter curses interface

Reported by: wagon Owned by: atagar
Priority: Medium Milestone:
Component: Core Tor/Nyx Version:
Severity: Normal Keywords: curses
Cc: Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

If you go to Nyx interpreter, run some commands, then press ESC and arrow up, you see that scrolling back in history is very fast and smooth.

Now do the following:

  1. Run GETINFO desc/all-recent command 3 times to get very huge output.
  2. Press ESC and then press HOME to get at the top of scroll buffer.
  3. Press arrow up and arrow down keys many times to scroll up/down.
  4. Press ENTER to return to command interface.
  5. Run some command with small output, e.g. GETINFO info/names.
  6. Press ESC and try to scroll few lines up by pressing arrow up many times.

You will see that scrolling is very slow. You need few seconds to scroll just few lines up in the buffer.

Nyx version is this one. The ticket is filed by atagar's request:

I suspect your initial hypothesis about the reason Nyx is freezing is inaccurate. Feel free to file a separate ticket with the nyx --debug output when Nyx freezes so I can see what's up.

Child Tickets

Change History (10)

comment:1 Changed 3 months ago by wagon

Now I cannot reproduce the slowing down of all Nyx windows (this is what I wrote in #28877), as I see problems only in interpreter Nyx window. If I do desc/all-recent test on a system which has about 1GB of RAM (I don't think it is too small, because I don't run browsers or any heavy applications in this system), it is still not sufficient for Nyx desc/all-recent command. System hangs for few minutes until OS finds a way to stop Nyx.

UPDATE. During experimentation with Nyx I found that the file $HOME/.nyx/cache.sqlite was created. It contained information about Tor relays. If these cached relays are only those replays which were used in Tor circuits, it is a privacy leak.

Last edited 3 months ago by wagon (previous) (diff)

comment:2 Changed 3 months ago by atagar

Ah! So that ticket was about the control interpreter. That makes a lot more sense (I thought you were talking about an internal 'GETINFO desc/all-recent' invocation).

If you're running that command three times the control interpreter has accumulated roughly 1080078 lines of content. The draw method then iterates over that which is probably the source of the problem...

https://gitweb.torproject.org/nyx.git/tree/nyx/panel/interpreter.py#n133

This should be easy to fix. Rather than looping over the whole list we should splice to the visible range.

Honestly I won't be getting to this any time soon, but I'd be happy to merge the fix if you'd care to whip it up. Otherwise I'll get to it when I have time.

comment:3 Changed 3 months ago by wagon

Ah! So that ticket was about the control interpreter.

Surely. I thought it was always clear:

If UseMicrodescriptors is set to 0 in Tor client, the command GETINFO desc/all-recent returns very huge listing which interpreter cannot manage properly. If amount of memory is big, after finishing this command Nyx starts to response slowly on any keyboard input everywhere and in all its windows, not only in interpreter window

I'd be happy to merge the fix if you'd care to whip it up. Otherwise I'll get to it when I have time.

Probably you will find time faster than I will find time to learn python and its libraries...

comment:4 Changed 3 months ago by atagar

which interpreter cannot manage

I thought you meant the python interpreter rather than the control interpreter. Both readings are accurate, but different problems.

comment:5 Changed 3 months ago by wagon

I thought you meant the python interpreter rather than the control interpreter.

It is a good opportunity to ask you why there is no keyword "interperter" in a list of keywords suggested for Nyx bugs. It would be logical to have it. Now we have only these ones:

header - top part of the interface showing the nickname, cpu usage, etc
graph - related to the bandwidth graph
log - related to the log panel
connections - related to the connection panel
config - related to the configuration editor
torrc - related to the torrc panel
menu - drop-down menus
curses - low level terminal rendering

comment:6 Changed 3 months ago by atagar

why there is no keyword "interperter"

Simply because it rarely gets tickets. I made those keywords to group similar tickets but the interpreter gets so little use (and by extension tickets) it didn't merit one. Personally I don't care much what our trac keywords are so feel free to add it to the list if you'd like.

comment:7 Changed 3 months ago by atagar

Resolution: fixed
Status: assignedclosed

Hi wagon, sorry about the long delay. Fix pushed. Feel free to reopen if this doesn't do the trick for ya.

https://gitweb.torproject.org/nyx.git/commit/?id=6492a3a

comment:8 Changed 2 months ago by wagon

Thanks, atagar! I would like to wait for your signature for the last git commit or the next signed release (what will come first).

comment:9 Changed 2 months ago by atagar

Hi wagon. I don't intend to continue to sign individual commits. That said, there will likely be a Nyx 2.1.0 release in a month or two.

comment:10 Changed 2 months ago by wagon

OK, good.

Note: See TracTickets for help on using tickets.