Opened 6 weeks ago

Closed 5 weeks ago

Last modified 5 weeks ago

#33374 closed defect (fixed)

Fix unicode warnings in practracker using python 2

Reported by: teor Owned by: teor
Priority: Medium Milestone: Tor: 0.4.3.x-final
Component: Core Tor/Tor Version: Tor: 0.4.3.1-alpha
Severity: Normal Keywords: 043-backport, consider-backport-immediately
Cc: Actual Points: 0.1
Parent ID: Points: 0.1
Reviewer: ahf Sponsor:

Description

When I run practracker using python 2, I get the following unicode warning:

$ scripts/maint/practracker/practracker.py --regen-overbroad
Traceback (most recent call last):
  File "scripts/maint/practracker/practracker.py", line 324, in <module>
    main(sys.argv)
  File "scripts/maint/practracker/practracker.py", line 268, in main
    for item in filt.filter(consider_all_metrics(files_list)):
  File "/Users/hyper/dev/tor/scripts/maint/practracker/problem.py", line 147, in filter
    for item in iter(sequence):
  File "scripts/maint/practracker/practracker.py", line 110, in consider_all_metrics
    for item in consider_metrics_for_file(fname, f):
  File "scripts/maint/practracker/practracker.py", line 134, in consider_metrics_for_file
    for item in consider_function_size(fname, f):
  File "scripts/maint/practracker/practracker.py", line 91, in consider_function_size
    for name, lines in metrics.get_function_lines(f):
  File "/Users/hyper/dev/tor/scripts/maint/practracker/metrics.py", line 58, in get_function_lines
    if line.startswith("}"):
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 22: ordinal not in range(128)
Exit 1

We can fix this issue by using the codecs module to open files in unicode mode.

Child Tickets

Change History (10)

comment:1 Changed 6 weeks ago by teor

See my PR:

It's a trivial fix, so I didn't do a master test PR or branch.

comment:2 Changed 6 weeks ago by teor

Status: assignedneeds_review

comment:3 Changed 6 weeks ago by teor

Summary: Fix unocide warnings in practracker using python 2Fix unicode warnings in practracker using python 2

comment:4 Changed 5 weeks ago by asn

Hm. Python2 is officially deprecated: https://pythonclock.org/

Do we still need to support it?

comment:5 Changed 5 weeks ago by teor

Python 2 is technically not EOL until 1 April 2020.

And yes, some systems still ship Python 2 as the default python. (And many installed systems still only have python 2.) It's an easy change, lets make it?

comment:6 Changed 5 weeks ago by teor

For example, our macOS CI has python 2 by default.
I think we have to specifically ask for python 3 at runtime.

Last edited 5 weeks ago by teor (previous) (diff)

comment:7 Changed 5 weeks ago by ahf

Reviewer: ahf

comment:8 Changed 5 weeks ago by ahf

Status: needs_reviewmerge_ready

Looks good. Didn't know of the codecs module, but it looks like it behaves similar on Python 3 and 2.

comment:9 Changed 5 weeks ago by nickm

Milestone: Tor: 0.4.4.x-finalTor: 0.4.3.x-final
Resolution: fixed
Status: merge_readyclosed

Merged to 0.4.3 and forward.

comment:10 in reply to:  8 Changed 5 weeks ago by teor

Replying to ahf:

Looks good. Didn't know of the codecs module, but it looks like it behaves similar on Python 3 and 2.

And it seems to have a very similar interface to python3's open().

Note: See TracTickets for help on using tickets.