Opened 7 years ago

Closed 6 years ago

#8157 closed defect (worksforme)

'test_skip_nondescriptor_contents' and 'test_non_ascii_descriptor' fails

Reported by: Abhishek Kumar Singh Owned by: atagar
Priority: Medium Milestone:
Component: Core Tor/Stem Version:
Severity: Keywords: testing
Cc: Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

On running ./run_tests.py --integ --all 'test_skip_nondescriptor_contents' and 'test_non_ascii_descriptor' fails. Actually test_skip_nondescriptor_contents fails, while test_non_ascii_descriptor shows error which means there is error in writing this test.

[[Image(======================================================================
ERROR: test_non_ascii_descriptor


Traceback:

File "/home/abhishek/checktor/stem/test/integ/descriptor/server_descriptor.py", line 197, in test_non_ascii_descriptor

desc = next(stem.descriptor.parse_file(descriptor_file, "server-descriptor 1.0"))

File "/home/abhishek/checktor/stem/stem/descriptor/init.py", line 186, in parse_file

for desc in file_parser(descriptor_file):

File "/home/abhishek/checktor/stem/stem/descriptor/init.py", line 207, in _parse_metrics_file

for desc in stem.descriptor.server_descriptor._parse_file(descriptor_file, is_bridge = False, validate = validate):

File "/home/abhishek/checktor/stem/stem/descriptor/server_descriptor.py", line 134, in _parse_file

yield RelayDescriptor(descriptor_text, validate, annotations)

File "/home/abhishek/checktor/stem/stem/descriptor/server_descriptor.py", line 641, in init

self._validate_content()

File "/home/abhishek/checktor/stem/stem/descriptor/server_descriptor.py", line 692, in _validate_content

self._verify_descriptor(key_as_bytes)

File "/home/abhishek/checktor/stem/stem/descriptor/server_descriptor.py", line 758, in _verify_descriptor

local_digest = self.digest().lower().decode('hex')

File "/home/abhishek/checktor/stem/stem/descriptor/server_descriptor.py", line 656, in digest

raw_descriptor = str(self)

UnicodeEncodeError: 'ascii' codec can't encode character u'\xe5' in position 925: ordinal not in range(128)


)]]

Also please check output file attached with this ticket.

Child Tickets

Attachments (3)

output (52.5 KB) - added by Abhishek Kumar Singh 7 years ago.
stem_op.txt (21.4 KB) - added by gsathya 7 years ago.
integ test o/p
0001-Fix-AssertionError-in-test_skip_nondescriptor_conten.patch (2.0 KB) - added by gsathya 7 years ago.
not sure if this is the approach you want, but this patch fixes test_skip_nondescriptor_contents

Download all attachments as: .zip

Change History (16)

Changed 7 years ago by Abhishek Kumar Singh

Attachment: output added

comment:1 Changed 7 years ago by Abhishek Kumar Singh

Component: - Select a componentStem
Owner: set to atagar

comment:2 Changed 7 years ago by atagar

Status: newneeds_information

Hi Abhishek. Thanks for the catches! I just checked and couldn't repro any of these, so you'll need to take the lead with troubleshooting.

======================================================================
FAIL: test_get_stats
----------------------------------------------------------------------
Traceback:
  File "/home/abhishek/checktor/stem/test/integ/util/proc.py", line 75, in test_get_stats
    self.assertTrue(float(stime) > 0)
AssertionError: False is not true

----------------------------------------------------------------------
Ran 5 tests in 0.008s

This is certainly odd. What distro are you running? If you print stime then what is it?

Expected:
  riddle,
  tiny.png,
  vote,
  new_metrics_type

Result:
  /home/abhishek/checktor/stem/test/integ/descriptor/data/riddle (Unrecognized mime type: None),
  /home/abhishek/checktor/stem/test/integ/descriptor/data/new_metrics_type (Unrecognized mime type: None),
  /home/abhishek/checktor/stem/test/integ/descriptor/data/tiny.png (Unrecognized mime type: image/png),
  /home/abhishek/checktor/stem/test/integ/descriptor/data/non-ascii_descriptor),
  /home/abhishek/checktor/stem/test/integ/descriptor/data/vote (Unrecognized mime type: None)

This and the following test_non_ascii_descriptor failure are for the same thing. What version of python are you using? I'm a little surprised that I'm unable to repro this. The ending of the stacktrace...

  File "/home/abhishek/checktor/stem/stem/descriptor/server_descriptor.py", line 656, in digest
    raw_descriptor = str(self)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe5' in position 925: ordinal not in range(128)

... is a little odd because RelayDescriptor extends ServerDescriptor, which extends Descriptor. The Descriptor class has a str method which should simply be returning the unicode content we were constructed through without trying to decode anything.

Cheers! -Damian

comment:3 in reply to:  2 Changed 7 years ago by Abhishek Kumar Singh

Hi Damian,

What distro are you running?

I am using Ubuntu 12.10 and python 2.7.3

I tested with the old branch of stem which I had downloaded earlier and found no failures with that. But these results(failures) I got from testing on the new branch of stem which I had downloaded recently(around three or four days ago).
I guess some of the files have undergone some changes recently. File /stem/test/integ/descriptor/server_descriptor.py differs in old and new branches because of some recent changes.

Changed 7 years ago by gsathya

Attachment: stem_op.txt added

integ test o/p

comment:4 Changed 7 years ago by gsathya

FWIW, I got a bunch of similar stacktraces with the latest master(e8784466401a8120669128682b75eaa947e91c7a)

comment:5 Changed 7 years ago by Abhishek Kumar Singh

Hi Damian,

What does _raw_contents attribute of 'Descriptor' class suppose to contain? Can it contain either of unicode object or string ?

comment:6 Changed 7 years ago by atagar

The _raw_contents is the unicode string that was used to construct the Descriptor (ie, the contents of the cached consensus or whatever was read to make it).

Changed 7 years ago by gsathya

not sure if this is the approach you want, but this patch fixes test_skip_nondescriptor_contents

comment:7 Changed 7 years ago by gsathya

Status: needs_informationneeds_review

comment:8 Changed 7 years ago by atagar

Status: needs_reviewneeds_revision

Hi Sathyanarayanan. No, disabling validation is just masking whatever the actual failure is. We (where 'we' is someone that can repro this :P) should dig into *why* it's failing at that point.

comment:9 in reply to:  8 Changed 7 years ago by gsathya

Replying to atagar:

Hi Sathyanarayanan. No, disabling validation is just masking whatever the actual failure is. We (where 'we' is someone that can repro this :P) should dig into *why* it's failing at that point.

I agree. It's because "non-ascii_descriptor" gets skipped. Should stem not skip it?

comment:10 Changed 7 years ago by atagar

Should stem not skip it?

No, it shouldn't. The descriptor validator is claiming that non-ascii_descriptor is malformed which is incorrect.

comment:11 Changed 6 years ago by atagar

The descriptor handling and tests have undergone several changes. Is this still occurring?

comment:12 in reply to:  11 Changed 6 years ago by gsathya

Replying to atagar:

The descriptor handling and tests have undergone several changes. Is this still occurring?

No.

comment:13 Changed 6 years ago by atagar

Resolution: worksforme
Status: needs_revisionclosed

Great, resolving.

Note: See TracTickets for help on using tickets.