BridgeDB stuck at 500 bridges

added component::circumvention/bridgedb owner::mikeperry priority::medium resolution::not a bug status::closed type::defect labels

That job I linked to is some weird relay job. Relay jobs on mturk are a bad idea because people don't know the risks of what they are getting into. We should make a bridge-specific job for this.

Apart of the idea to run an additional 100 bridges, see #2053 (moved) for a somewhat closer investigation of the problem.

Replying to mikeperry: Provided mikeperry with one week of Bridge Authority info-level log files upon request.

Ok, I ran this script over the logs:

for i in tonga_logs_20101*; do echo ; echo $i echo -n "New:" grep dirserv_add_descriptor $i | grep -c accepted echo -n "Updated:" grep dirserv_add_descriptor $i | grep updated | awk '{ print $11; }' | sort | uniq | wc -l done

This was the output:

tonga_logs_20101025 New:199 Updated:650

tonga_logs_20101026 New:136 Updated:631

tonga_logs_20101027 New:112 Updated:593

tonga_logs_20101028 New:101 Updated:643

tonga_logs_20101029 New:135 Updated:614

tonga_logs_20101030 New:122 Updated:613

tonga_logs_20101101 New:123 Updated:644

So it appears we are seeing about 250 more descriptors in a 24 hour period than tonga is publishing, give or take 20 or so?

Here is an updated script based on input from Roger:

for i in tonga_logs_20101*; do echo ; echo $i new=grep dirserv_add_descriptor $i | grep -c accepted echo "New: $new" updated=grep dirserv_add_descriptor $i | grep updated | awk '{ print $11; }' | sort | uniq | wc -l echo "Updated: $updated" unreachable=grep -c rep_hist_note_router_unreachable $i unreachable=echo "$unreachable/(3600*24/128/10)" | bc echo "Unreachable:

unreachable" running=`echo "

new + $updated - $unreachable" | bc` echo "Running: $running" done

It produces output more in line with what tonga is reporting. For example:

tonga_logs_20101028 New: 101 Updated: 643 Unreachable: 291 Running: 453

Trac:
Status: new to accepted
Owner: N/A to mikeperry

Trac:
bridgedb_scraper.sh

Attaching the script as a proper attachment to avoid filtering...

The question that still remains is might tonga be hitting some other issue that is preventing it from testing more than a certain number of bridges successfully. Supposedly it tests 1/256 of the keyspace every 10 seconds, which means a connection every 3 seconds with the current bridge pool.

From the logs now, it looks like we're not hitting file descriptor limits, but it seems like eventually we might. We could also be hitting some other limit on the box that is causing it to fail reachability testing for half of these bridges.

Err, two corrections. It's apparently 1/128 of the keyspace, and I believe all the connection attempts happen immediately in parallel, and are not spread out serially. So we've got a burst of 6-7 tcp connect attempts on tonga every 10 seconds...

I think this might mean the stat we should look at next is what is the failure rate for each round of these reachability tests.. If it is always 2 or 3/6, then we might be on to something.

Hrmm. rep_hist_note_router_unreachable() is called only 2x an hour.. I guess this is because it's only called during NS/vote generation.. bleh. Info logs might not have what we need for this.

This graph implies that there's no artificial limit at 500 bridges. See #2053 (moved) for the metrics part of this question.

Trac:
Points: N/A to N/A

Sounds like this one is closeable too.

closing as notabug; reopen if needed.

Trac:
Pointsdone: N/A to N/A
Resolution: N/A to not a bug
Status: accepted to closed
Actualpointsdone: N/A to N/A
Actualpoints: N/A to N/A

closed

moved to tpo/anti-censorship/bridgedb#2071 (closed)

BridgeDB stuck at 500 bridges

Child items ...

Activity