obfsproxy assertion failure src/container.c:66: size <= MAX_CAPACITY

After we were beating on it for a while, it exited in this surprising way.

added component::archived/obfsproxy owner::asn priority::medium resolution::fixed status::closed type::defect labels

Trac:
Owner: N/A to asn
Component: - Select a component to Obfsproxy

In my case sizeof size_t is 8, and sizeof int is 4, so

#define MAX_CAPACITY (INT_MAX)

Not that it should matter how big MAX_CAPACITY is. Seems like this was an underflow.

We are using two smartlists in the networking code. A smartlist for listeners, which has a static size till the halt of obfsproxy, and a smartlist for connections.

My guess would be that we do an extra `smartlist_remove()` in the connection smartlist, the smartlist counter becomes negative, and then we try to add another connection and we notice it in `smartlist_ensure_capacity()`. The circuit concept makes connection removal kinda groovy, and we should look at how `conn_free()` does it carefully.

Got a stack trace there? (bt full, ideally)

Also, how would you get an underflow there? If size is negative, then the size > sl->capacity check should be false, unless sl->capacity is (ug) negative too.

And come to think of it, size is an int. If MAX_CAPACITY is INT_MAX, how can size <= MAX_CAPACITY ever be false?

Replying to nickm:

Got a stack trace there? (bt full, ideally)

Nobody will have any stack traces until #5075 (closed) gets merged.

Also, how would you get an underflow there? If size is negative, then the size > sl->capacity check should be false, unless sl->capacity is (ug) negative too.

And come to think of it, size is an int. If MAX_CAPACITY is INT_MAX, how can size <= MAX_CAPACITY ever be false?

A fine question.

It just happened on my client, when I had configured it to use 25 bridges.

#0  0x00007f50e92f1475 in *__GI_raise (sig=<optimized out>)
    at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
        pid = <optimized out>
        selftid = <optimized out>
#1  0x00007f50e92f46f0 in *__GI_abort () at abort.c:92
        act = {__sigaction_handler = {sa_handler = 0x2053f10, 
            sa_sigaction = 0x2053f10}, sa_mask = {__val = {1, 2379064, 
              0, 33865376, 18, 139985486109600, 139985498210304, 1, 
              4294967295, 0, 1, 6357936, 0, 5353, 18, 0}}, 
          sa_flags = -372902078, sa_restorer = 0x1}
        sigs = {__val = {32, 0 <repeats 15 times>}}
#2  0x0000000000406649 in log_error_abort (format=<optimized out>)
    at src/util.c:580
        ap = {{gp_offset = 8, fp_offset = 48, 
            overflow_arg_area = 0x7fff94780c70, 
            reg_save_area = 0x7fff94780bb0}}
#3  0x0000000000402218 in smartlist_ensure_capacity (
    size=<optimized out>, sl=<optimized out>) at src/container.c:66
No locals.
#4  0x0000000000407eaa in smartlist_ensure_capacity (
    size=<optimized out>, sl=0x204ce20) at src/container.c:65
        higher = <optimized out>
#5  smartlist_add (sl=0x204ce20, element=0x204c000) at src/container.c:81
No locals.
#6  0x00000000004041da in open_outbound_hostname (port=5353, 
    addr=0x204d79c "x.x.x.x", af=2, conn=0x204cd90)
    at src/network.c:615
        base = <optimized out>
        buf = 0x2053f00
        newconn = 0x204c000
#7  socks_read_cb (bev=<optimized out>, arg=0x204cd90)
    at src/network.c:647
        af = 2
        r = <optimized out>
        port = 5353
        addr = 0x204d79c "x.x.x.x"
        status = <optimized out>
        conn = 0x204cd90
        socks_ret = <optimized out>
        __func__ = "socks_read_cb"
#8  0x00007f50e9a26e1d in ?? () from /usr/lib/libevent-2.0.so.5
No symbol table info available.
#9  0x00007f50e9a1c97c in event_base_loop ()
   from /usr/lib/libevent-2.0.so.5
No symbol table info available.
#10 0x0000000000403084 in launch_managed_proxy () at src/managed.c:685
        r = -1
        proxy = 0x203a010
#11 0x00000000004026e8 in obfs_main (argc=<optimized out>, 
    argv=<optimized out>) at src/main.c:265
        begin = 0x7fff94780f78
#12 0x00007f50e92ddead in __libc_start_main (main=<optimized out>, 
    argc=<optimized out>, ubp_av=<optimized out>, init=<optimized out>, 
    fini=<optimized out>, rtld_fini=<optimized out>, 
    stack_end=0x7fff94780f58) at libc-start.c:228
        result = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {0, 
                -9055663285749715144, 4203048, 140735684284256, 0, 0, 
                9055888689421840184, 9003259033683455800}, 
              mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x40b550, 
        r = -1
        proxy = 0x203a010
#11 0x00000000004026e8 in obfs_main (argc=<optimized out>, 
    argv=<optimized out>) at src/main.c:265
        begin = 0x7fff94780f78
#12 0x00007f50e92ddead in __libc_start_main (main=<optimized out>, 
    argc=<optimized out>, ubp_av=<optimized out>, init=<optimized out>, 
    fini=<optimized out>, rtld_fini=<optimized out>, 
    stack_end=0x7fff94780f58) at libc-start.c:228
        result = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {0, 
                -9055663285749715144, 4203048, 140735684284256, 0, 0, 
                9055888689421840184, 9003259033683455800}, 
              mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x40b550, 
              0x7fff94780f68}, data = {prev = 0x0, cleanup = 0x0, 
              canceltype = 4240720}}}
        not_first_call = <optimized out>
#13 0x0000000000402251 in _start ()
No symbol table info available.

Built it with -O0 and tried again. I can reproduce this each time.

And isn't that odd:

#3  0x000000000040a358 in smartlist_ensure_capacity (sl=0x1d24e20, 
    size=17) at src/container.c:66
66            obfs_assert(size <= MAX_CAPACITY);

17 doesn't seem that big. But it's suspiciously bigger than 16, which is what smartlists start at:

/** All newly allocated smartlists have this capacity. */
#define SMARTLIST_DEFAULT_CAPACITY 16

I bet your smartlists have never been able to grow.

Grr, snarl. See 5f4b127a97d38556f7b60bdffc490872f054d8dc for the fix and the explanation. Not a fun bug. The next person who wants to redefine autoconf internal functions needs to post bond, forfeitable on the first bug caused thereby.

Trac:
Resolution: N/A to fixed
Status: new to closed

closed