Bdtjtk

I am writing a native DTLS module for NodeJS using OpenSSL. It uses memory BIOs so node's own sockets can be used to control the flow of data. Everything seems to be working but I am running into some problems with the DOS mitigation.

According to the spec, the initial ClientHello sent to the server should be rejected and the server will send a HelloVerifyRequest containing a cookie to be resent back from the client. This all works fine but when the client sends back the second ClientHello, for some reason the DTLSv1_listen() call is causing my cookie generation method to fire a second time instead of the cookie verification method. Strangely enough, if I send back the second HelloVerifyRequest (exact same length and content as the first) I end up with a ClientHello that seems to trigger the verification method.

Here's a small test I've written to illustrate the kind of thing I'm doing (not exactly, skipped some stuff like importing the cert/key, result code checking for read/writes after calling handshake, freeing memory etc).

TEST(New, Test) {

    // Init context

    auto ctx = SSL_CTX_new(DTLS_method());

    SSL_CTX_set_cipher_list(ctx, "ALL:!ADH:!LOW:!EXP:!MD5:@STRENGTH");

    SSL_CTX_set_verify(ctx, SSL_VERIFY_NONE, (int ok, X509_STORE_CTX * context) { return 1; });

    SSL_CTX_set_cookie_generate_cb(ctx, (SSL * ssl, unsigned char * cookie, unsigned int * cookie_len) { 

        return 1; 

        });

    SSL_CTX_set_cookie_verify_cb(ctx, (SSL * ssl, const unsigned char * cookie, unsigned int cookie_len) { 

        return 1; 

        });



    // Init connections

    auto client = SSL_new(ctx);

    auto client_rbio = BIO_new(BIO_s_mem());

    auto client_wbio = BIO_new(BIO_s_mem());

    SSL_set_bio(client, client_rbio, client_wbio);

    SSL_set_connect_state(client);



    auto server = SSL_new(ctx);

    auto server_rbio = BIO_new(BIO_s_mem());

    auto server_wbio = BIO_new(BIO_s_mem());

    SSL_set_bio(server, server_rbio, server_wbio);

    SSL_set_accept_state(server);



    std::vector<unsigned char> data;



    // Client Hello, no cookie

    SSL_do_handshake(client);

    auto data_len = BIO_ctrl_pending(client_wbio);

    data.resize(data_len);

    BIO_read(client_wbio, data.data(), data.size());



    ASSERT_EQ(data[13], 1);



    // Hello Verify Request

    BIO_write(server_rbio, data.data(), data.size());

    DTLSv1_listen(server, NULL);

    data_len = BIO_ctrl_pending(server_wbio);

    data.resize(data_len);

    BIO_read(server_wbio, data.data(), data.size());



    ASSERT_EQ(data[13], 3);



    // Client Hello, with cookie

    BIO_write(client_rbio, data.data(), data.size());

    SSL_do_handshake(client);

    data_len = BIO_ctrl_pending(client_wbio);

    data.resize(data_len);

    BIO_read(client_wbio, data.data(), data.size());



    ASSERT_EQ(data[13], 1);



    // Should be pass...?

    BIO_write(server_rbio, data.data(), data.size());

    ASSERT_EQ(DTLSv1_listen(server, NULL), 1);

}

The last assert fails -- it is -1 in this example, 0 in my actual code (and subsequent BIO_read gets me data[13]=3 aka HelloVerifyRequest) but the important thing to note here is that if you attach a debugger and put a breakpoint on the verification lambda it will not be hit.

You could try using nodejs-dtls

Sample usage can be found in
IOTBroker. cloud nodejs client

Take a look at CoAP and MQTT-SN protocols

BR

Yulian Oifa

Mobius Software