Introduction to Datagram Transport Layer Security (DTLS)

DTLS is a UDP-based protocol that serves the application layer. TLS protocol cannot guarantee the security of data transmitted by UDP. Therefore, the DTLS protocol has been extended on the existing TLS protocol architecture to support UDP, and becomes a version of TLS protocol that supports data packet transmission. DTLS 1.0 is based on TLS 1.1, and DTLS 1.2 is based on TLS 1.2. The encryption algorithm, certificate, and encryption process of the DTLS protocol are basically the same as those of the TLS protocol, thus will not be described in this section.

1. Differences Between DTLS and TLS

The working principle of the DTLS protocol is basically the same as that of the TLS protocol, except for the following differences:

  • In the handshake stage, DTLS protocol has added the Cookie mechanism. The DTLS protocol has added a Cookie mechanism in version 1.0, which is used by the server to verify the client, and can avoid DoS attacks. When the client sends the Client Hello message to the server, the server does not directly reply to the Server Hello message to carry out the handshake process. Instead, the server replies the Hello Verify Request message, which carries the Cookie value, to the client. When the client receives the message, it will write the Cookie value into the Client Hello message and resend it to the server. After receiving it, the server checks the local Cookie list to determine whether a handshake is required.

  • DLTS supports the retransmission mechanism. Since the UDP protocol itself does not support retransmission like the TCP protocol, the DTLS protocol introduces a retransmission mechanism. Taking the above Client Hello message as an example, after the client sends the Client Hello message, the client will start a timer to receive the Hello Verify Request message replied by the server; if the server does not reply within a certain period of time, the client will resend the Client Hello message. Similarly, once a message is sent, the server will activate a timer to monitor for timeouts and determine if the message needs to be resent.

  • DLTS supports orderly reception. UDP does not guarantee the order of delivered packets. In contrast, DTLS protocol has added a message_seq field in the handshake message. The receiver will provide a receiving buffer to receive out-of-order messages (similarly to TCP), and process the messages in order according to the message_seq field.

  • DLTS supports packet size limitation. UDP is a packet-oriented protocol, and TCP is a stream-oriented protocol. TCP supports packet fragmentation and reassembly. However, when a UDP message exceeds the maximum transmission unit (MTU) of the link layer, it may be forcibly fragmented at the IP layer. The receiver then needs to process the fragmented packet based on the IP header and reassemble the original data. If one packet is lost, the entire UDP message will be invalid. Therefore, in DTLS protocol, the handshake messages are segmented on top of UDP. This is done by adding the fragment_offset field and fragment_length field to the handshake message, which represent the offset of this message relative to the beginning of the message and the length of this message, respectively.

2. Creating a CoAP+DTLS server with ESP-IDF

The following example introduces how to create a CoAP+DTLS server. This example is actually the same as the CoAP example introduced in Section 8.3.4, except that two functions are added to support the DTLS protocol. The function coap_context_set_psk() is used to set the PSK encryption key in the DTLS protocol, and can also use certificate (PKI) for the DTLS protocol handshake. The coap_new_endpoint(ctx, &serv_addr, COAP_PROTO_DTLS) function indicates that the node supports the DTLS protocol.

📝 Source code

For the complete example code of coap_context_set_psk(), please refer to book-esp32c3-iot-projects/test_case/coap. For instructions on how to use PKI, please see esp-idf/examples/protocols/coap_server.

static char psk_key[] = "esp32c3_key";
static void esp_create_coaps_server(void)
    coap_context_t *ctx = NULL;
    coap_address_t serv_addr;
    coap_resource_t *resource = NULL;
    while (1) {
        coap_endpoint_t *ep = NULL;
        unsigned wait_ms;

        //Create a CoAP server socket
        serv_addr.addr.sin6.sin6_family = AF_INET6;
        serv_addr.addr.sin6.sin6_port = htons(COAP_DEFAULT_PORT);

        //Create CoAP ctx
        ctx = coap_new_context(NULL);
        if (!ctx) {
            ESP_LOGE(TAG, "coap_new_context() failed");

        //Add PSK encryption key
        coap_context_set_psk(ctx, "CoAP",
                            (const uint8_t *)psk_key,
                            sizeof(psk_key) - 1);

        //Set CoAP node
        ep = coap_new_endpoint(ctx, &serv_addr, COAP_PROTO_UDP);
        if (!ep) {
            ESP_LOGE(TAG, "udp: coap_new_endpoint() failed");
            goto clean_up;

        //Add DTLS node and port
        if (coap_dtls_is_supported()) {
            serv_addr.addr.sin6.sin6_port = htons(COAPS_DEFAULT_PORT);
            ep = coap_new_endpoint(ctx, &serv_addr, COAP_PROTO_DTLS);
            if (!ep) {
                ESP_LOGE(TAG, "dtls: coap_new_endpoint() failed");
                goto clean_up;
            } else {
                ESP_LOGI(TAG, "MbedTLS (D)TLS Server Mode not configured");

        //Set CoAP resource URI
        resource = coap_resource_init(coap_make_str_const("light"), 0);
        if (!resource) {
            ESP_LOGE(TAG, "coap_resource_init() failed");
            goto clean_up;

        //Register callback functions for GET and PUT method corresponding to CoAP resource URI
        coap_register_handler(resource, COAP_REQUEST_GET, esp_coap_get);
        coap_register_handler(resource, COAP_REQUEST_PUT, esp_coap_put);
        //Set CoAP GET resource visible
        coap_resource_set_get_observable(resource, 1);

        //Add resource to CoAP ctx
        coap_add_resource(ctx, resource);
        wait_ms = COAP_RESOURCE_CHECK_TIME * 1000;
        while (1) {
            //Wait to receive CoAP data
            int result = coap_run_once(ctx, wait_ms);
            if (result < 0) {
            } else if (result && (unsigned)result < wait_ms) {
                //Decrease waiting time
                wait_ms -= result;
            } else {
                //Reset waiting time
                wait_ms = COAP_RESOURCE_CHECK_TIME * 1000;