Skip to content

dtls: large update to the dtls implementation - #65511

Closed
jasnell wants to merge 85 commits into
nodejs:mainfrom
jasnell:jasnell/dtls-improvements
Closed

jasnell wants to merge 85 commits into
nodejs:mainfrom
jasnell:jasnell/dtls-improvements

Conversation

@jasnell

@jasnell jasnell commented Aug 23, 2026

Copy link
Copy Markdown
Member

node:dtlslanded with the transport working but with many gaps. This addresses those, and fills in the API surface.

This is a large PR but the commits are structured logically and sequentially. I chose to keep multiple PRs rather than squashing due to the size. Each has it's own description. I recommend stepping through and reviewing commit-by-commit.

A separate review guide comment will be included.

@jasnell
jasnell requested a review from mcollina August 23, 2026 21:51
@jasnell jasnell added net Issues and PRs related to the net subsystem. experimental Issues and PRs related to experimental features. large-pr PRs subject to the large-PR policy. dtls Issues and PRs related to Datagram Transport Layer Security (DTLS). labels Aug 23, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Review requested:

  • @nodejs/net

@nodejs-github-bot nodejs-github-bot added lib / src Issues and PRs involving general changes in the lib/ or src/ directories. needs-ci PRs that need a full CI run. labels Aug 23, 2026
@jasnell

jasnell commented Aug 23, 2026

Copy link
Copy Markdown
Member Author

Review guide

Most of the following was AI agent generated, verified by me.

80 commits is a lot to read end to end, so here is a route through them. The commits are ordered by dependency, not by theme, so the groups below jump around the history; each commit appears in exactly one group.

Every commit builds and passes the suite on its own, so anything here can be checked out and run in isolation.

Numbers are positions in the branch, oldest first.

Datagram framing and the BIO layer

OpenSSL's DTLS record layer assumes one BIO read yields exactly one datagram. The module used byte-stream BIOs, so that assumption held only by accident. Start here: several later commits depend on both BIOs being datagram BIOs.

  • 21836c0b034 3. dtls: preserve record boundaries on the outbound BIO
  • 1f5f0aed8fd 5. dtls: preserve datagram boundaries on the inbound BIO
  • 82c18b0f73b 4. dtls: drop empty datagrams before the accept path
  • 0d0092a249d 11. dtls: size the outbound datagram buffers to the datagram
  • b482852be5f 21. dtls: do not resend a datagram that was already sent
  • e95699e1c3b 29. dtls: drop truncated datagrams instead of ignoring the recv flags

Denial of service and resource bounds

Work an unauthenticated peer could make the server do, and limits on what an authenticated one can hold or retain.

  • 1e44ba3287f 9. dtls: screen datagrams before allocating a handshake for them
  • a4bd742ca99 10. dtls: bound the server session table
  • d773d1ee80f 15. dtls: stop retaining server sessions, and scope them to the server
  • 1a15956aa69 37. dtls: add handshake timeout

Peer address identity

The session table is keyed on the peer address, so what counts as the same peer matters. Note that 910a8cf changes shared code and 6f27438 reverts that part -- read them together; the net effect on node_sockaddr is additive only.

  • 910a8cf110f 13. src: pair SocketAddress::Hash with a matching equality
  • a055d55784b 14. dtls: derive the cookie from a canonical peer address
  • 6f2743890c7 54. src: do not change the address map QUIC uses

Certificate verification and peer identity

There was no way to see why a handshake was rejected, and two paths where verification silently did not happen.

  • bfa94e355bd 6. dtls: add session.authorized and session.authorizationError
  • 8dc044561c0 7. dtls: correct the requestCert/rejectUnauthorized matrix
  • 948ab90a20a 42. dtls: do not report verification results before the handshake
  • 2132fb9cf67 44. dtls: require servername to be a string
  • f171f95e838 32. dtls: add session.peerX509Certificate
  • 8473c64381c 24. dtls: check the BIO allocation in GetPeerCertificate

ALPN

Protocol list encoding and what happens when nothing is shared.

  • 3d387550c9c 16. dtls: validate ALPN protocol lists
  • f4883ee296e 17. dtls: fail the handshake when no ALPN protocol is shared

New features: secure contexts, SNI, PSK, resumption

The largest group and the bulk of the new API surface. Read in order -- the later commits fix interactions the earlier ones created.

  • 1eb3262e824 33. dtls: add createSecureContext and the secureContext option
  • e2514270022 34. dtls: add server-side SNI
  • e0437bd3c96 39. dtls: accept a callback for SNI
  • 6afaf7d2034 36. dtls: add pre-shared key support
  • 9d68f7ab0f1 35. dtls: add session resumption
  • 97ae7774269 31. dtls: support a passphrase for encrypted private keys
  • 9f41eafeb8a 52. dtls: make sni a property of the secure context
  • 0c0dbd4c11d 48. dtls: keep pre-shared keys working on a server that serves SNI
  • 062e0d8be30 49. dtls: hold SNI contexts weakly
  • 7b642efd102 50. dtls: clear a callback when one is not supplied
  • e01868deb74 51. dtls: check that a value is a context before unwrapping it
  • 853c48adc9d 55. dtls: remove a callback that cannot be invoked

Exception safety and OpenSSL error reporting

Callbacks that run inside SSL_do_handshake() cannot report anything to JavaScript from where they stand, and OpenSSL's error queue is shared process-wide.

  • 57e0f80d8bc 2. dtls: drain the OpenSSL error queue
  • ce44c13ec79 58. dtls: read the error this operation queued, not the oldest one present
  • d8fbfee904b 27. dtls: stop aborting on failed value creation in the accessors
  • 8e2d2fb2ee5 28. dtls: stop aborting on failed value creation in the callback paths
  • 355ea7a9e0b 45. dtls: report exceptions thrown by the keylog callback
  • f48cc0b1af6 46. dtls: report exceptions thrown while reading a callback's result
  • c35fbec5f81 67. dtls: say why a psk callback's result was unusable
  • 483c4e87d3f 57. dtls: report a record that could not be sent
  • b607875cafe 30. dtls: report why loading a certificate or key failed
  • 5531b961b4b 63. dtls: report bind and listen failures with the operating system's code
  • af3892bc037 66. dtls: check the return values that were being discarded

Session and endpoint lifecycle

Promises that never settled, and ordering between a session reaching JavaScript and its handshake running.

  • 8308eb1ea0e 19. dtls: settle session.opened when the session is torn down
  • c004a477029 38. dtls: emit new sessions before driving the handshake
  • f9a0ea419ff 53. dtls: settle a session's promises when its endpoint is destroyed
  • 16146577f72 62. dtls: mark stats stale when their source goes away
  • e26f7a83198 56. dtls: state the session constructor's endpoint invariant once

Public surface and argument validation

Options that reached a CHECK in the binding (a caller typo aborting the process), and internals that were reachable as public API.

  • 078341290d6 41. dtls: stop exposing session and endpoint state
  • 68c5fc6a532 43. dtls: validate the options that reach a CHECK in the binding
  • bba2a1c53aa 59. dtls: make ownsEndpoint internal
  • 7947d87bb90 61. dtls: validate isServer and rejectUnauthorized as booleans
  • 100663a0fcd 68. dtls: accept any view over bytes in send()
  • 866c9999586 69. dtls: refuse server-only options on a client context
  • 3d18e9a98a1 70. dtls: use a type error for a session that is not a Buffer
  • fb714f03a0e 12. dtls: validate the exportKeyingMaterial length
  • 2eae15ff1fe 18. dtls: report why send() could not send
  • f742fafd500 47. dtls: register the two missing external references
  • 711927309cf 71. dtls: explain why the binding's closed-session guard stays

Sockets and addressing

Which local socket an endpoint binds, and the UDP options it exposes.

  • 09026d21c93 20. dtls: bind the local socket in the peer's address family
  • 6c9b4e47724 72. dtls: bind dual stack by default, and make ipv6Only an option
  • 8674e407aa0 73. dtls: add reusePort and the UDP buffer and TTL options
  • a7d66f1b46f 74. dtls: list the new socket options in the listen() and connect() jsdoc

Allocation gating

Two paths that built V8 values whether or not anything was listening.

  • 8ef12e57d29 8. dtls: only extract key material when something is listening
  • f1a67ba7e30 23. dtls: only copy incoming data when something is listening

Documentation

Corrections and additions. 7988ed8 is structural (heading levels only, anchors preserved); the rest are content.

  • 536ec083021 25. doc: record two deliberate dtls limits
  • 86f306f2b95 26. doc: describe the dtls cookie design and the internal-only members
  • 7988ed8df36 40. doc: lift the DTLS topic sections to their own level
  • d2f00579bcc 60. doc: fix dtls examples that throw, and a garbled option entry
  • 54b96872d88 64. doc: correct three dtls claims and file two sections under the right class
  • 088bf9f5b9d 65. doc: document the dtls members that were public but unlisted
  • 3e303b44566 75. doc: remove mention of C++ internals from dtls.md
  • f442a298a3f 77. doc: restore a dropped word in the endpoint.stats description
  • 6275168af6b 22. dtls: describe the OpenSSL options accurately

Housekeeping

Test fixes and mechanical cleanups.

  • cc9742518e4 1. test: fix dtls default CA test identity check
  • 06ba1a6e77c 76. test: fix the dtls permission test for the symbol-keyed bind()
  • d9c4d8739d2 78. src: replace NewFromUtf8 with ToV8Value
  • c3c5fef3990 79. src: fixup c++ linting after multiple commits
  • 3da12923f44 80. dtls: take the psk error message as a string_view

Worth a closer look

Behaviour changes that could affect an existing user of the experimental module:

  • f4883ee296e 17. — an ALPN mismatch now fails the handshake instead of connecting with no protocol agreed (RFC 7301 requires the alert)
  • 2eae15ff1fe 18. — send() throws where it used to return -1; callers testing send(x) < 0 are affected
  • 8dc044561c0 7. — { requestCert: true, rejectUnauthorized: false } now actually requests a certificate
  • 6c9b4e47724 72. — endpoints on :: are dual-stack by default, and IPv4 peers are reported as ::ffff: mapped addresses
  • 078341290d6 41. — session.state, endpoint.state and endpoint.sessions are no longer public
  • 5531b961b4b 63. — bind and listen failures throw the libuv code (EADDRINUSE) rather than ERR_INVALID_STATE

Security-relevant:

  • 948ab90a20a 42. — reading session.authorized before the handshake segfaulted
  • 2132fb9cf67 44. — a non-string servername silently skipped hostname verification
  • 9d68f7ab0f1 35. — resumption binds the session blob to the verified identity (the class of bug fixed in node:tls by CVE-2026-48934)
  • fb714f03a0e 12. — three ordinary-looking exportKeyingMaterial() arguments aborted the process
  • 68c5fc6a532 43. — several options reached a CHECK in the binding, so a caller typo aborted the process
  • 8ef12e57d29 8. — master secrets were copied into V8 strings on every handshake whether or not anything listened

Notes for the reviewer

  • Shared code. 910a8cf110f changed SocketAddress::Map, which QUIC uses via SocketAddressLRU. 6f2743890c7 reverts that and adds a separate PeerMap alias instead. The net change to src/node_sockaddr.{h,cc} is additive: a new Equal and a new alias, with the existing Map byte-for-byte unchanged.
  • Deliberately not done. DNS resolution in connect() (it would make the call asynchronous), DTLS Connection ID (not supported by the bundled OpenSSL 3.5.7), and the parsed certificate fields node:tls exposes beyond peerX509Certificate.
  • Coverage exclusions. The c8 ignore comments match the pattern node:quic uses; DTLS is behind a compile-time flag and is not built in the default CI configuration.

@codecov

This comment was marked as outdated.

Comment thread test/parallel/test-dtls-message-listener-gate.mjs
@jasnell
jasnell force-pushed the jasnell/dtls-improvements branch from 3da1292 to 272930e Compare August 25, 2026 02:09
@nodejs-github-bot

This comment was marked as outdated.

@jasnell

This comment was marked as outdated.

@jasnell

This comment was marked as resolved.

@jasnell
jasnell force-pushed the jasnell/dtls-improvements branch from 272930e to 34f252d Compare August 26, 2026 03:56
@nodejs-github-bot

This comment was marked as outdated.

@nodejs-github-bot

This comment was marked as outdated.

@jasnell
jasnell force-pushed the jasnell/dtls-improvements branch 2 times, most recently from a021dc5 to 92c6847 Compare August 29, 2026 17:14
@jasnell
jasnell requested a review from aduh95 August 30, 2026 04:22
@jasnell

jasnell commented Aug 30, 2026

Copy link
Copy Markdown
Member Author

@nodejs/net

@jasnell
jasnell force-pushed the jasnell/dtls-improvements branch from 92c6847 to 584f021 Compare September 5, 2026 03:20
@nodejs-github-bot

This comment was marked as outdated.

Comment thread lib/internal/dtls/dtls.js Outdated
Comment thread lib/internal/dtls/dtls.js Outdated
Comment thread lib/internal/dtls/dtls.js Outdated
Comment thread lib/internal/dtls/dtls.js Outdated
@jasnell
jasnell requested a review from mcollina September 5, 2026 19:42
@nodejs-github-bot

This comment was marked as outdated.

@nodejs-github-bot

This comment was marked as outdated.

@jasnell
jasnell force-pushed the jasnell/dtls-improvements branch from bc5fe2a to 22bb59d Compare September 6, 2026 21:03
@nodejs-github-bot

This comment was marked as outdated.

send() took a Buffer or a string and refused a Uint8Array, which is the
obvious thing to send, while exportKeyingMaterial() on the same object
accepted one. Bare ArrayBuffers stay refused, as they are there too.

The gate was Buffer.isBuffer() in JavaScript. The binding's check was
Buffer::HasInstance(), which is defined as IsArrayBufferView() and so had
been accepting every view all along. It is spelled IsArrayBufferView()
now, and reads the bytes through ArrayBufferViewContents, so what it
takes is stated rather than inherited from what a Buffer happens to be.

A view sends the bytes it covers and not the buffer behind it: a
subarray, a DataView at an offset, and an Int16Array all arrive as the
bytes they span.

Signed-off-by: James M Snell <jasnell@gmail.com>
Assisted-by: Opencode
Five options only a server can act on were handled four different ways
when a client named one: sni threw, sessionIdContext was ignored,
ticketKeys was applied to a client that has no tickets to issue, and
requestCert was validated and then ignored.

All refused now, by one rule checked before any of them is read. A client
naming one has misunderstood the option, and the difference between
"ignored" and "applied" was not something a caller could see. sni's own
check goes away in favour of the shared one.

pskIdentityHint names which key a client should pick. Given without psk
there was no key to name, so it was dropped and the handshake failed for
want of a PSK without mentioning the option that had been set.

Each option is still accepted by a server context, so the rule is about
which side may use it. ticketKeys and sni keep their own validation.

Signed-off-by: James M Snell <jasnell@gmail.com>
Assisted-by: Opencode
unwrapSession folded the Buffer check in with the prefix and length
checks, so all four failures reported ERR_INVALID_ARG_VALUE. Passing a
string got the code that means the type was right and the contents were
wrong.

Split out. A Buffer that is not one of ours still reports
ERR_INVALID_ARG_VALUE, which is what it is: the right type, contents that
cannot be resumed.

Signed-off-by: James M Snell <jasnell@gmail.com>
Assisted-by: Opencode
The binding says "Session is closed" where JavaScript says "Session is
destroyed" for what looks like the same situation. The first is
unreachable: JavaScript drops the handle on close and on destroy, and
send() refuses a null handle before the binding is reached. That holds
for a peer-initiated close too, where the close callback clears the
handle before control returns to user code.

The guard stays, because being unreachable today is not a reason to write
into a closed SSL if that changes. The comment records why its wording is
not being brought into line with a message it will never appear beside.

Signed-off-by: James M Snell <jasnell@gmail.com>
Assisted-by: Opencode
Bind() set UV_UDP_IPV6ONLY for every IPv6 address, unconditionally. An
endpoint on :: therefore served IPv6 only and an IPv4 peer could not
reach it, with nothing to say so and no way to ask for anything else:

  listen(..., { host: '::' })
  connect('127.0.0.1', port)   // handshake timeout

node:dgram and node:quic both bind dual stack by default. DTLS does now
too, and ipv6Only: true selects the old behaviour.

A dual-stack socket reports IPv4 peers with mapped addresses,
::ffff:127.0.0.1 rather than 127.0.0.1, so maxSessionsPerHost and
anything else keyed on the peer address sees them in that form.

The plumbing is a setSocketOptions() binding method read by Bind().

Signed-off-by: James M Snell <jasnell@gmail.com>
Assisted-by: Opencode
An endpoint took whatever socket the system gave it. There was no way to
spread a server over several processes, and no way to give it room for
bursts the default buffers drop.

reusePort sets SO_REUSEPORT, where the kernel spreads datagrams between
everyone bound to the port. Not SO_REUSEADDR, which libuv also offers and
node:dgram exposes: on Linux that lets the last binder take the port from
a running server. Without reusePort the port stays exclusive.

udpReceiveBufferSize, udpSendBufferSize and udpTTL are applied once the
bind succeeds, since there is no socket to set them on before that. Not
naming one leaves the system default rather than substituting a number of
ours.

Signed-off-by: James M Snell <jasnell@gmail.com>
Assisted-by: Opencode
Both blocks enumerate the options they take and neither mentioned the
five added for the UDP socket.

Signed-off-by: James M Snell <jasnell@gmail.com>
Assisted-by: Opencode
Mentioning C++ in the dtls.md doc exposes implementation detail

Signed-off-by: James M Snell <jasnell@gmail.com>
Assisted-by: Opencode
bind() moved to a symbol key so an endpoint cannot be rebound from
outside. test-permission-net-dtls.mjs still called endpoint.bind() and
had been failing with:

  TypeError: endpoint.bind is not a function

which assert.throws() reported as the wrong error rather than as a
missing method, so it read like a permission-check failure.

Signed-off-by: James M Snell <jasnell@gmail.com>
Assisted-by: Opencode
The entry read "live and updated data flows through the endpoint". The
session equivalent reads "updated as data flows".

Signed-off-by: James M Snell <jasnell@gmail.com>
Assisted-by: Opencode
Signed-off-by: James M Snell <jasnell@gmail.com>
Signed-off-by: James M Snell <jasnell@gmail.com>
ReportPSKError took a const char* and passed it to ToV8Value(), which
already has a std::string_view overload. Every call site hands it a
literal, so the length is known rather than recovered with strlen().

Signed-off-by: James M Snell <jasnell@gmail.com>
Use timers/promise setTimeout and fix a hang in a test

Signed-off-by: James M Snell <jasnell@gmail.com>
Signed-off-by: James M Snell <jasnell@gmail.com>
Signed-off-by: James M Snell <jasnell@gmail.com>
Signed-off-by: James M Snell <jasnell@gmail.com>
Signed-off-by: James M Snell <jasnell@gmail.com>
Assisted-by: Opecode
@jasnell
jasnell force-pushed the jasnell/dtls-improvements branch from 22bb59d to 36be49d Compare September 15, 2026 16:00
@jasnell
jasnell requested a review from panva September 15, 2026 16:01
@panva panva added author ready PRs with CI started, the required approvals, and no outstanding review comments. request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. labels Sep 15, 2026
@github-actions github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. label Sep 15, 2026
@nodejs-github-bot

This comment was marked as outdated.

@nodejs-github-bot

This comment was marked as outdated.

@nodejs-github-bot

nodejs-github-bot commented Sep 15, 2026

Copy link
Copy Markdown
Collaborator

jasnell added a commit that referenced this pull request Sep 16, 2026
Signed-off-by: James M Snell <jasnell@gmail.com>
Assisted-by: Opecode
PR-URL: #65511
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Filip Skokan <panva.ip@gmail.com>
@jasnell

jasnell commented Sep 16, 2026

Copy link
Copy Markdown
Member Author

Landed in bd2b98f

@jasnell jasnell closed this Sep 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

author ready PRs with CI started, the required approvals, and no outstanding review comments. dtls Issues and PRs related to Datagram Transport Layer Security (DTLS). experimental Issues and PRs related to experimental features. large-pr PRs subject to the large-PR policy. lib / src Issues and PRs involving general changes in the lib/ or src/ directories. needs-ci PRs that need a full CI run. net Issues and PRs related to the net subsystem.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants