Analysis · October 16, 2026
The check stops at the leaf
A position paper from Amazon and OpenSSL says the server ignores the client's signature algorithms on the chain. In August I measured OpenSSL honoring them. Both are right, one certificate deep.
On 2026-08-25, Panos Kampanakis of Amazon and Viktor Dukhovni, an OpenSSL committer, dated a three-page position paper for the IAB workshop on post-quantum authentication. It reports that OpenSSL 3.5.5, s2n-tls 1.7.7, and nginx 1.31.4 choose which certificate to send “based on the leaf public key” and were “not honoring the client advertised sigAlgs in any of the signatures in the cert chain.”
Fifteen days earlier, on 2026-08-10, I had measured OpenSSL 3.5.5 honoring exactly that extension. Six dual-chain cells, and signature_algorithms_cert was the deciding variable in every one.
Both results are correct. The difference is one function and how far down the chain it looks.
If you are planning a cross-signed transition and expect the client’s advertised signature algorithms to keep the wrong intermediate off the wire, read that function before you read either of us. It is shorter than this article.
What they measured
The paper tests cross-signing as a post-quantum transition mechanism, in two shapes.
In the first, one ECDSA key gets two leaf certificates: one issued by an ECDSA intermediate, one by an ML-DSA intermediate. A legacy client should get the ECDSA chain and a post-quantum client the ML-DSA one. In the second, one ML-DSA intermediate is cross-signed by an ECDSA root and an ML-DSA root, so a verifier that understands ML-DSA but has not yet received the post-quantum root can still build a path.
Their finding is the same in both shapes. The three servers pick the leaf by its public key and send whichever chain the configuration order puts first, without consulting what the client said it could verify. A legacy client can receive the ML-DSA intermediate and fail. Their conclusion is that “some TLS implementations would need well-defined certificate chain picking logic updates.”
What I measured
My deciding cell was one configured chain, an ECDSA leaf key under an ML-DSA-44-signed intermediate, and a client that offered ecdsa_secp256r1_sha256 in both signature_algorithms and signature_algorithms_cert. The client could use the leaf’s key and had said it would not accept ML-DSA signatures on certificates. OpenSSL refused with handshake_failure before sending a certificate. nginx did the same. Caddy, rustls, and Envoy served the chain.
With two chains configured, classical and post-quantum, OpenSSL served the one the client’s signature_algorithms_cert allowed in every cell, and when no configured chain satisfied both extensions it failed closed. I wrote that up in August as “If you run OpenSSL or nginx, the extension works, the fallback is fail-closed.” That sentence needs a qualifier, and the paper is why.
The function
The TLS 1.3 server selection path in OpenSSL 3.5.5 is tls_choose_sigalg(), which calls find_sig_alg(), which calls check_cert_usable(), all in ssl/t1_lib.c at the openssl-3.5.5 tag. Four things in that last function decide everything above.
First, the certificate is chosen by key type. find_sig_alg() walks the signature schemes the client and server share, taken from signature_algorithms, and for each one asks whether the server holds a certificate whose key can sign with it. That is the “leaf public key” in the paper’s sentence, and it is correct under RFC 8446, which ties signature_algorithms to the CertificateVerify signature.
Second, the certificate’s own signature is checked only if the client sent signature_algorithms_cert. The source comment says the extension “places restrictions on the sigalg with which the certificate was signed (by its issuer)”, and the code reads the signature algorithm off the one certificate it was handed and compares it against the client’s list. My deciding cell lived here. The leaf’s signature was ML-DSA-44, the client’s list said ECDSA, and the function said no.
Third, the function receives one certificate. Its own comment says so, explaining why it cannot tell two RSA-PSS variants apart: “we do not have a chain here that lets us look at the key OID in the signing certificate.” The intermediate’s signature, which is where a cross-sign lives, is never examined on this path, with or without the extension. It is a bouncer who checks the ID of the person at the front of the line and never asks who vouched for them.
And when the client sends no signature_algorithms_cert at all, the comment is one line: “Without signat_algorithms_cert, any certificate for which we have a viable public key is permitted.” RFC 8446 section 4.2.3 says that in the absence of that extension, signature_algorithms “also applies to signatures appearing in certificates.” OpenSSL does not apply it there. That gap is already on the record as OpenSSL #32223, filed 2026-08-06, open, labeled “help wanted” with a note that it closes after a year untouched.
The check stops at the leaf.
Where each result lands
My cell exercised the second rule: a client that sent the extension, and a leaf whose own signature failed it. OpenSSL refused, which is what the code says it will do.
The paper’s first shape is a legacy client, and the paper does not say whether it sent signature_algorithms_cert. Go’s client always does. OpenSSL’s own client does not, with a comment in ssl/statem/extensions.c reading “We do not generate signature_algorithms_cert at present.” A client that omits it lands in the one-line case: every ECDSA-keyed leaf is permitted and configuration order picks. That is #32223 wearing a post-quantum intermediate. A client that sent the extension with ECDSA alone would have been refused the ML-DSA-signed leaf, as mine was, so the paper’s result tells me their client omitted it.
“Configuration order” has a precise meaning on the library’s ordinary loading path, too. OpenSSL keeps one certificate slot per key type, and ssl_set_cert() in ssl/ssl_rsa.c frees whatever is in the slot before storing the new one. Load two ECDSA-keyed leaves that way and the server holds the second. There is no choice being made badly; there is no choice.
The paper’s second shape moves the cross-sign one certificate up. Both candidate chains carry the same ML-DSA leaf with the same ML-DSA signature on it, and they differ only in who signed the intermediate. The function never reads that certificate. Nothing the client puts in either extension can reach it.
I ran that shape on 2026-10-17, the day after this went up. One ML-DSA-44 leaf, the EC-signed intermediate configured, a client sending signature_algorithms_cert with ML-DSA alone. OpenSSL served the ECDSA-signed intermediate, identified by its bytes in the Certificate message rather than by length. The same configuration under a client that excluded ML-DSA on certificates was refused, because that exclusion reaches the leaf’s own signature. The check stops one certificate short of the cross-sign, measured.
So the paper measured the depth the check does not reach, and I measured the one certificate it does. Neither of us was wrong, and neither result generalizes past its own depth. I have a retraction record for exactly that mistake, three claims on 2026-08-10, all from reading one configuration as a property of a library, so I would rather say it here than have it said for me.
Thirteen days
The dates are the evidence that this is a transition problem rather than a bug.
On 2026-08-12, in OpenSSL #32221, a report that a dual-chain server completed a handshake on a chain whose CA signature the client had not advertised, Viktor Dukhovni wrote that the deviation was “a technicality with little to no practical relevance,” that “CAs try hard to avoid mixing algorithms, and this is basically a non-issue,” and recommended closing it as won’t fix. It closed the next day.
On 2026-08-25 the same author dated the paper above. Its appendix explains why the behavior had never surfaced: cross-signing used to apply to new CAs, and “it has not been common for an entity to get issued and use two identity certificates with the same public key tying to different issuing CAs and certificate chains.”
Both statements are true. In the classical PKI the behavior was a non-issue twice over: CAs avoided mixing algorithms inside a chain, and when one did sign a P-256 leaf with a P-384 key, every client could verify both anyway. The post-quantum transition breaks both halves at once. Mixing is the design, and ML-DSA is a signature a current client may not be able to verify at all. The code was written for the old world, and the person who said so in August is the person who co-wrote the paper measuring the new one.
Still conformance
RFC 8446 section 4.4.2.2 makes the chain-signature rule a MUST only “if it is able to provide such a chain,” and tells a server that cannot to “continue the handshake by sending the client a certificate chain of its choice.” Checking one signature and stopping is a reading of that text, not a violation of it. The paper calls the result “error prone” and says implementations “would need” updates. It does not say violation, and neither do I. My own repository forbids the word, and the reason it forbids it is that a result dismissed as a standards complaint is a result nobody acts on.
What to do with it
The paper’s second shape is not a signature-algorithm question at all. The client understands ML-DSA; what it lacks is a root. The extension that carries which roots a client holds is certificate_authorities, in RFC 8446 section 4.2.4, and the draft built to carry it compactly is TLS Trust Anchor Identifiers, at revision 06 as of 2026-09-30. Expecting signature_algorithms_cert to answer “which root do you trust” asks the wrong extension.
For the first shape, and for any dual-chain server during a migration, test it with a client that actually sends the constraint. OpenSSL’s s_client will not, so an OpenSSL client against an OpenSSL server never exercises the check at all and tells you nothing about what a Go client, which always sends it, will get. The probe I built for the August measurement sends both extensions independently and reports which chain came back; it is in the repository linked below.
And do not read a passing dual-chain test on one leaf as a statement about the chain under it. The check stops at the leaf, so the test does too.
Limits
The paper’s s2n-tls result is theirs; I have not run s2n-tls. Their nginx is 1.31.4, mine was 1.31.3 on nginx:alpine with OpenSSL 3.5.7, and nginx inherited its library’s behavior exactly in my runs. The source reading is at the openssl-3.5.5 tag, which is what both of us measured. I diffed check_cert_usable() against master as of 2026-08-13 and the only change is a spelling fix in a comment, so the 52 lines are the same code today, but line numbers move and I have cited none. I have not measured the paper’s two shapes myself. The mapping of their results onto the function is argued from the source and from their description of the shapes, and it is stamped that way below.
Provenance
Measured here (Verified). The deciding cell and the six OpenSSL dual-chain cells, all runs 2026-08-10, and the four cross-signed intermediate cells of 2026-10-17 (runners/cross-signed-ica.sh, runners/FINDINGS-cross-signed.md), scripts and captured output public at pqc-chain-selection (DOI 10.5281/zenodo.21911032), one script and one output file per cell under runners/evidence/.
Read in source, not measured (Reported). check_cert_usable(), find_sig_alg(), and tls_choose_sigalg() in ssl/t1_lib.c, ssl_set_cert() in ssl/ssl_rsa.c, and the extension table entry in ssl/statem/extensions.c, read at the openssl-3.5.5 tag (commit 67b5686b, 2026-01-27) on 2026-10-16. The chain-walking function tls1_check_chain() exists in the same file and is reached from SSL_check_chain() and, under a strict-mode certificate flag that is off by default, from cipher selection; the TLS 1.3 selection path above does not read its result. RFC 8446 sections 4.2.3 and 4.4.2.2, quoted from the RFC Editor’s text.
Cited, not claimed (Reported). The paper is Practical Challenges with Cross-signed Certs as a signature algorithm transition mechanism, Kampanakis and Dukhovni, dated 2026-08-25, position paper S23 for the IAB Workshop on Accelerating the Deployment of Post-Quantum Authentication, downloaded and read in full. The two measurement shapes, the three stacks, the versions, and the quoted sentences are theirs. OpenSSL #32221 (closed 2026-08-13) and #32223 (open), with the quoted comments read on GitHub on 2026-10-16. draft-ietf-tls-trust-anchor-ids revision 06 and its date from the datatracker record. s2n-tls 1.7.7 is a release dated 2026-08-04 and nginx 1.31.4 one dated 2026-08-19, checked against the projects’ own release records.
Argued, not measured (Proposed). That the paper’s first shape falls into the no-extension branch, which assumes their legacy client omitted signature_algorithms_cert; the paper does not state it, and the inference is mine. It follows from the source and the paper’s own description of the shape, and it has not been reproduced on this bench. The second shape was, on 2026-10-17, and moved up a paragraph. What their client sent in either shape is theirs to say.
The paper is three pages. The function is 52 lines. Read the function.