Fix : accept non-canonical public values per RFC 7748

[jackctj117]

This pull request significantly expands and clarifies the test coverage for Curve25519 and Curve448 public key validation and shared secret derivation, especially around edge cases involving non-canonical and low-order public values as specified by RFC 7748. The changes ensure that the implementation strictly adheres to the RFC's requirements for accepting or rejecting specific encodings, and adds known-answer tests to confirm correct shared secret computation for non-canonical inputs.

The most important changes are:

Curve25519 Test Enhancements:

• Added comprehensive tests in test_wc_curve25519_check_public_le and test_wc_curve25519_check_public_be to cover non-canonical encodings of low-order values (e.g., p, p+1, p+2...p+18), verifying that they are accepted or rejected according to RFC 7748 and that verdicts match canonical encodings. [1] [2]
• Introduced a new known-answer test function, test_wc_curve25519_shared_secret_noncanonical_kat, which validates that shared secrets derived from non-canonical public values (p+2, p+9, p+18) match those from their canonical counterparts, using expected results from OpenSSL. This function is registered in the test suite. [1] [2] [3]
• Extended test_wc_curve25519_shared_secret_zero_check to test additional low-order points (both canonical and non-canonical encodings) to ensure they are properly rejected during shared secret computation, regardless of public key validation.

Curve448 Test Clarifications:

• Updated comments and test logic in test_wc_curve448_check_public_le and test_wc_curve448_check_public_be to clarify the handling of non-canonical and low-order public values, ensuring acceptance or rejection aligns with RFC 7748. This includes explicit tests for p+1 (non-canonical 1, rejected), p+2 (accepted), and p-1 (rejected). [1] [2] [3]

These changes improve the robustness of the test suite, ensure standards compliance, and clarify the expected behavior for edge-case public key encodings.

[wolfSSL-Bot]

Can one of the admins verify this patch?

[Copilot]

Pull request overview

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Expands validation and shared-secret test coverage for X25519/X448 around RFC 7748 non-canonical encodings and low-order public values, and updates public key validation logic to accept required non-canonical inputs while rejecting cheaply detectable low-order points.

Changes:

• Update wc_curve25519_check_public() / wc_curve448_check_public() to reject only specific low-order (and non-canonical low-order) encodings while accepting the RFC-mandated non-canonical ranges.
• Add new known-answer tests and edge-case tests for shared secret derivation and public key validation (canonical vs non-canonical equivalence).
• Add a TLS 1.3 server-side test that feeds a canned ClientHello to verify X25519 non-canonical key shares are accepted while low-order points are rejected.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
wolfcrypt/test/test.c	Adds non-canonical accept/reject vectors and non-canonical shared-secret KATs for Curve25519/Curve448 in the wolfcrypt test suite.
wolfcrypt/src/curve448.c	Refines Curve448 public-key validation to reject p-1/p/p+1 (and canonical 0/1) while accepting other non-canonical values per RFC 7748.
wolfcrypt/src/curve25519.c	Refines Curve25519 public-key validation to reject only p-1/p/p+1 (and 0/1) while accepting p+2..p+18 per RFC 7748.
tests/api/test_tls13.h	Registers a new TLS 1.3 X25519 non-canonical key share API test.
tests/api/test_tls13.c	Adds the TLS 1.3 canned-ClientHello test to exercise X25519 non-canonical key shares and low-order rejection behavior.
tests/api/test_curve448.h	Registers a new Curve448 non-canonical shared-secret KAT API test.
tests/api/test_curve448.c	Adds Curve448 non-canonical shared-secret KATs and clarifies/extends check_public tests for non-canonical cases.
tests/api/test_curve25519.h	Registers a new Curve25519 non-canonical shared-secret KAT API test.
tests/api/test_curve25519.c	Extends Curve25519 zero-check tests for canonical/non-canonical low-order points and adds non-canonical shared-secret KATs.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[github-actions]

retest this please

[wolfSSL-Fenrir-bot]

Fenrir Automated Review — PR #11272

Scan targets checked: wolfcrypt-bugs, wolfcrypt-port-bugs, wolfcrypt-rs-bugs, wolfcrypt-src, wolfssl-bugs, wolfssl-src

Findings: 1
1 finding(s) posted as inline comments (see file-level comments below)

This review was generated automatically by Fenrir. Reported findings require changes before merge.

[github-actions]

MemBrowse Memory Report

gcc-arm-cortex-m4-dtls13

• FLASH: .text +64 B (+0.0%, 186,492 B / 1,048,576 B, total: 18% used)

gcc-arm-cortex-m4-openssl-compat

• FLASH: .text +64 B (+0.0%, 780,716 B / 1,048,576 B, total: 74% used)

gcc-arm-cortex-m4-tls13

• FLASH: .text +64 B (+0.0%, 242,063 B / 262,144 B, total: 92% used)

linuxkm-pie

• Data: __patchable_function_entries +16 B (+0.1%, 26,952 B)

linuxkm-standard

• Data: __patchable_function_entries +16 B (+0.0%, 49,760 B)
  No memory changes detected for:
• gcc-arm-cortex-m0plus
• gcc-arm-cortex-m3
• gcc-arm-cortex-m4
• gcc-arm-cortex-m4-baremetal
• gcc-arm-cortex-m4-crypto-only
• gcc-arm-cortex-m4-min-ecc
• gcc-arm-cortex-m4-pkcs7
• gcc-arm-cortex-m4-pq
• gcc-arm-cortex-m4-rsa-only
• gcc-arm-cortex-m4-sp-math
• gcc-arm-cortex-m4-tls12
• gcc-arm-cortex-m7
• gcc-arm-cortex-m7-pq
• gcc-arm-cortex-m7-tls13
• stm32-sim-stm32h753

[jackctj117]

retest this please