crypto,build: require OpenSSL 3, drop the bundled FIPS provider build

[panva]

Important

Prior to marking this ready for review, or more specifically, running a full CI, a change similar to nodejs/build#4386 needs to land to exclude OpenSSL 1.1.1 builds for gte(27) (nodejs/build#4409).

This will hang around as draft for comments until we get to v27.x alphas

Drops support for building against OpenSSL 1.x, and with it a FIPS build path that never worked. #64211 split the native crypto backends so this could be a focused follow-up.

configure now fails when --shared-openssl points at anything older than 3.0.0, instead of failing later at compile or link time. The check skips BoringSSL, whose version macros claim 1.1.1. crypto.fips is removed. It was runtime-deprecated as DEP0093 in v23.0.0 and has an automated migration. crypto.getFips() and crypto.setFips() are unaffected. --openssl-is-fips now requires --shared-openssl, since with the bundled OpenSSL it no longer has anything to configure.

--openssl-is-fips against the bundled OpenSSL never worked. openssl-fipsmodule has no dependency edge from node.gyp, so gyp emitted no rules for it and fipsinstall's input was produced by nothing. The edge was commented out in the commit that introduced the machinery and never restored. Repairing it would not help either, since OpenSSL requires a FIPS provider to be built from a release carrying a FIPS certificate, so a provider built out of the Node.js tree has no validation status regardless.

A note for reviewing the guards. BoringSSL does not define OPENSSL_VERSION_MAJOR and sets OPENSSL_VERSION_NUMBER to 1.1.1, so every version check was excluding it too. The legacy arms are live BoringSSL code, not dead 1.x code. Only guards already nested inside an #ifndef OPENSSL_IS_BORINGSSL were unwrapped. The rest had their condition rewritten to OPENSSL_IS_BORINGSSL with both arms kept. Getting one of those wrong usually breaks the BoringSSL build outright, with two exceptions that still compile: crypto_hash.cc's three-way EVP_MD_do_all_sorted chain, whose final arm serves BoringSSL with libdecrepit, and ncrypto.cc's X509 info-access newline.

The unreleased DEP0183 Runtime deprecation is reverted before DEP0183 moves directly from Documentation-only to End-of-Life. crypto.setEngine(), crypto.constants.ENGINE_METHOD_*, and the native ENGINE implementation are removed. The former clientCertEngine, privateKeyEngine, and privateKeyIdentifier TLS and HTTPS options remain recognized and throw ERR_CRYPTO_CUSTOM_ENGINE_NOT_SUPPORTED when given a non-nullish value, so engine-dependent configurations cannot be silently ignored.

Fixes: #56733
Refs: #64211
Refs: #63966
Refs: #53329
Refs: #48950
Refs: #45465
Refs: #42827
Refs: #47076
Refs: #62862
Refs: #38512

[nodejs-github-bot]

Review requested:

• @nodejs/actions
• @nodejs/build
• @nodejs/config
• @nodejs/crypto
• @nodejs/gyp
• @nodejs/security-wg
• @nodejs/startup
• @nodejs/tsc
• @nodejs/userland-migrations

[panva]

@nodejs/tsc given the move to OpenSSL 4.x during 27.x alphas (nodejs/TSC#1869) I'd like to request an exception to the deprecation process and accelerate DEP0183 (OpenSSL engine-based APIs) on 27.x from Documentation-only straight to EOL instead of just Runtime (#63966). With OpenSSL 4.x (which drops engine support) we'd only be carrying forward engine support for the sake of shared openssl configurations and deployments that aren't using OpenSSL 4.x yet.