Constify the return value of X509_get0_pubkey_bitstr()

Part of #28654
Fixes: https://github.com/openssl/project/issues/1772
Reviewed-by: Neil Horman <nhorman@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/29434)

Interop: c_rehash was removed; don't look for it

Signed-off-by: Norbert Pocs <norbertp@openssl.org>
Reviewed-by: Neil Horman <nhorman@openssl.org>
Reviewed-by: Nikola Pajkovsky <nikolap@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/29474)

SHA512 performance optimized by RISCV RVV

This patch is dedicated to improve SHA512 speed with RISCV
Cryptographic Vector Extension.

Below performance output is calculated by Xuantie C930 FPGA with VLEN256.
  - sha512 speed might be improved from 197032K to 1010986KB

Reviewed-by: Paul Yang <paulyang.inf@gmail.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/29263)

Added AVX2 encoding + scalar improvements

Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/29178)

Remove BIO_f_reliable() as it is broken

It was broken since the OpenSSL 3.0 release and
nobody complained. Apparently nobody is using it.
It would be practically impossible to reimplement
it with the provided EVP_MDs in backwards-compatible
manner.

Fixes #29413

Reviewed-by: Eugene Syromiatnikov <esyr@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Neil Horman <nhorman@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/29445)

Add ASN1_BIT_STRING_get_length()

From tb@openbsd.org with tests adapted by beck for OpenSSL.

Fixes: https://github.com/openssl/openssl/issues/29184
Reviewed-by: Saša Nedvědický <sashan@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Neil Horman <nhorman@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/29387)

Update doc/man3/OPENSSL_init_crypto.pod

Co-authored-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Neil Horman <nhorman@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/29385)

Update NEWS.md

Co-authored-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Neil Horman <nhorman@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/29385)

Update CHANGES.md

Co-authored-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Neil Horman <nhorman@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/29385)

- update NEWS.md

Reviewed-by: Neil Horman <nhorman@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/29385)

- fix CHANGES.md style

Reviewed-by: Neil Horman <nhorman@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/29385)

- fix .cstyle

Reviewed-by: Neil Horman <nhorman@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/29385)

Introduce OPENSSL_ATEXIT_CLEANUP env. variable.

libcrypto does not arm OPENSSL_cleanup() function as atexit(3) handler by default.
If application/user wants libcrypto to install OPENSSL_cleanup() as atexit handler,
then OPENSSL_ATEXIT_CLEANUP env. variable must be set.

If platform's libc does not provide atexit(3), then OPENSSL_ATEXIT_CLEANUP has no effect.

The OPENSSL_atexit() is wrapper of atexit(3) provided by libc now.

Reviewed-by: Neil Horman <nhorman@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/29385)

BIO_send/recvmmsg(): Pass msgs_processed to the return callback

This avoids crash if a legacy BIO callback is set and
BIO_sendmmsg() or BIO_recvmmsg() is used.

Reported by Stanislav Fort (Aisle Research)

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Norbert Pocs <norbertp@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/29395)

win-makefile.tmpl: Fix program install check

When the array is empty then copy.pl fails, because there is nothing to
copy. The empty check was done on a different variable.

Signed-off-by: Norbert Pocs <norbertp@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Nikola Pajkovsky <nikolap@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/29427)

Remove the c_rehash script

The `openssl rehash` should be used instead.

Signed-off-by: Norbert Pocs <norbertp@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Nikola Pajkovsky <nikolap@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/29427)

test/run_tests.pl: Ensure that all HARNESS_VERBOSE values are respected

... with perl truthiness in mind

Most of all, this means not having undue expectations that its value
is numerical (this is particularly true when HARNESS_VERBOSE isn't given
by the user, and this script's default is "yes").

We do this by ensuring that $tap_verbosity is turned into an appropriate
number if HARNESS_VERBOSE's value isn't numerical.

Reviewed-by: Saša Nedvědický <sashan@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/29443)

ppc64le: Restore VSX registers in AES/GCM ppc64le encrypt/decrypt

The implementation failed to restore VSX vector registers from vs14-vs22.

Re-implemented based on Linux kernel version.

Fixes #28961

Signed-off-by: Danny Tsen <dtsen@us.ibm.com>
Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/28990)

Add engine stubs test

This test tries to use all stub macros and compare return values
to verify that OPENSSL_ENGINE_STUBS define works.

Fixes openssl/project#1762

Signed-off-by: Milan Broz <gmazyland@gmail.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Norbert Pocs <norbertp@openssl.org>
Reviewed-by: Neil Horman <nhorman@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/29414)

Remove ossl_err_load_ENGINE_strings from stubs

This was an internal function and should not be here.

Signed-off-by: Milan Broz <gmazyland@gmail.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Norbert Pocs <norbertp@openssl.org>
Reviewed-by: Neil Horman <nhorman@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/29414)

Fix stub definitions for no-deprecated config

Used types are only available if OPENSSL_NO_DEPRECATED_3_0
is not defined.

Signed-off-by: Milan Broz <gmazyland@gmail.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Norbert Pocs <norbertp@openssl.org>
Reviewed-by: Neil Horman <nhorman@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/29414)

Fix typo in engine.h stub

Signed-off-by: Milan Broz <gmazyland@gmail.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Norbert Pocs <norbertp@openssl.org>
Reviewed-by: Neil Horman <nhorman@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/29414)

Harden digest context deserialization

The deserialization functions for SHA2 and SHA3 digest contexts did not
sufficiently validate the incoming data. Corruption in transmission or
on saved disk data could cause a out-of-bounds memory access if buffer
sizes did not match expected values.

Add sanity checks to the SHA2 and SHA3 deserialization functions to validate
buffer-related fields before they are used. The serialization format for these
digests has been changed to place these critical fields early in the stream to
enable this validation.

Additionally, add a note to the EVP_DigestInit man page to warn users that
deserialization should only be performed on trusted data. The checks we
implement are not meant to address processing of untrusted data
maliciously crafted by an attacker.

Application that need to store data or transmit it through untrusted
media SHOULD implement proper encryption and message authentication
on their own using things like CMS or other appropriate secure message
containers.

These check have been added also to quiet a bit security researchers
that try to find any way to claim CVE bounties even in completely
unlikely or invalid scenarios.

Signed-off-by: Simo Sorce <simo@redhat.com>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/29404)

evp_test Fix tests that were always skipped

The "Verify-Message" type looks for private keys only,
which meant anything using a public key got skipped.
Changed the entries to "Verify-Message-Public" which tries the public
key first.
This uncovered one test that needed to be removed since ECDSA-MD5 is not
a valid fetchable algorithm (in any OpenSSL provider).

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Neil Horman <nhorman@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/28040)

util: add reformat-patches script

A small script that tries to perform reformatting of a patch set.

Signed-off-by: Eugene Syromiatnikov <esyr@openssl.org>
Reviewed-by: Neil Horman <nhorman@openssl.org>
Reviewed-by: Saša Nedvědický <sashan@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/29356)

Use 'codespell:ignore' where justifiable

There's this one random string where we have the word "Hellow".  It's a
random string, "correct" spelling is really not important, so we tell
codespell to just ignore that line.

Reviewed-by: Norbert Pocs <norbertp@openssl.org>
Reviewed-by: Nikola Pajkovsky <nikolap@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/29437)

Fix legitimate spelling errors

Reviewed-by: Norbert Pocs <norbertp@openssl.org>
Reviewed-by: Nikola Pajkovsky <nikolap@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/29437)

Adjust .codespellrc

- add 'external/*' to the list of patterns to skip
- add '*.csr' to the list of patterns to skip

Reviewed-by: Norbert Pocs <norbertp@openssl.org>
Reviewed-by: Nikola Pajkovsky <nikolap@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/29437)

Add new public API for checking certificate times.

Fixes: #1631
This changes the previously internal ossl_x509_check_certificate_times()
to be the public X509_check_certificate_times(). It adds documentation
for the new function and marks X509_cmp_time, X509_cmp_timeframe,
and X509_cmp_current_time as deprecated in 4.0, as discussed in #1631.

Since the function is now public, we can replace the remaining
uses of deprecated stuff with this function.

Reviewed-by: Neil Horman <nhorman@openssl.org>
Reviewed-by: Saša Nedvědický <sashan@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/29152)

Fix silent failure in ASYNC_start_job when size is 0

When ASYNC_start_job is called with args != NULL but size == 0,
OPENSSL_malloc(0) is called. Depending on the libc implementation,
malloc(0) may return NULL, causing a silent failure.

This patch modifies the logic to skip allocation if size is 0.

CLA: trivial

Reviewed-by: Norbert Pocs <norbertp@openssl.org>
Reviewed-by: Saša Nedvědický <sashan@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/29377)

Ensure path to libcrypto can be resolved on AIX.

On AIX, the environment of the test executable for test_includes doesn't
contain a resolvable search path to the just built libcrypto. Setup
LIBPATH to point to the build results.

Fixes #29352.

Signed-off-by: Matthias Kraft <Matthias.Kraft@ibm.com>
Reviewed-by: Simo Sorce <simo@redhat.com>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/29375)

Add documentation for X509_V_FLAG_OCSP_RESP_CHECK and X509_V_FLAG_OCSP_RESP_CHECK_ALL

Reviewed-by: Norbert Pocs <norbertp@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/29327)

apps/x509.c: factor out add_object() for handling -addtrust and -addreject options

Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/28371)

Add mention of OPENSSL_ENGINE_STUBS to openssl_user_macros(7)

Fixes openssl/project#1743
Fixes openssl/project#1744

Signed-off-by: Milan Broz <gmazyland@gmail.com>
Reviewed-by: Neil Horman <nhorman@openssl.org>
Reviewed-by: Norbert Pocs <norbertp@openssl.org>
Reviewed-by: Eugene Syromiatnikov <esyr@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/29424)

Add removed EVP_PKEY_meth* functions to ossl-removed-api.pod

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Paul Yang <paulyang.inf@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/29384)

Remove all EVP_PKEY_meth_* functions from the documentation

These functions have been removed so the documentation should be updated
accordingly.

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Paul Yang <paulyang.inf@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/29384)

Clean up some final vistigial remnants of EVP_PKEY_METHOD

We retain the opaque type because it is still used by the ENGINE stubs.

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Paul Yang <paulyang.inf@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/29384)

Remove references to the pmeth field in an EVP_PKEY_CTX

We don't set this field anymore so we should remove all usage of it.

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Paul Yang <paulyang.inf@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/29384)

Delete the EC EVP_PKEY_METHOD

It is no longer used so can be removed

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Paul Yang <paulyang.inf@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/29384)

Delete the DSA EVP_PKEY_METHOD

It is no longer used so can be removed

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Paul Yang <paulyang.inf@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/29384)

Delete the RSA EVP_PKEY_METHOD

It is no longer used so can be removed

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Paul Yang <paulyang.inf@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/29384)

Delete the ECX EVP_PKEY_METHOD

It is no longer used so can be removed

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Paul Yang <paulyang.inf@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/29384)

Delete the DH EVP_PKEY_METHOD

It is no longer used so can be removed

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Paul Yang <paulyang.inf@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/29384)

Remove custom EVP_PKEY_METHODs

We retain custom EVP_PKEY_ASN1_METHODs for now - to be removed in some
subsequent PR.

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Paul Yang <paulyang.inf@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/29384)

Check return code of UTF8_putc

Signed-off-by: Norbert Pocs <norbertp@openssl.org>
Reviewed-by: Nikola Pajkovsky <nikolap@openssl.org>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/29376)

Harden ASN1_mbstring_ncopy

Reported by Murali Aniruddhan

Signed-off-by: Norbert Pocs <norbertp@openssl.org>
Reviewed-by: Nikola Pajkovsky <nikolap@openssl.org>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/29376)

The stream garbage collector loop in ossl_quic_stream_map_gc() is never entered

Also remove ready_for_gc_next(), it is not needed any more.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Norbert Pocs <norbertp@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/29402)

Revert "tests: temporarily skip MLDSA pkcs11-provider test"

This reverts commit 2b97f4d300f18b209983046f49e12f3028ba79f5.

Reviewed-by: Simo Sorce <simo@redhat.com>
Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
Reviewed-by: Neil Horman <nhorman@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/28717)

Add a couple of DH related StatementMacros, and adjust

Reviewed-by: Paul Yang <paulyang.inf@gmail.com>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Neil Horman <nhorman@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/29383)

Unbreak some function signature that got broken up after the return type

Reviewed-by: Paul Yang <paulyang.inf@gmail.com>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Neil Horman <nhorman@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/29383)

Redo source reformatting with the TypeNames added

This resulted in some source reformatting becoming a bit less trippy

This is the script I used to run clang-format appropriately enough:

    #! /bin/bash

    include_re=$(yq -r '.repos.[].hooks.[] | select(.id == "clang-format") | .files' < .pre-commit-config.yaml)
    known_generated=( crypto/bn/bn_prime.h
                      crypto/objects/obj_dat.h
                      crypto/objects/obj_xref.h
                      include/openssl/obj_mac.h
                      crypto/conf/conf_def.h
                      crypto/asn1/charmap.h
                      '*_err.c'
                      '*err.h' )

    exclusions=()
    for g in "${known_generated[@]}"; do
        # Convert pattern to a regex
        g=${g//\./\\.}
        g=${g//\*/.\*}
        exclusions+=( "^${g}\$" )
    done
    # Join all exclusions into one regex
    exclude_re=$(IFS='|'; echo "${exclusions[*]}")

    (set -x; git ls-files | grep -E $include_re | grep -Ev $exclude_re | xargs clang-format -i)

Reviewed-by: Paul Yang <paulyang.inf@gmail.com>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Neil Horman <nhorman@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/29383)

Dump all known typedefs into .clang-format

This was done by calling a script I made on the spot:

    bash add-types-to-clang-format > .clang-format.new && mv .clang-format.new .clang-format

This is what that script looked like:

    #! /bin/bash

    sed -ne '1,/^CommentPragmas:/p' < .clang-format

    cat <<_____
    # OpenSSL uses typedefs extensively. Tell clang-format about them.
    TypeNames:
    _____

    include_re=$(yq -r '.repos.[].hooks.[] | select(.id == "clang-format") | .files' < .pre-commit-config.yaml)
    include_dirs=( apps crypto include providers ssl )
    git ls-files "${include_dirs[@]}" \
        | grep -E $include_re \
        | xargs ctags -f - \
        | awk -F "\t" -- '$2 ~ /\.h/ && $4 == "t" { printf "  - \"%s\"\n", $1 }' \
        | sort \
        | uniq

    cat <<_____
      # The following types are macros, and need to remain that way, unfortunately
      - "HASH_CTX"
      - "HASH_LONG"
      - "MD32_REG_T"
    _____

    sed -ne '/^# OpenSSL uses macros extensively/,$p' < .clang-format

Reviewed-by: Paul Yang <paulyang.inf@gmail.com>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Neil Horman <nhorman@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/29383)

Refactor types that are '#define'd into proper typedefs

I looked for them with the following grep pipe, and then read through
the output to discern what are actual type definitions, and converted
what I found manually.

    git grep -En -e '#define [a-zA-Z_0-9]+ ' -- '*.h' \
        | sed -e 's|//.*||' -e 's|/\*.*\*/||g' \
        | grep -Ev ' ([0-9]|.*".*|.*\\$)'

Reviewed-by: Paul Yang <paulyang.inf@gmail.com>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Neil Horman <nhorman@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/29383)

Add a few more ASN.1 related StatementMacros

This resulted in an adjustment of reformatting that is a bit less trippy
in some places.

Reviewed-by: Paul Yang <paulyang.inf@gmail.com>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Neil Horman <nhorman@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/29383)

Missed a spot!

For some reason, source reformatting didn't catch everything.

Reviewed-by: Paul Yang <paulyang.inf@gmail.com>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Neil Horman <nhorman@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/29383)

Defang the lhash test

This is bascially eating my mac, as it now runs for 80 seconds
and eats all the CPU's exercising lock contention.

This dials it back to consume at most a quarter of the CPU's in
use by HARNESS_JOBS, unless LHASH_WORKERS is set to override it
in which case we use that.

Reviewed-by: Neil Horman <nhorman@openssl.org>
Reviewed-by: Tom Cosgrove <tom.cosgrove@arm.com>
(Merged from https://github.com/openssl/openssl/pull/29406)

apps/req.c: Always set permissions for private key output

The key output will be always private.

Reported with a proposed fix by Stanislav Fort (Aisle Research).

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/29397)

Fix Coverity issue in kdf_snmpkdf_set_ctx_params

Coverity flagged a use before NULL check error in
kdf_snmpkdf_set_ctx_params.

In this function the ctx pointer was dereferenced to obtain a
OSSL_LIB_CTX pointer, and only after that dereference was ctx checked
for being NULL.

fix is pretty clear, just move the OSSL_LIB_CTX computation down to a
point after ctx is checked for NULL.

fixes openssl/project#1765

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Eugene Syromiatnikov <esyr@openssl.org>
Reviewed-by: Norbert Pocs <norbertp@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/29400)

New -encopt option for pkey(1) and genpkey(1)

This allows setting the ML-KEM and ML-DSA output formats.
At the same fixing surprising lack of password encryption
of PKCS#8 private keys in DER output form in the CLI apps.

Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
Reviewed-by: Nikola Pajkovsky <nikolap@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/29324)

Fix codespell error in sslapitest

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/29392)

Fix error in PAC/BTI AArch64 build

Clang format removes a space here, which creates a compiler error when
building for PAC/BTI. Fix this and Turn off clang format for this line

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Tom Cosgrove <tom.cosgrove@arm.com>
(Merged from https://github.com/openssl/openssl/pull/29399)

Make the test output show how long each test takes by default.

Unless we have reduced the verbosity level, or overridden it with
a HARNESS_TIMER setting.

This makes it easier to realize when you are making a test that is
starting to dominate testing time for developers, and makes it
easier for developers to see it.

For: https://github.com/openssl/project/issues/1767

Reviewed-by: Neil Horman <nhorman@openssl.org>
Reviewed-by: Saša Nedvědický <sashan@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/29408)

method store performance improvements

The proposed architectural change focuses on improving concurrency and
reducing contention within the method store. The fundamental concept
involves moving away from a monolithic synchronisation
mechanism—specifically, a single read-write lock (rwlock)—that
currently guards the entire method store.

Instead of this single point of contention, the strategy is to
introduce per-shard synchronisation. This means the method store will
be partitioned, or sharded, into several independent segments. Each of
these segments, or shards, will be protected by its own dedicated
read-write lock.

The data in the table below was generated by running evp_fetch twenty times per thread.

|---------+----------+---------+---------+---------+---------+---------+---+--------+--------+--------+--------+--------|
|                    | Shards (u/sec)                                  |   | Improvements %
|---------+----------+---------+---------+---------+---------+---------+---+--------+--------+--------+--------+--------|
| Threads | Base     |       2 |       4 |       8 |      16 |      32 |   |      2 |      4 |      8 |     16 |     32 |
|---------+----------+---------+---------+---------+---------+---------+---+--------+--------+--------+--------+--------|
|       1 |  0.18282 | 0.18497 | 0.18306 | 0.18314 | 0.18485 | 0.18352 |   |   1.17 |   0.13 |   0.18 |   1.11 |   0.39 |
|       2 |  0.43588 | 0.35560 | 0.34131 | 0.32516 | 0.33948 | 0.35076 |   | -18.42 | -21.70 | -25.40 | -22.12 | -19.53 |
|       4 |  1.58185 | 1.06459 | 1.06258 | 0.98698 | 0.98700 | 1.06689 |   | -32.70 | -32.83 | -37.61 | -37.60 | -32.55 |
|       8 |  3.15686 | 1.75061 | 1.67458 | 1.50241 | 1.62453 | 1.74750 |   | -44.55 | -46.95 | -52.41 | -48.54 | -44.64 |
|      16 |  5.53647 | 2.83137 | 2.58007 | 2.65972 | 2.64882 | 2.82755 |   | -48.86 | -53.40 | -51.96 | -52.16 | -48.93 |
|      32 | 10.72727 | 4.97483 | 4.43692 | 4.52524 | 4.68358 | 4.84840 |   | -53.62 | -58.64 | -57.82 | -56.34 | -54.80 |
|      64 | 21.12103 | 9.43241 | 7.79981 | 7.91148 | 8.33305 | 8.34230 |   | -55.34 | -63.07 | -62.54 | -60.55 | -60.50 |

Perf tests were running on the system:
  Architecture: x86_64
  CPU op-mode(s): 32-bit, 64-bit
  Address sizes: 46 bits physical, 48 bits virtual
  Byte Order: Little Endian
  CPU(s): 96
  On-line CPU(s) list: 0-95
  Vendor ID: GenuineIntel
  Model name: Intel(R) Xeon(R) Gold 6248R CPU @ 3.00GHz
  CPU family: 6
  Model: 85
  Thread(s) per core: 2
  Core(s) per socket: 24
  Socket(s): 2

The most performant option is a configuration with 512 cache entries with
4 shards. There are two new defines NUM_SHARDS, and CACHE_SIZE which
can be tweaked at will.

Signed-off-by: Nikola Pajkovsky <nikolap@openssl.org>
Reviewed-by: Saša Nedvědický <sashan@openssl.org>
Reviewed-by: Norbert Pocs <norbertp@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/29205)

Update ossl-removed-api to mention the EVP_MD_meth* functions

All the EVP_MD_meth* functions have been removed so they should be
documented as such.

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/29366)

Remove mentions of EVP_MD_meth* from the documentation

With the removal of EVP_MD_meth* we need to update the documentation
accordingly.

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/29366)

Remove support for custom MD methods

Custom MD methods are considered legacy and have been deprecated
since 3.0. With the removal of ENGINEs they become a lot less useful
and add significant complexity to the code. We should therefore remove
them in 4.0.

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/29366)

Prevent freeing of parent listener in SSL_listen_ex

Its been reported that, when using SSL_listen_ex to obtain a new
connection from a listener, that, if the listener is freed prior to the
obtained connection, we get use-after-free conditions when freeing said
obtained connections.

This occurs because SSL_listen_ex fails to take a reference on the
parent listener SSL object (in the same way that SSL_new_from_listener
does).  If the listener is freed first, then several listener resources
are freed, which the obtained connection still makes use of, hence the
use-after-free.

The fix is to do what SSL_new_from_listener does, namely:
1) Increase the reference count on the listener SSL object.
2) Ensure that the connection qc->listener points to the listener object
   so that, when the connection is freed, we call SSL_free on the
   listener object, dropping the reference count we take in
   SSL_listen_ex.

While we're at it, this PR also modifies the quicapi test for testing
the SSL_listen_ex call, freeing the listener first to ensure that the
increased refcount holds the SSL object data stable until the connection
is freed.

Thanks to Stanislav Fort at Asile Research for pointing out this issue.

fixes openssl/project#1766

Reviewed-by: Saša Nedvědický <sashan@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/29398)

make snmpkdf.c good openssl citizen

cstyle change only

Reviewed-by: Simo Sorce <simo@redhat.com>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/29369)

make snmpkdf.c good openssl citizen

cstyle change only

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Simo Sorce <simo@redhat.com>
(Merged from https://github.com/openssl/openssl/pull/29369)

Derive EC public key from private key if missing

Update ossl_ec_key_fromdata to compute the public key if it is not provided in
the input parameters but the private key is. This allows for the creation of a
complete EC_KEY object from only private key data.

Signed-off-by: Simo Sorce <simo@redhat.com>
Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
Reviewed-by: Neil Horman <nhorman@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/29054)

Documentation for BIO flags and related functions.

Reviewed-by: Frederik Wedel-Heinen <fwh.openssl@gmail.com>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/29311)

interop: fix for engine removal

CLA: trivial

Reviewed-by: Neil Horman <nhorman@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/29333)

Remove crypto-mdebug-backtrace option from config

We still build with crypto-mdebug-backtrace enabled in a few ci jobs,
but it does nothing.

With the upcoming merge of feature/removesslv3, the code changes there
prevent the use of this option (i.e. enabling it results in
configuration failure).

It seems the most sensible thing to do here, given we have a major
release is to eliminate the option entirely, as it hasn't done anything
since 1.0.2.

Fixes openssl/project#1763

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Simo Sorce <simo@redhat.com>
Reviewed-by: Nikola Pajkovsky <nikolap@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/29380)

CONF_modules_load_file.pod: The ssl_module no longer uses global data

Remove the paragraph that is no longer relevant.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/29374)

ASN1: Reject negative BIGNUM components

In the ASN.1 structures we define the BIGNUM as positive and enforce
this during parsing. If the encoded value is negative, we raise an error
and reject the material.

Fixes #29210
Fixes #27407

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/29370)

Update ossl-removed-api to mention the EVP_CIPHER_meth* functions

All the EVP_CIPHER_meth* functions have been removed so they should be
documented as such.

Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/29299)

Remove mentions of EVP_CIPHER_meth* from the documentation

With the removal of EVP_CIPHER_meth* we need to update the documentation
accordingly.

Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/29299)

Remove support for custom cipher methods

Custom cipher methods are considered legacy and have been deprecated
since 3.0. With the removal of ENGINEs they become a lot less useful
and add significant complexity to the code. We should therefore remove
them in 4.0.

Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/29299)

Fail serialization of finalized MD contexts

Add tests to verify that `EVP_MD_CTX_serialize` and
`EVP_MD_CTX_deserialize` fail when called on a finalized `EVP_MD_CTX`.

A finalized context is in a terminal state and should not be serializable for
resumption or have a new state deserialized into it. These tests confirm the
expected failure behavior.

Signed-off-by: Simo Sorce <simo@redhat.com>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/28837)

Add documentation for digest serialization

Documents EVP_MD_CTX_[s|des]erialize functions.

Signed-off-by: Simo Sorce <simo@redhat.com>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/28837)

Use generated param decoders in SHA provider

Refactor the SHA provider's context parameter handling in sha2_prov.c to use
the generated parameter decoder framework.

This change replaces manual parameter lookups using `OSSL_PARAM_locate` and
static `OSSL_PARAM` arrays with generated decoder functions and structs. A new
template, `sha2_prov.inc.in`, is added to create the necessary decoders during
the build.

This simplifies the code, reduces boilerplate, and improves type safety.

Signed-off-by: Simo Sorce <simo@redhat.com>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/28837)

Add EVP_MD_CTX serialization for SHA3/SHAKE

Implement context serialization and deserialization for the SHA3, KECCAK,
SHAKE, and KMAC provider-based digests.

This is achieved by handling the `OSSL_DIGEST_SERIALIZATION` parameter in
get_ctx_params and set_ctx_params. A custom format is used to store the KECCAK
state, including a magic number and an algorithm identifier to ensure the
context is not loaded into an incompatible digest instance.

This allows an EVP_MD_CTX to be saved and restored, which is useful for
applications that need to checkpoint hashing operations. The existing EVP
serialization tests have been extended to cover these new algorithms.

Signed-off-by: Simo Sorce <simo@redhat.com>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/28837)

Add serialization for SHA-2 digest contexts

This commit introduces the ability to serialize and deserialize the internal
state of SHA-2 digest contexts (SHA-256 and SHA-512 families).

This functionality is exposed via the new OSSL_DIGEST_SERIALIZATION parameter,
which can be used with EVP_MD_CTX_get_params() to retrieve the state and with
EVP_DigestInit_ex2() to restore it into a new context.

This allows an application to save the state of a hash operation and resume it
later, which is useful for process migration or for saving the state of long-
unning computations. A new test case has been added to verify this.

Signed-off-by: Simo Sorce <simo@redhat.com>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/28837)

Add EVP digest context serialization

This commit introduces two new functions, EVP_MD_CTX_serialize and
EVP_MD_CTX_deserialize, to the EVP digest API.

These functions allow an application to save the state of a digest
context (EVP_MD_CTX) and restore it later. This is useful for
checkpointing long-running computations, enabling them to be paused
and resumed without starting over.

The implementation adds the OSSL_FUNC_DIGEST_SERIALIZE and
OSSL_FUNC_DIGEST_DESERIALIZE dispatch functions for providers to
supply this functionality.

Signed-off-by: Simo Sorce <simo@redhat.com>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/28837)

Add static_ASN1_SEQUENCE_END to the list of statment macros

This one is pretty special, we should ponder simplifying some of
the clever preprocessor stuff here, but for now..

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/29359)

Enable send-/recvmmsg for AIX >= 7.2 and disable SUPPORT_LOCAL_ADDR.

AIX doesn't support this implementation for local addresses. The AF_INET
case is unimplemented when sending. The AF_INET6 case is limited to 110
messages. The limiting factor is currently unclear.

Fixes #29292

Signed-off-by: Matthias Kraft <Matthias.Kraft@ibm.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Norbert Pocs <norbertp@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/29334)

Fix memory leak in error path of ec_gen_init()

ec_gen_set_params() can fail after some big numbers have already been
copied over. Those need to be cleaned to avoid a memory leak on failure.
This can be done with ec_gen_cleanup(), which is also consistent in how
the ecx_gen code does it.

Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/29335)

The return value of RSA_*_{en,de}crypt() is signed

The functions RSA_(public|private)_(en|de)crypt() return a signed
result, in particular `-1` may be returned on error, so the caller
MUST treat the value as signed.

Reviewed-by: Saša Nedvědický <sashan@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/29323)

Fix memory leak in abnormal branch of mlx_kem_dup function

Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/29318)

Fix ML-KEM/ML-DSA macro typo

This is largely cosmetic, since the macro expands to "seed" either way,
but it is best to avoid this type of error.

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/29313)

doc/man3: Add OPENSSL_ppccap.pod

This patch adds a man page documenting the OPENSSL_ppccap environment
variable that is analogous to capability environment variable man pages
for other architectures.

Fixes #17046

Signed-off-by: George Wilson <gcwilson@linux.ibm.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
Reviewed-by: Eugene Syromiatnikov <esyr@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/29230)

dtls: fix DTLSv1_listen msg_callback to report HelloVerifyRequest

DTLSv1_listen built the HelloVerifyRequest in wbuf but invoked
msg_callback with buf and DTLS1_RT_HEADER_LENGTH, and version 0.
That caused incorrect logging and could disclose the ClientHello
to write callbacks. Use wbuf and the actual record version for the
record header, and add a second callback that reports the handshake
message bytes. No change to on-wire behavior.

Signed-off-by: Joshua Rogers <MegaManSec@users.noreply.github.com>
Reviewed-by: Frederik Wedel-Heinen <fwh.openssl@gmail.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/28916)

ktls_read_record(): Harden linux recv path

- drop tag subtraction in recv buffer sizing
- enforce MSG_EOR and reject MSG_CTRUNC
- zero prepended header bytes before recvmsg

Signed-off-by: Joshua Rogers <MegaManSec@users.noreply.github.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/28861)

Fix regression when X509_V_FLAG_CRL_CHECK_ALL is set, while X509_V_FLAG_CRL_CHECK is clear

Fixes #28758

When X509_V_FLAG_CRL_CHECK is not set, the man pages document that X509_V_FLAG_CRL_CHECK_ALL is ignored.
Prior to 3.6.0, this was indeed the case.

In 3.6.0, the behavior changed, and setting X509_V_FLAG_CRL_CHECK_ALL began to imply X509_V_FLAG_CRL_CHECK.
This unfortunately breaks the majority of ruby installations, which relied on the documented behavior.

For consistency, this commit applies the same logic to the new X509_V_FLAG_OCSP_RESP_CHECK and X509_V_FLAG_OCSP_RESP_CHECK_ALL flags,
which are still undocumented as of 3.6.0.

All existing tests continue to pass.  They also make the assumption that the xxx_CHECK_ALL flags are irrelevant unless xxx_CHECK is set.
We could add a new test for this regression.  I'll leave that to another commit.

CLA: trivial

Reviewed-by: Nikola Pajkovsky <nikolap@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/28797)

(cherry picked from commit cbaf28ce48805f47345f39dc6aaf39e181ab4861)

apps/cmp.c: add -ta_in_ip_extracerts permitting non-authenticated trust anchors in IP extracerts according to 3GPP TS 33.310

Fixes #27888

Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/28015)

OSSL_CMP_OPT_PERMIT_TA_IN_EXTRACERTS_FOR_IR: use new TA(s) for whole transaction (also pkiconf); update doc

Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/28015)

run codespell pre-commit hook on staged files only

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Neil Horman <nhorman@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/29360)

Document skey decoder

Reviewed-by: Neil Horman <nhorman@openssl.org>
Reviewed-by: Simo Sorce <simo@redhat.com>
(Merged from https://github.com/openssl/openssl/pull/28278)

Implementing store support for EVP_SKEY

Reviewed-by: Neil Horman <nhorman@openssl.org>
Reviewed-by: Simo Sorce <simo@redhat.com>
(Merged from https://github.com/openssl/openssl/pull/28278)

Add SNMPKDF implementation

  In compliance with SP800-135 and RFC7860

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/29195)