The error happens when the PEM contains a public key, as it will be
first tried to be parsed as a certificate. The parsing as a certificate
fails, which then leads to a corresponding error tracked by PHP with
the next call to php_openssl_store_errors().
This change introduces an error marking to be able to reset the stored
errors to the state before trying the certificate.
Closes GH-11055
The used error code differ signficantly, so use a separate test
file.
openssl_encrypt() no longer throws an error for invalid key length,
which looks like an upstream bug.