For historical reasons, the "ignored" tests in outcome analysis are not
actually ignored: they must not be covered, otherwise the script complains
about an unnecessary exception. In coverage analysis, rename this behavior
to "uncovered", and have "ignored" tests be actually ignored. In driver test
parity analysis, which is now only done in the 3.6 LTS branch, keep the
historical behavior
Consuming branches are currently defining `IGNORED_TESTS` with the
expectation that the test cases must be uncovered. They will need to rename
their definition to `UNCOVERED_TESTS`.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Outcome analysis tasks can have "ignored" tests. Both coverage and driver
tasks actually don't ignore "ignored" tests: an "ignored" test must fail the
verification if it wasn't ignored.
In preparation for distinguishing between truly ignored tests and tests that
must be uncovered, generalize the test case lookup mechanism.
No intended behavior change for `CoverageTask` and `DriverVSReference`.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
The `read_file_lines` context manager supported either text or binary
streams, based on a parameter passed to the constructor. But the type
annotation on the iterator claimed that all lines were text. The version of
mypy that we use on the CI was happy with that, but modern versions are not.
The advantage of `read_file_lines` over built-in functions is better
tracking of line numbers. We never took advantage of this in our code with
binary streams. Change the one place where `read_file_lines` was used with a
binary stream to use built-in functions instead, and specialize
`read_file_lines` to text streams.
This fixes a legitimate complaint of modern mypy on `macro_collectory.py`.
There was no runtime bug: the behavior was correct, only the type
annotations were wrong.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
We were missing `mbedtls_framework/code_wrapper/*.py` because we were not
traversing the directory recursively.
Also improve the comment explaining the duplicate-code workaround for code
that's being moved between files in different repositories.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Move a bunch of files from `scripts` and `mbedtls/scripts` to the framework.
The following files will be added (moved from Mbed TLS `development`):
* `scripts/ecp_comb_table.py`
* `scripts/massif_max.pl`
* `tests/scripts/audit-validity-dates.py` (moved to `scripts/`)
* `tests/scripts/gen_ctr_drbg.pl` (moved to `scripts/`)
* `tests/scripts/gen_gcm_decrypt.pl` (moved to `scripts/`)
* `tests/scripts/gen_gcm_encrypt.pl` (moved to `scripts/`)
* `tests/scripts/gen_pkcs1_v21_sign_verify.pl` (moved to `scripts/`)
* `tests/scripts/generate-afl-tests.sh` (moved to `scripts/`)
* `tests/scripts/generate_server9_bad_saltlen.py` (moved to `scripts/`)
* `tests/scripts/run-metatests.sh` (moved to `scripts/`)
* `tests/scripts/run_demos.py` (moved to `scripts/`)
* `tests/scripts/test_config_script.py` (moved to `scripts/`)
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Since Mbed TLS 3.6.0, all officially supported versions of Visual Studio
a printf function family that is sufficiently compliant to C99 for our
purposes, in particular supporting `%zu` for `size_t`. The only platform
without `%zu` that we semi-officially support is older versions of MinGW,
still used in our CI. MinGW provides either a Windows legacy printf or a
standards-compliant printf depending on the value of
`__USE_MINGW_ANSI_STDIO` when compiling each C file. Force the use of the
compliant version. Don't rely on `MBEDTLS_PRINTF_SIZET`, which is defined in
`<mbedtls/debug.h>` and no longer considers the Windows legacy version in
Mbed TLS >= 4.1.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Support check and always-update mode. Update-if-needed mode falls back to
always-update mode because test_data_generation.py doesn't support
update-if-needed.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
As part of a long-term unification effort of generation scripts, add an
option for test generators to list oudated targets without writing to files.
This corresponds to functionality that the new generate_files_helper module
offers.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
A new utility to give a common interface to committed generated files.
For the time being, this module is only intended for committed generated
files, and `make_generated_files_common.py` is only intended for non-committed
generated files. The two may be unified at some point in the future.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
`TextChangelogFormat.version_title_text` never worked: it's missing a
parameter in the call to `re.sub`. Fortunately it's unused, so just remove it.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Use NIST test vectors. I copied the first test case for each parameter set.
For ML-KEM and ML-DSA, I did it manually. For SLH-DSA, I used the following
Python script:
```
<gen-val/json-files/SLH-DSA-keyGen-FIPS205/internalProjection.json python -c '
import json, re, sys
data = json.load(sys.stdin)
types = {family: {} for family in ["SHA2_F", "SHA2_S", "SHAKE_F", "SHAKE_S"]}
for group in data["testGroups"]:
t = group["tests"][0]
m = re.match(r"SLH-DSA-(\w+)-([0-9]+)(\w+)", group["parameterSet"])
type = m.group(1) + "_" + m.group(3).upper()
size = m.group(2)
prv = t["skSeed"] + t["skPrf"] + t["pkSeed"] + t["pk"]
pub = t["pkSeed"] + t["pk"]
types[type][size] = (prv, pub)
for type in sorted(types.keys()):
print("")
print(f" # https://github.com/usnistgov/ACVP-Server/blob/v1.1.0.41/gen-val/json-files/SLH-DSA-keyGen-FIPS205/internalProjection.json")
print(f" \x27SLH_DSA(PSA_SLH_FAMILY_{type})\x27: {{")
for size in sorted(types[type].keys()):
entry = types[type][size]
print(f" {size}: (\"{entry[0]}\",")
print(f" \"{entry[1]}\"),")
print(f" }},")
'
```
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Define the necessary new categories (key wrap, key encapsulation, XOF).
This completes the list of mechanisms in the PSA Crypto API version 1.4,
except for PAKE that are more challenging (family parametrization, need to
find sample keys in `asymmetric_key_data.py`), and except for the PQC
extension.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Ben Taylor