md5/sha1/ripemd160: fix the same 4GiB ilen truncation as sha256

Per mpg's review, MD5, SHA-1 and RIPEMD-160 have the exact same bug as
SHA-256: mbedtls_*_update() truncates ilen to its low 32 bits before
adding it to ctx->total[0], and the overflow check right after only
catches the single wraparound of that 32-bit addition -- neither step
folds in bits 32 and up of ilen itself when a single update() call is
passed >= 2^32 bytes on a 64-bit platform.

SHA-512 does not have this bug (confirmed by TimothyPMann and
independently by me): it tracks its 128-bit byte count in two
uint64_t words, so ilen (a 64-bit size_t at most) can never overflow
the low word before the carry-in step even runs.

Applied the identical fix to all three: fold ilen >> 32 into total[1],
guarded on SIZE_MAX so 32-bit platforms are unaffected.

Verified live with the same method as the SHA-256 fix: hashed a real
4GiB + 64 byte file with mbedtls_md5_update()/sha1_update()/
ripemd160_update() in a single call and compared against md5sum(1),
sha1sum(1), and `openssl dgst -rmd160`.

  algorithm    before fix (wrong)                          after fix (matches reference tool)
  MD5          0e8b3fdad1980e2c4e66ef714abc3f0c             8b647773014d9f640da9887274f444cd
  SHA-1        b190a761fceaafdc19b62cbcc729ec22b02e7d0c     703fe5b4441eb203af67ed4fbc20263d44bc5f5c
  RIPEMD-160   68956d1517f2ebcef4761d2c0fddf11331a1e6b6     d14d7e80c32ddc603c677151bfd59fd8cf867c42

Also reworded the ChangeLog entry per feedback to cover all four
functions concisely instead of describing the SHA-256 internals in
detail.

Signed-off-by: Johnson Ajibi <[email protected]>

ChangeLog.d/sha256-update-ilen-over-4gib.txt

@@ -1,6 +1,5 @@
 Bugfix
-   * Fix mbedtls_sha256_update() computing the wrong digest when a single
-     call is passed 4GiB (2^32 bytes) or more of input on a 64-bit
-     platform: the upper 32 bits of the input length were silently
-     dropped from the running byte count used at finalization. Reported
-     by TimothyPMann in #10964.
+   * Fix mbedtls_md5_update(), mbedtls_sha1_update(),
+     mbedtls_sha256_update() and mbedtls_ripemd160_update() computing the
+     wrong digest when a single call is passed 4GiB or more of input on a
+     64-bit platform. Reported by TimothyPMann in #10964.

library/md5.c

@@ -222,6 +222,10 @@ int mbedtls_md5_update(mbedtls_md5_context *ctx,
         ctx->total[1]++;
     }
 
+#if SIZE_MAX > 0xFFFFFFFF
+    ctx->total[1] += (uint32_t) (ilen >> 32);
+#endif
+
     if (left && ilen >= fill) {
         memcpy((void *) (ctx->buffer + left), input, fill);
         if ((ret = mbedtls_internal_md5_process(ctx, ctx->buffer)) != 0) {

library/ripemd160.c

@@ -285,6 +285,10 @@ int mbedtls_ripemd160_update(mbedtls_ripemd160_context *ctx,
         ctx->total[1]++;
     }
 
+#if SIZE_MAX > 0xFFFFFFFF
+    ctx->total[1] += (uint32_t) (ilen >> 32);
+#endif
+
     if (left && ilen >= fill) {
         memcpy((void *) (ctx->buffer + left), input, fill);
 

library/sha1.c

@@ -257,6 +257,10 @@ int mbedtls_sha1_update(mbedtls_sha1_context *ctx,
         ctx->total[1]++;
     }
 
+#if SIZE_MAX > 0xFFFFFFFF
+    ctx->total[1] += (uint32_t) (ilen >> 32);
+#endif
+
     if (left && ilen >= fill) {
         memcpy((void *) (ctx->buffer + left), input, fill);