The Metadata Format That Wasn't a Valid HTTP Header

Porting an application's file storage off a proprietary platform and onto a standard S3-compatible provider, most of the migration was mechanical: swap the client library, point it at a new endpoint, keep the same operations — read, write, list, delete. One piece wasn't mechanical at all, because the old platform's metadata key format turned out to be structurally invalid as an actual HTTP header, and the new provider, correctly, wouldn't accept it as one.

How an invalid format survives in production

The old platform's storage client was a black box — you called its methods, it handled metadata however it internally chose to, and the actual wire-level format of that metadata was never something the application needed to know or think about. It worked, for years, because the entire request-to-storage path lived inside one system's control, so an internally inconsistent format never had to cross a boundary that would reject it. Ported to a provider that speaks a real, standards-based S3 protocol, the same metadata key format hit an HTTP header validator for the first time in its life — and failed.

The actual rebuild

This meant more than swapping a client. It meant reworking the ACL and metadata model to use header-token-safe key formats from the start, and separately handling the fact that the new provider's approach to updating metadata on an existing object worked differently — the old platform allowed in-place metadata updates, while the standards-based approach required rewriting the object to itself with replacement metadata attached, a copy-to-self operation rather than a simple patch.

Why verifying byte-for-byte mattered here specifically

Given how much of the storage layer changed underneath the application, correctness couldn't be assumed from "the migration script ran without errors." I built a parity harness comparing real output — not synthetic test data — generated from the old backend against the same output generated from the new one, across a meaningful sample of real records. The results came back byte-identical. That's the actual proof point for a storage migration: not that the write succeeded, but that reading the migrated data back produces exactly, bit-for-bit, what reading the original data would have produced. Anything less than byte-identical on a storage migration is a bug you haven't found yet.