Three Layers of Integrity

A common misconception in secure data pipelines is that a single hash solves all problems. It doesn’t. To prove a file is authentic, you need to separate the logical content from the physical artifact.

What SPCMP is for. SPCMP gives a float32 dataset one canonical, reversible form with a built-in commitment to its content. Same data, same container. Altered data, failed verification.

See the three layers ↓ Two profiles ↓ What I tested ↓ Deployment path ↓

The Layers

Each demo below is an illustration of behaviour I measured. Try the attacks.

LAYER 1

Logical-Content Integrity

SPCMP’s content commitment, built on SHA-256 Merkle hashing, detects any alteration of the decoded data: the float32 values and their original order.

If the decoded content is identical, the commitment passes. A change that touches only the compression layer, and leaves the data byte-for-byte the same, is not a change to the data.

Limit: the commitment travels inside the file. Someone who can rewrite the file can recompute it. Layer 1 catches corruption and edits that leave the commitment alone. It does not say who made the file.

content check · illustration12 values shown

commitment matches

LAYER 2

Artifact Integrity & Authenticity

If your threat model requires the exact serialized bytes to be unchanged, or you need to know who produced the file, sign the physical artifact.

The producer’s signing accessory takes a SHA-256 hash of the finished .pcmp file and signs it with Ed25519 into a detached manifest. The combined verifier checks the signature, the pinned key and the exact file hash before it parses the file at all, then runs the Layer 1 content check.

The private key stays with whoever is asserting provenance, protected by a passphrase or an HSM. Samayuktam never holds it.

Detached: the signature lives in a separate manifest, so the .pcmp file itself is unchanged.

receiving verifier · illustrationchecked in this order

LAYER 3 · EXTERNAL

Execution Attestation

A signed, verified file establishes what the data is and who produced it. It does not establish how it was generated.

Proving that a specific script ran on a specific machine requires hardware attestation or execution logs. Those sit outside the SPCMP protocol.

hardware attestationexecution logsbuild provenancebusiness validation

Two Deployment Profiles

Same core artifact. Choose how much authentication surrounds it. These are two selectable profiles, not two competing implementations.

Profile

Send it

	Core Verification	Authenticated Verification
Core SPCMP	Yes	Yes
Logical verification	Core verifier	Runs after authentication passes
Exact-byte authentication	No signature layer	SHA-256 digest + Ed25519 signature
Provenance across distribution	Depends on the surrounding trust model	Explicit signed-manifest verification
Integration burden	Lower	Key and manifest management

Best fit: Core Verification

Clients who generate and verify SPCMP artifacts inside their own trusted environment, and need logical-integrity verification without adding a signing workflow.

Best fit: Authenticated Verification

Clients who move artifacts across trust boundaries or need provenance, auditability and exact-byte integrity: cloud pipelines, AI artifact distribution, scientific data workflows.

The signing key belongs to the client, who is the one asserting the provenance of its own artifact. Samayuktam does not distribute its private signing key. Authenticated Verification does not replace Core Verification. It adds an authentication layer around the same core artifact, and a valid signature does not prove a computation was honest, which is why the content check still runs afterward.

What I Tested

I flipped every single bit of a small SPCMP file, one at a time, and asked the verifier about each copy.

38,440single-bit flips tested

38,430rejected by the verifier

10accepted · decoded content identical to the original

rejectedaccepted, content identical Counts are measured. Dot positions are illustrative.

Take a 1,000-element float32 file in SPCMP form (4,805 bytes).
Flip one bit. Run the verifier on the altered copy. Repeat for every bit of every byte.
Record every altered copy the verifier still accepts.
Decode each accepted copy and compare it with the original.

Result. 38,430 of 38,440 flips were rejected. The 10 that passed were bits inside the compression layer, and all 10 decoded to content byte-identical to the original. If the decoded content is identical, the commitment passes. That is the correct answer for Layer 1, and exactly why Layer 2 signs the file’s exact bytes.

Test	Scope	Result
Change a value	SPCMP test file	Rejected
Swap two positions	SPCMP test file	Rejected
Hostile-artifact regression	Malformed and hostile test artifacts	15 of 15 rejected
Boundary cases	Empty input, single element, size boundaries	Passed
Flip every single bit	1,000-element float32 file, 4,805 bytes, 38,440 flips	38,430 rejected; 10 accepted, content identical

Authenticated Verification campaign

Test	Expected	Result
Original signed artifact (100,000 elements)	Accept	Accepted
Wrong public key	Reject	Rejected
Missing manifest	Reject	Rejected
Tampered file bytes, original manifest	Reject	Rejected
Altered signature	Reject	Rejected
Valid signature, invalid content commitment	Reject	Rejected at the content check
Original artifact after all tests	Accept	Accepted

22 / 22regression checks passed

The harness covers malformed, missing and corrupted manifests and signatures; wrong, missing and unreadable keys; truncated and corrupted files; altered structure; tampered commitments; and a check that the core verifier itself was left unmodified. A valid signature alone cannot make an invalid artifact acceptable.

What Each Layer Stops

A file moving from a model-serving system to a risk platform, and what each control catches.

Attack or failure	Layer 1 · content commitment	Layer 2 · signed artifact
A value or the original order is changed	Detected	Detected (file bytes change)
A bit changes in the compression layer, decoded data identical	May pass	Detected (file bytes change)
File replaced by a forgery with a recomputed commitment	Passes	Detected (no valid signature)
The signed manifest’s digest is altered	Not necessarily	Detected (signature fails)
An older, validly signed file is replayed	Passes	Detected (release identifier in the signed manifest)
A trusted key signs outside its authorised scope	Not detected	Not detected by the signature alone; needs key scope and revocation policy
An authorised producer signs incorrect data	Not detected	Not detected; business validation
Wrong code ran, but the output is validly signed	Not detected	Not detected; execution attestation

Assumptions: Layer 1 binds both the values and their original order. Layer 2 signs a manifest that contains the exact file digest. The verifier pins the signing key independently, not from the file it is checking.

Cryptographic commitments establish the integrity of the data they cover. Signatures bind an artifact to a signing key and an authenticated statement about it. Neither establishes that the computation was correct or that the signer was authorised for this specific operation. A validly signed artifact can faithfully preserve an incorrect model output.

The Deployment Path

Start where an artifact changes hands, not inside every live transaction. The example below is a bank moving a risk-model artifact between environments. Illustrative, not a statement of any customer’s setup.

STEP 1

Data and computation

Bank data, model, code and execution environment.

STEP 2

Canonicalize and commit

SPCMP puts the data into its canonical form and computes the content commitment.

STEP 3

Finish the artifact

The .pcmp file is finalized. Only now is it hashed, so the signature never has to cover a file that contains its own hash.

STEP 4

Authenticate the exact file

A signing service backed by an HSM or key-management service signs the file’s SHA-256 digest. The private key never leaves it.

STEP 5

Distribute file and signed manifest

The manifest binds the digest to the profile and version, file length, content commitment, key ID, and a release identifier that blocks replay of older files.

STEP 6

Independent verification

The bank’s own verifier checks the signature and exact file hash first, then runs the SPCMP content and structure checks.

STEP 7

Accept or quarantine

Accept only if every required check passes. Otherwise reject, quarantine and log. There is no silent fallback to the weaker check.

Controls the receiving organisation still owns

Key custody

Signing keys stay apart from whoever produces the file. The bank pins trusted public keys, defines what each key may sign, and runs rotation and revocation.

Signed manifest

A defined, canonically encoded manifest, so the exact signed bytes are unambiguous.

Replay and rollback

A valid signature does not mean newest. The signed manifest carries a release identifier the verifier checks, so an old file cannot replace a new one.

Audit evidence

Digest, commitment, key ID, verifier version, result, time and release ID go to tamper-evident storage, with duties separated.

Where It Fits

Anywhere a float32 artifact changes hands and someone downstream has to trust it without redoing the work.

Banking & capital markets

The handoff: Models and checkpoints moving between environments; risk computations recorded for audit.
What SPCMP adds: A canonical form and a content commitment the receiving system checks on its own.
Stays outside: Key custody, release approval, proof the computation ran as claimed.

Healthcare & clinical ML

The handoff: A validated model moving from development into the clinical environment.
What SPCMP adds: Evidence that the model running today is the one that was validated.
Stays outside: Clinical validation and regulatory approval.

Insurance & actuarial

The handoff: Pricing and reserving inputs and outputs passed between teams and vendors.
What SPCMP adds: Same data, same container, and any change detectable at the receiving end.
Stays outside: Model governance and sign-off.

Science & engineering

The handoff: Simulation output moving between clusters, labs and archives.
What SPCMP adds: A form that is identical across machines and checkable on retrieval.
Stays outside: Scientific validity of the result.

These are deployment patterns, not certifications. SPCMP alone does not supply every control a regulated environment requires.

Test it at one boundary

A pilot is one artifact handoff and one verifier. Start there.

Request Pilot Access SPCMP specs Divergence record