The trust chain at a glance
Two Ed25519 keypairs anchor the system:- Your customer’s signing key, generated on your customer’s workstation when they first sign anything (a pre-approval, a release manifest). The private key lives in a local keychain on their workstation; the public key is pinned into the appliance controller’s secret store by a cloud-native write your customer runs in their own cloud account.
- The appliance controller’s signing key, generated by the appliance controller the first time the install runs. The private key lives in the appliance controller’s secret store under your customer’s IAM (Tensor9 cannot read it). The public key is registered on the install record in your control plane so anyone can fetch it to verify signatures.
What the three signatures prove
The lifecycle has three signature transitions. Each covers different bytes and proves a different fact.| Signature | Signed when | What bytes are covered | What it proves |
|---|---|---|---|
commandApproval | Your customer approves the request in Step 1 of the approval UI | Canonical form of {cmdId, decision, at, approver, reason, sha256(rawCommand)} | The exact command body and variable values were approved by this person at this time |
outputIntegrity | Output capture finishes on the appliance controller, before encryption | Canonical form of {cmdId, executedAt, exitCode, sha256(stdout + stderr)} where stdout / stderr are the small [blob: bucket=..., key=..., size=..., sha256=...]\\n<presigned-url> payloads stored on the command record | The blob-payload bytes the vendor reads on retrieve are the exact payloads the appliance controller produced. The payload’s embedded sha256 then binds the URL to the actual stream bytes, which the vendor independently re-verifies on curl. |
outputApproval | Your customer releases output in Step 4 | Canonical form of {cmdId, decision, at, approver, reason, sha256(exitCode + stdout + stderr)} over the same payloads | The release decision covers these specific blob-payload bytes (and, transitively via the embedded sha256, the actual stream bytes) and was made by this person at this time |
- The appliance controller signs the cmd record’s stdout / stderr (the small blob-payload strings).
- Inside each payload, a
sha256=<hex>field binds the presigned URL to specific bytes. - When the customer or vendor fetches the URL and re-computes the sha256, any byte-level tamper between the appliance controller’s upload and the read is detected (the release script exits non-zero on mismatch; you can do the same client-side).
commandApproval (which is
signed against their pinned pubkey) or repudiate the pinning step
itself (which they did with their own cloud credentials). Neither
is plausible without their key material.
Where keys live
Your customer’s signing key
Your customer owns and stores their own private signing key. The support-portal approval UI handles the setup on first approval (the Set up your signing keypair step) and renders the bash snippets your customer pastes into their own terminal. Today the approval UI supports two storage backends, matched to the appliance environment:- AWS appliances: private key in your customer’s AWS SSM Parameter Store as a SecureString (KMS-encrypted at rest).
- Kubernetes appliances: private key as a Kubernetes Secret in the cluster namespace.
| Property | Value |
|---|---|
| Algorithm | Ed25519 |
| Private key | Lives in your customer’s own secret store: AWS SSM Parameter Store SecureString on AWS appliances, or a Kubernetes Secret on Kube appliances. The browser never touches the private key. |
| Public key | Pinned to the appliance controller’s secret store at a path the appliance controller verifier reads on every poll. The approval UI’s “Pin the public key” snippet writes to the same backend (SSM or Kubernetes Secret) under a parallel path the appliance controller has IAM to read. See Standing pre-approvals for the exact paths. |
| How it gets there | One-time per appliance: your customer opens any support link, the approval UI detects no pinned key, and walks them through three bash snippets they paste into their terminal: openssl genpkey -algorithm Ed25519 on their workstation to generate the keypair, an aws ssm put-parameter / kubectl create secret to store the private key, and a second put-parameter / create secret to pin the public key. The approval UI polls until the appliance controller reports the pubkey is visible, then advances. |
| Used to sign | Pre-approval grant manifests and per-command approval / release manifests. Signing is local: the approval UI’s Step 5 / release snippets fetch the private key from your customer’s storage, sign the canonical bytes with openssl pkeyutl, and emit a base64 signature your customer pastes back into the approval UI. |
| Recovery | The private key lives in your customer’s own SSM Parameter Store or Kubernetes Secret, so it survives workstation loss: any workstation with your customer’s cloud credentials can refetch it and resume signing. If the secret itself is deleted (e.g., your customer accidentally wipes the parameter), see “Lost-key recovery” in Standing pre-approvals. |
The appliance controller’s signing key
| Property | Value |
|---|---|
| Algorithm | Ed25519 |
| Private key | Lives in the appliance controller’s secret store under your customer’s IAM. Your control plane cannot read it; only the appliance controller process can. |
| Public key | Registered on the install record in your control plane so both you and your customer can fetch it to verify signatures. |
| How it gets there | The appliance controller generates the keypair when the install starts and writes the private key to its own vault before any ops command can run. |
| Used to sign | Every commandApproval, outputIntegrity, and outputApproval transition the appliance controller acts on. |
| Recovery | Lost only if the appliance is destroyed. A fresh install mints a new keypair and registers it on the install record (overwriting the previous pubkey slot). Signatures produced under the old key stop verifying once that overwrite happens; the install record keeps only the current pubkey, not a history. Plan for: if you need to verify old signatures across an appliance rebuild, archive the customer’s opsCmdPubKey before the rebuild. |
How non-repudiation survives revocation
Pre-approval involves two distinct artifacts stored in two distinct places. They serve different purposes and have different deletion properties.| Artifact | Stored in | Purpose | Can your customer delete it? |
|---|---|---|---|
| Signed pre-approval manifest | Your control plane (on the template lineage record) | The non-repudiation evidence: the Ed25519-signed bytes plus the signer’s embedded pubkey + fingerprint. The appliance controller fetches it from your control plane at decide time to verify against the controller-pinned buyer-signing pubkey. | No, your customer has no access to vendor infrastructure. |
| Revocation record | Appliance controller vault (your customer’s own cloud account) | Drives runtime enforcement: the appliance controller reads the revocation list on every decide cycle and refuses to act on a manifest that’s been revoked. The revocation record carries no signature. | Yes, this is how revocation works. |
Key rotation
Customers rotate their signing key for a few reasons: retiring a workstation, suspected compromise of the laptop holding the key, an employee with access leaving, or routine rotation per their own security policy. The mechanics:- Generate a fresh keypair on the new workstation. Your customer
opens any support link; the approval UI’s Step 2 setup detects no
pinned key and walks them through
openssl genpkeyfollowed by theput-parameter/create secretsnippets to store the private key and pin the public key. - Pin the new pub key to the appliance controller using the same cloud-native command pattern as the initial pin. Your customer can pin alongside the old key or replace it.
- Decide what happens to the old key:
- Leave it pinned. Pre-approvals signed by the old key keep auto-approving until they expire. New approvals get signed by the new key.
- Unpin it. Every pre-approval signed by the old key immediately fails verification. Auto-approval reverts to manual for all of them.
tensor9 ops command audit verify walks each record using the
recorded fingerprint.
What revocation does and doesn’t do
Three distinct operations get called “revocation” loosely. They have different effects.| Operation | What it does | What it does NOT do |
|---|---|---|
| Delete the appliance controller vault entry for one pre-approval | Stops auto-approval for that pre-approval on the appliance controller’s next decide cycle (within seconds) | Does not erase the signed manifest from your control plane. Does not affect other pre-approvals. |
| Delete the pinned customer pubkey | Invalidates every pre-approval signed by that key. Subsequent verification on the appliance controller fails closed; the cmd falls back to manual. | Does not erase any history. Does not retroactively undo commands that already executed. |
| Reject a per-command approval (in the approval UI) | Sets the command’s lifecycle to CmdRejected. The command never executes. | Does not erase the (rejected) approval record from your control plane. A future audit can show the customer was offered the command and declined. |
Independent verification
Both you and your customer can verify the full audit chain on a specific ops command:- Pulls the command record from your control plane.
- Fetches the appliance controller’s signing pubkey (recorded at sign time, so later key rotation does not invalidate older signatures).
- Reconstructs the canonical signed-data for each of the three signatures and verifies them against that pubkey.
- Exits zero if all signatures verify; non-zero on any failure.
[OK] and prints
the appliance controller’s signer fingerprint at the top. Failures appear as
[FAIL] with a one-line reason.
For compliance pipelines that need to fail on legacy unsigned
records (commands authored before the signature chain was required),
pass --strict. For programmatic use, pass --output json and read
the per-check signer fingerprints + signed-payload digests so the
chain can be archived independently.
A customer disputing “you ran something I didn’t approve” doesn’t
have to take your word. They run tensor9 ops command audit verify
themselves and the Ed25519 signatures either hold or they don’t.
Related
- Running commands: the lifecycle states each signature attaches to.
- Standing pre-approvals: pre-approval grant + revocation mechanics in detail.
- Authoring templates: how data-access and side-effect declarations bound what a customer is consenting to at approval time.