Testing

Testing with Tensor9 BYOC involves validating releases in test appliances before deploying to customer appliances. Test appliances are isolated environments that mirror customer deployment targets, allowing you to verify functionality, performance, and compatibility without affecting production customers.

Test appliances

A test appliance is an automatically managed environment that Tensor9 BYOC provisions for testing purposes. Test appliances:

Run in vendor-controlled infrastructure (your cloud account)
Mirror the form factor and configuration of customer appliances
Allow you to validate releases before production deployments
Require an existing test setup arranged with Tensor9 before release workflows
Support the full deployment lifecycle (compile, deploy, observe, operate)

Test appliances function identically to customer appliances but are designed for pre-production validation. This ensures what you test matches what customers will experience.

Why test in test appliances

Testing in test appliances before deploying to customers provides several safeguards:

What to Test	What You Validate
Compilation	Origin stack compiles correctly for the target form factor. Catch compilation errors, missing resources, or incompatible service mappings.
Deployment	Deployment stack deploys successfully. Identify issues with resource dependencies, configuration errors, or provider-specific constraints.
Functionality	Application works correctly in the target environment. Test API endpoints, database connections, storage access, and other application features.
Observability	Telemetry (logs, metrics, traces) flows correctly to your observability sink. Verify monitoring dashboards and alerts work as expected.
Performance	Application performance in the target form factor. Identify performance differences across cloud providers or managed service implementations.
Form Factor Compatibility	Application works consistently across all target environments when supporting multiple form factors (AWS, Google Cloud, Azure, private).

Prepare test appliances

This guide uses existing, ready test appliances for your app. Test-appliance creation is not available through the public CLI; contact Tensor9 if you need one. To test a release yourself without a test appliance, onboard a second AWS account of your own as a customer with tensor9 app setup and release to it with -customerName, as the Terraform quick start shows.

Select an AWS test appliance

Choose your existing AWS test appliance with the required form factor and region. Use tensor9 report to check that it is ready before creating a release.

Select test appliances for multiple form factors

Choose a ready test appliance for each form factor you want to test. Use your existing appliance names in the release commands.

Viewing test appliances

Use tensor9 report to see all your test appliances and their status:

tensor9 report

Example output showing test appliances:

Test Appliances: (2)
    Test Appliance: test-aws-us-west-2 [id: 000000000000009a]:
        Status:              Live
        Name:                test-aws-us-west-2
        Customer:            Acme Software Test [id: 000000000000003c]
        Cloud Details:       Aws(us-west-2)
        Form Factor:         aws-connected
        Appliance Id:        000000000000009a

        Test Appliance Name:  test-aws-connected
        Test Appliance Id:    000000000000003b:0000000000000006

        Installs:
            Acme Software/my-app → Acme Software Test [id: 0000000000000213:000000000000009a:000000000000016a]
                Vendor:   Acme Software [id: 000000000000003b]
                App:      my-app [id: 0000000000000213]
                Customer: Acme Software Test [id: 000000000000003c]
                Release:  Acme Software/my-app → Acme Software Test [version: 1.3.0-rc1]
                    Version: 1.3.0-rc1
                    Outputs:
                        api_endpoint: https://api.test-aws-us-west-2.my-app.acme.co

        Releases:
            Effective Releases: (1)
                Acme Software/my-app → Acme Software Test [version: 1.3.0-rc1]
                    Version:      1.3.0-rc1
                    Lifecycle:    Submitted
                    Deployment:   Deployed
                    Created:      4 hours ago
                    Updated:      4 hours ago
                    Description:  Release candidate for testing
                    Origin:       s3://t9-ctrl-000001/my-stack.tf.tgz
                    Id:           s3://t9-ctrl-000001/my-stack.tf.tgz:5ef42c62:0000000000000812
                    Notes:        Testing new features
            Prepped Releases: (0)

        Hardware: (updated 45 seconds ago)
            Uptime:              6 hours 23 minutes
            Capacity Machines:   1

    Test Appliance: test-gcp-us-central1 [id: 000000000000009b]:
        Status:              Live
        Name:                test-gcp-us-central1
        Customer:            Acme Software Test [id: 000000000000003c]
        Cloud Details:       Gcp(us-central1)
        Form Factor:         gcp-connected
        Appliance Id:        000000000000009b

        Installs:
        Releases:
            Effective Releases: (0)
            Prepped Releases: (0)

The report shows:

Status: Whether the appliance is Live and ready for deployments
Cloud Details: The cloud provider and region
Form Factor: Which form factor the appliance uses
Installs: Apps deployed to the appliance
Releases: Deployed releases with version, deployment status, and outputs
Hardware: Appliance uptime and capacity

Testing workflow

Follow this workflow to test releases before deploying to customers:

Select an existing test appliance

Select a ready test appliance for your app and check its status with tensor9 report. The example below uses test-aws-us-west-2; replace it with your appliance name.

Publish origin stack

Publish your origin stack to your control plane:

tensor9 stack publish-direct \
  -appName my-app \
  -stackType TerraformWorkspace \
  -stackName my-stack \
  -dir /path/to/terraform

This uploads your infrastructure code through your control plane and binds it to your app. Publishing again with the same -stackName keeps the binding.

Create release for test appliance

Create a release targeting your test appliance:

tensor9 stack release create \
  -appName my-app \
  -testApplianceName test-aws-us-west-2 \
  -vendorVersion "1.5.0-rc1" \
  -description "Testing new analytics feature" \
  -notes "By [email protected]"

Tensor9 BYOC compiles your origin stack for the test appliance and downloads the deployment stack into a directory named after your test appliance.

Deploy to test appliance

Deploy the compiled stack using standard tooling:

cd test-aws-us-west-2
tofu init
tofu apply

The tensor9 provider in the deployment stack reads your API key from T9_API_KEY. Set it at a hidden prompt for the tofu commands only, and unset it as soon as they finish, as the Terraform quick start shows: while it is set, tensor9 commands use the key instead of your controller profile.The deployment executes in your test appliance.

Validate the release

Test your application in the test appliance:

# Check outputs from deployment
tofu output

# Test API endpoints
curl https://api.test-aws-us-west-2.my-app.acme.com/health

# Run integration tests
./scripts/integration-test.sh https://api.test-aws-us-west-2.my-app.acme.com

# Verify observability
# Check logs, metrics, and traces in your observability sink

Validate functionality, performance, and observability before proceeding.

Deploy to customer appliances

Once validated, deploy the same release to customer appliances:

tensor9 stack release create \
  -appName my-app \
  -customerName acme-corp \
  -vendorVersion "1.5.0" \
  -description "New analytics feature" \
  -notes "By [email protected]"

cd acme-corp-production
tofu init
tofu apply

The tested release deploys to the customer appliance.

Testing across form factors

To test across cloud providers or deployment environments, choose a ready test appliance for each form factor. Replace the example names below with your existing appliance names. Then test your release in each environment:

# Test in AWS
tensor9 stack release create -appName my-app -testApplianceName test-aws-us-west-2 -vendorVersion "1.5.0-rc1" -description "Test release 1.5.0-rc1" -notes "By [email protected]"
cd test-aws-us-west-2 && tofu init && tofu apply
# Validate...

# Test in Google Cloud
tensor9 stack release create -appName my-app -testApplianceName test-gcp-us-central1 -vendorVersion "1.5.0-rc1" -description "Test release 1.5.0-rc1" -notes "By [email protected]"
cd test-gcp-us-central1 && tofu init && tofu apply
# Validate...

# Test in Azure
tensor9 stack release create -appName my-app -testApplianceName test-azure-eastus -vendorVersion "1.5.0-rc1" -description "Test release 1.5.0-rc1" -notes "By [email protected]"
cd test-azure-eastus && tofu init && tofu apply
# Validate...

This validates that service equivalents work correctly and your application performs consistently across all target environments.

Integration testing

Integrate test appliances into your CI/CD pipeline for automated testing:

# Example GitHub Actions workflow
name: Test Release

on:
  push:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Install Tensor9 CLI
        run: |
          curl -sSL https://t9-artifacts-prod-1.s3.us-west-2.amazonaws.com/install-latest.sh | sh
          echo "$HOME/.tensor9/bin" >> $GITHUB_PATH

      - name: Publish origin stack
        env:
          T9_API_KEY: ${{ secrets.T9_API_KEY }}
        run: |
          tensor9 stack publish-direct \
            -appName my-app \
            -stackType TerraformWorkspace \
            -stackName my-stack \
            -dir ./terraform

      - name: Create release for test appliance
        env:
          T9_API_KEY: ${{ secrets.T9_API_KEY }}
        run: |
          tensor9 stack release create \
            -appName my-app \
            -testApplianceName test-ci \
            -vendorVersion "${GITHUB_SHA:0:7}" \
            -description "CI test build" \
            -notes "By [email protected]"

      - name: Deploy to test appliance
        env:
          T9_API_KEY: ${{ secrets.T9_API_KEY }}
        run: |
          cd test-ci
          tofu init
          tofu apply -auto-approve

      - name: Run integration tests
        run: |
          cd test-ci
          API_ENDPOINT=$(tofu output -raw api_endpoint)
          ./scripts/integration-test.sh $API_ENDPOINT

This automatically validates every change in a test appliance before deploying to customers.

Versioning for test releases

Use version suffixes to distinguish test releases from production releases:

# Release candidate
tensor9 stack release create \
  -appName my-app \
  -testApplianceName test-aws \
  -vendorVersion "1.5.0-rc1" \
  -description "Test release 1.5.0-rc1" \
  -notes "By [email protected]"

# Beta release
tensor9 stack release create \
  -appName my-app \
  -testApplianceName test-aws \
  -vendorVersion "1.5.0-beta" \
  -description "Test release 1.5.0-beta" \
  -notes "By [email protected]"

# Development build
tensor9 stack release create \
  -appName my-app \
  -testApplianceName test-aws \
  -vendorVersion "1.5.0-dev" \
  -description "Test release 1.5.0-dev" \
  -notes "By [email protected]"

After validation, create the production release without the suffix:

tensor9 stack release create \
  -appName my-app \
  -customerName acme-corp \
  -vendorVersion "1.5.0" \
  -description "Release 1.5.0" \
  -notes "By [email protected]"

This makes it clear which releases are for testing vs. production.

Retiring test appliances

Test-appliance retirement is not available through the public CLI. Contact Tensor9 to arrange retirement, including data retention and ongoing infrastructure costs.

Best practices

Use existing test appliances

Use a suitable existing test appliance for each form factor you intend to validate. Keep it up to date and review its runtime costs.

Test before every customer deployment

Always validate releases in test appliances before deploying to customer appliances. This catches issues early and reduces customer-facing incidents.

Integrate test appliances into your CD pipeline

Integrate test appliances into CI/CD pipelines. Automatically deploy to test appliances and run validation tests on every commit or pull request. Regularly deploy to test appliances to keep them in sync with your latest changes - stale test appliances may not accurately reflect production behavior.

Test all form factors

If you support multiple cloud providers, test in all of them. Service equivalents may behave differently across providers, and testing catches these differences.

Monitor test appliances

Use the same observability setup for test appliances as you do for customer appliances. This validates that telemetry flows correctly and your monitoring works.

Retire unused test appliances

Contact Tensor9 about test appliances you no longer need, including data retention and infrastructure costs.

Deployments: How to create releases and deploy
Appliances: Understanding customer vs. test appliances
Observability: Monitoring test appliances
Operations: Operating test appliances

Tensor9 BYOC

Getting Started

Self-Service Onboarding

How It Works

Deploying

Your Customer's Environment

Service Adapters

Costs

Auto-Customizations

Security

Connectivity & Networking

Observability & Operations

Support

tensor9 CLI

Test appliances

Why test in test appliances

Prepare test appliances

Select an AWS test appliance

Select test appliances for multiple form factors

Viewing test appliances

Testing workflow

Testing across form factors

Integration testing

Versioning for test releases

Retiring test appliances

Best practices

​Test appliances

​Why test in test appliances

​Prepare test appliances

​Select an AWS test appliance

​Select test appliances for multiple form factors

​Viewing test appliances

​Testing workflow

​Testing across form factors

​Integration testing

​Versioning for test releases

​Retiring test appliances

​Best practices

​Related topics

Test appliances

Why test in test appliances

Prepare test appliances

Select an AWS test appliance

Select test appliances for multiple form factors

Viewing test appliances

Testing workflow

Testing across form factors

Integration testing

Versioning for test releases

Retiring test appliances

Best practices

Related topics