Runbook - CLI Deploy to AWS or Azure

Use this runbook to deploy Finnest Power with the shipped finnest operator CLI into an AWS or Azure environment owned by the operator.

The CLI performs the full Runtime CD flow: preflight, state bootstrap, platform infrastructure, Kubernetes runtime, Helm release, and smoke checks. Operators do not need this repository to run the flow.

What You Need

• The installed finnest CLI and the matching runtime bundle for the current version.
• A target cloud and environment:
  • --cloud=aws or --cloud=azure
  • --environment=sandbox or --environment=prod
• Cloud credentials for the target account, subscription, tenant, and region.
• Permission to create or update network, cluster, identity, secret, DNS, and state-backend resources in that target.
• A private image pull credential for Finnest GHCR runtime images: set FINNEST_IMAGE_PULL_USERNAME and FINNEST_IMAGE_PULL_TOKEN to the read-only credential Finnest provided.
• DNS delegation and TLS/certificate material for the target domain.
• Runtime secret inputs required by the target, including database, Keycloak, gateway, ICP-Brasil, and Finnest license values.

Image pull credential

The Finnest runtime images are served from private GHCR packages, so the cluster needs a read-only pull credential. Finnest grants a GitHub identity package-read access and provides you its username plus token. Set both values and the CLI threads them into the runtime as a dockerconfigjson image-pull secret.

• Most customers: set the read-only token Finnest provided for you. It grants pull access to the Finnest runtime images and nothing else; rotate it on Finnest's schedule.

  export FINNEST_IMAGE_PULL_USERNAME="<github username Finnest provided>"
  export FINNEST_IMAGE_PULL_TOKEN="<token Finnest provided>"

• If you have direct registry access (a Finnest team member), use your own GitHub token with read:packages:

  gh auth refresh -h github.com -s read:packages
  export FINNEST_IMAGE_PULL_USERNAME="$(gh api user --jq .login)"
  export FINNEST_IMAGE_PULL_TOKEN="$(gh auth token)"

• If you mirror the images into your own registry, configure FINNEST_RUNTIME_IMAGE_REGISTRY / FINNEST_RUNTIME_IMAGE_OWNER for that registry and leave the Finnest GHCR credential unset.

Runtime image version

By default you do not choose a runtime version. finnest deploy resolves the latest published Finnest runtime and pins that exact version into your stack, and finnest doctor pull-tests it before apply — so a fresh install never lands on a missing image.

• Pin a specific version (for reproducible deploys): set FINNEST_RUNTIME_IMAGE_TAG to a published version.

  export FINNEST_RUNTIME_IMAGE_TAG="0.2.0"

• Pin one service (advanced — roll a single service independently): set FINNEST_RUNTIME_IMAGE_TAG_<SERVICE>, or FINNEST_RUNTIME_IMAGE_DIGEST_<SERVICE> to pin by digest — for example FINNEST_RUNTIME_IMAGE_TAG_BANKING. Every service you do not pin uses the version above, keeping the deployed set consistent.

If the registry cannot be reached while resolving, the deploy does not fall back to the version that ships with your CLI. It resolves a fail-closed unresolved-runtime-image-tag sentinel instead, and finnest doctor's runtime-image check pull-tests that sentinel and flags it as unpublished — surfacing the real published version to set when the registry is reachable enough to report one.

AWS Requirements

Before running finnest deploy --cloud=aws, confirm:

• The AWS CLI session, profile, or SSO login is active for the target account.

• For AWS SSO/profile deployments, select the profile before running preview or apply:

  export AWS_PROFILE="<sso profile>"
  aws sts get-caller-identity

• The selected region matches the environment plan.

• The principal can manage EKS, VPC, IAM, KMS, S3, DNS, load balancers, and Kubernetes runtime resources.

• The DNS zone is delegated and can route the API, admin, and auth hostnames.

• The Pulumi state backend can be created or reused by the CLI bootstrap step.

• AWS SSO or profile authentication has been refreshed on the machine that will run the CLI. If SSO expires, refresh it before rerunning deploy or smoke.

Azure Requirements

Before running finnest deploy --cloud=azure, confirm:

• az login is complete and the active subscription is the intended target:

  az account show --query '{subscription:id, tenant:tenantId}' -o table

• ARM_SUBSCRIPTION_ID is set when the deploy runner should use a specific subscription:

  export ARM_SUBSCRIPTION_ID="$(az account show --query id -o tsv)"

• finnest doctor --all and finnest deploy --preview can read the active Azure subscription roles. The active login needs either Owner, or Contributor plus User Access Administrator, on the target subscription. If this check fails, ask your Azure subscription admin to grant one of those role combinations before deploying.

• The Entra tenant and subscription allow application registration, service principal, federated credential, group membership, and RBAC updates.

• When Finnest creates the default AKS admin group, the active login also needs Microsoft Entra permissions to manage groups and applications. Use Groups Administrator plus Application Administrator, or have your Entra admin pre-create the AKS admin group and provide its object id through FINNEST_AZURE_AKS_ADMIN_GROUP_OBJECT_ID.

• The principal can manage resource groups, AKS, networking, Key Vault, Blob storage, DNS, managed identities, and Kubernetes runtime resources.

• The AKS admin group and runtime identity model are approved for the target environment. If no existing group id is provided, Finnest creates a security-enabled Entra admin group and adds the current Azure CLI principal as a member during the Azure platform apply.

• Optional: set FINNEST_AZURE_AKS_ADMIN_GROUP_OBJECT_ID to reuse an existing Entra group that should receive AKS cluster-admin access:

  export FINNEST_AZURE_AKS_ADMIN_GROUP_OBJECT_ID="<entra group object id>"

• The Pulumi state backend can be created or reused by the CLI bootstrap step.

• The target AKS kubeconfig can authenticate through Azure CLI or kubelogin when running finnest smoke, finnest logs, or manual Kubernetes checks.

If AKS access prompts for repeated device-code login, convert the kubeconfig to Azure CLI auth before rerunning smoke or diagnostics:

KUBECONFIG=<path> kubelogin convert-kubeconfig -l azurecli

Golden Path

Run these commands from the operator workstation or CI runner that owns the cloud credentials:

finnest doctor --cloud=<aws|azure> --environment=<sandbox|prod>
finnest deploy --cloud=<aws|azure> --environment=<sandbox|prod> --preview
finnest deploy --cloud=<aws|azure> --environment=<sandbox|prod> --apply
finnest smoke --cloud=<aws|azure> --environment=<sandbox|prod>

For automation, use --json and collect the evidence directory printed in the command envelope.

For live deploy monitoring in automation, prefer NDJSON output:

finnest deploy --cloud=<aws|azure> --environment=<sandbox|prod> --apply --output=ndjson

Each line is a JSON object. The stream ends with the normal versioned envelope, so automation can both follow phase events and parse the final result.

Diagnostics and Logs

Every deploy run writes local evidence under:

.finnest/runs/deploy/<run-id>/

Read files in this order:

1. envelope.json - final command status, exit code, target, phase summary, error code, docs link, and next steps.
2. events.ndjson - workflow/lifecycle stream for the command. For standalone smoke this includes smoke.failed with the completed check list.
3. deploy-events.ndjson - deploy phase events such as preflight, apply summary, Helm, and smoke phase status.
4. raw/pulumi-bootstrap.log and raw/pulumi-bootstrap.ndjson - state bootstrap output.
5. raw/pulumi-deploy.log and raw/pulumi-deploy.ndjson - platform and runtime output.

Keep the whole evidence directory for support. It is redacted at write time and is the fastest way to know where the run stopped.

Stop Point Map

Use the envelope error and deploy events to identify the failing layer:

• doctor or preflight: local CLI, cloud login, permissions, DNS, runtime bundle, or image-pull prerequisites are missing.
• bootstrap: state backend or cloud identity setup failed.
• platform: cloud infrastructure failed, such as networking, cluster, identity, DNS, or load balancer resources.
• runtime or helm: Kubernetes runtime, operators, secrets, chart values, or image pulls failed.
• smoke: infrastructure is up, but a product endpoint, auth flow, or regulatory journey did not pass.

For example, a smoke failure like this means the deploy reached the runtime and then failed the OPIN product probe:

{
  "code": "AUTH_CERT_BINDING_REQUIRED",
  "detail": "x-client-cert header required"
}

That is not an AKS or EKS provisioning failure. It means the Power admin OPIN probe called the insurance service without the certificate header required by the mTLS binding contract.

Recovery After a Failed Deploy

Start with the failed run id from the command output. Do not delete cloud resources or the Pulumi state backend until you know which layer failed.

1. Read the final envelope:

   finnest recover deploy --run=<run-id>
   finnest support bundle --run=<run-id>

2. Identify the stop point:

   • preflight: fix the missing prerequisite, then rerun finnest doctor and finnest deploy --preview.
   • bootstrap: preserve the state backend evidence. If the error says a resource or identity already exists, this is state drift; do not delete the state resource group unless the goal is a full stop-spend cleanup.
   • platform: rerun finnest deploy --preview after fixing the reported cloud prerequisite. If preview still collides on existing cloud resources, collect a support bundle.
   • runtime or helm: fix values, secrets, image pull access, or runtime operator readiness, then rerun finnest deploy --apply.
   • smoke: infrastructure may already be live. Fix the product/auth failure and rerun finnest smoke before applying again.

3. Use rollback only for runtime revisions:

   finnest rollback --cloud=<aws|azure> --environment=<sandbox|prod>

   Rollback restores a prior successful runtime revision. It does not repair a failed state bootstrap, missing cloud prerequisite, or orphaned resource group.

4. For support, keep the full evidence directory and the support bundle. The bundle includes redacted run evidence and local diagnostics needed to decide whether the safe next path is resume, rollback, state adoption, or cleanup.

Use the recovery advisor before retrying or deleting resources:

finnest recover deploy --run=<run-id>
finnest support bundle --run=<run-id>

The recovery advisor classifies a failed deploy and prints the next safe action without mutating cloud resources. --adopt --preview is also read-only. Mutating recovery actions such as adoption apply, resume, rollback, or stop-spend cleanup fail closed until their provider-specific proof gates are implemented.

For a smoke-only failure, the advisor can also read the smoke evidence run:

finnest recover deploy --run=<smoke-run-id>

If it classifies the failure as product-auth, do not roll back infrastructure by default. Fix the product/auth input and rerun finnest smoke.

Deployed Links

After a successful run, verify the target with the domain chosen for the environment:

https://api.<domain>/insurance/healthz
https://api.<domain>/power/admin/opin/probes
https://admin.<domain>/finnest-admins
https://auth.<domain>/realms/finnest-opin/.well-known/openid-configuration

For the June 6, 2026 Azure sandbox deploy evidence, the runtime reached these public endpoints:

https://api.azure-sandbox.power.finnest.com.br/insurance/healthz
https://api.azure-sandbox.power.finnest.com.br/power/admin/opin/probes
https://admin.azure-sandbox.power.finnest.com.br/finnest-admins
https://auth.azure-sandbox.power.finnest.com.br/realms/finnest-opin/.well-known/openid-configuration

Current Azure evidence from June 6, 2026:

• Deploy rerun applied no infrastructure changes, passed Helm, and stopped at product-auth smoke: .finnest/runs/deploy/2026-06-06T21-54-46-365Z-azure-sandbox/.
• Standalone smoke evidence: .finnest/runs/smoke/2026-06-06T21-46-03-577Z-azure-sandbox/.
• Support bundle: .finnest/runs/support/2026-06-06T21-56-48-842Z/support-bundle.json.
• finnest recover deploy --run=deploy-2026-06-06T21-54-46-365Z classified the stop as product-auth, with high confidence.

The final smoke failed on OPIN certificate binding:

opin-admin-probes-run smoke check failed with status 401:
AUTH_CERT_BINDING_REQUIRED: x-client-cert header required

Treat the public links as live infrastructure evidence, not a passed regulatory deploy.

Current DX Gaps to Improve

These are product improvements for the CLI experience:

• finnest doctor should verify GHCR read:packages access before apply.
• Deploy errors should print the exact stopped phase and the first relevant diagnostics file.
• A support-bundle command should package envelope.json, events, raw Pulumi logs, Kubernetes summaries, and redaction metadata.
• OPIN smoke should preflight certificate-binding material before a full apply.
• Registry and image-tag overrides should be first-class for ACR/ECR recovery when GHCR publish or pull access is blocked.
• Bootstrap should automatically detect and reuse a prior state backend when it is safe, instead of letting partial local state drift confuse reruns.
• Operators need finnest deploy status and finnest logs to follow the active run without opening raw files manually.
• TTY snapshots and non-TTY JSON assertions should stay part of deploy DX regression testing. The current branch uses the repo-local .agents/skills/cli-inspector/scripts/tui-inspect.ts harness for this.