> ## Documentation Index > Fetch the complete documentation index at: https://docs.stateset.com/llms.txt > Use this file to discover all available pages before exploring further. # Sandbox Deployment: Azure > Deploying the StateSet Sandbox on Azure. # Azure Deployment Guide (AKS) Complete guide for deploying StateSet Sandbox on Azure Kubernetes Service (AKS). ## Prerequisites * Azure CLI configured with appropriate permissions * kubectl installed * Helm 3.x installed * Domain name for API endpoint * Azure subscription with required resource providers enabled ## Quick Start (Kustomize Overlay) Use the Azure-specific overlay to apply the correct ingress/controller defaults: ```bash theme={null} kubectl apply -k k8s/overlays/azure # or ./scripts/deploy-k8s.sh azure # or (CLI) stateset deploy azure stateset deploy azure --preflight --wait --smoke ``` For smoke checks, export credentials before running deploy: ```bash theme={null} export STATESET_API_KEY="your_api_key" export STATESET_API_URL="https://api.sandbox.stateset.com/api/v1" ``` Before applying the overlay, provision the required controller auth secret: ```bash theme={null} read -rsp "Admin password: " ADMIN_PASSWORD; echo bash scripts/create-controller-secret.sh \ --admin-username admin \ --admin-password "$ADMIN_PASSWORD" unset ADMIN_PASSWORD ``` Before applying, update the Azure SAS token secret: ```bash theme={null} # Edit the placeholder SAS token vi k8s/overlays/azure/secret.yaml ``` ## Architecture Overview ``` ┌─────────────────────────────────────────────────────────────────────────────────┐ │ Microsoft Azure │ │ ┌───────────────────────────────────────────────────────────────────────────┐ │ │ │ VNet (10.0.0.0/16) │ │ │ │ ┌────────────────────┐ ┌────────────────────┐ ┌────────────────────┐ │ │ │ │ │ Subnet: AKS │ │ Subnet: Database │ │ Subnet: Redis │ │ │ │ │ │ 10.0.0.0/20 │ │ 10.0.16.0/24 │ │ 10.0.17.0/24 │ │ │ │ │ │ ┌──────────────┐ │ │ ┌──────────────┐ │ │ ┌──────────────┐ │ │ │ │ │ │ │ AKS Cluster │ │ │ │ Azure DB for │ │ │ │ Azure Cache │ │ │ │ │ │ │ │ - System Pool│ │ │ │ PostgreSQL │ │ │ │ for Redis │ │ │ │ │ │ │ │ - Sandbox │ │ │ │ Flex Server │ │ │ │ │ │ │ │ │ │ │ │ Pool │ │ │ └──────────────┘ │ │ └──────────────┘ │ │ │ │ │ │ └──────────────┘ │ │ │ │ │ │ │ │ │ └────────────────────┘ └────────────────────┘ └────────────────────┘ │ │ │ └───────────────────────────────────────────────────────────────────────────┘ │ │ │ │ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ ┌──────────────────────┐ │ │ │ Azure Blob │ │ Key Vault │ │ Application │ │ Container Registry │ │ │ │ Storage │ │ Secrets │ │ Gateway │ │ (ACR) │ │ │ └──────────────┘ └──────────────┘ └──────────────┘ └──────────────────────┘ │ └─────────────────────────────────────────────────────────────────────────────────┘ ``` ## Step 1: Set Up Azure Environment ```bash theme={null} # Login to Azure az login # Set subscription export SUBSCRIPTION_ID="your-subscription-id" az account set --subscription $SUBSCRIPTION_ID # Set variables export RESOURCE_GROUP="stateset-sandbox-rg" export LOCATION="eastus" export AKS_CLUSTER="stateset-sandbox-aks" # Register required providers az provider register --namespace Microsoft.ContainerService az provider register --namespace Microsoft.DBforPostgreSQL az provider register --namespace Microsoft.Cache az provider register --namespace Microsoft.KeyVault az provider register --namespace Microsoft.Storage az provider register --namespace Microsoft.Network # Create resource group az group create --name $RESOURCE_GROUP --location $LOCATION ``` ## Step 2: Create Virtual Network ```bash theme={null} # Create VNet az network vnet create \ --resource-group $RESOURCE_GROUP \ --name stateset-sandbox-vnet \ --address-prefix 10.0.0.0/16 # Create subnet for AKS az network vnet subnet create \ --resource-group $RESOURCE_GROUP \ --vnet-name stateset-sandbox-vnet \ --name aks-subnet \ --address-prefix 10.0.0.0/20 # Create subnet for PostgreSQL az network vnet subnet create \ --resource-group $RESOURCE_GROUP \ --vnet-name stateset-sandbox-vnet \ --name db-subnet \ --address-prefix 10.0.16.0/24 \ --delegations Microsoft.DBforPostgreSQL/flexibleServers # Create subnet for Redis az network vnet subnet create \ --resource-group $RESOURCE_GROUP \ --vnet-name stateset-sandbox-vnet \ --name redis-subnet \ --address-prefix 10.0.17.0/24 # Get subnet IDs AKS_SUBNET_ID=$(az network vnet subnet show \ --resource-group $RESOURCE_GROUP \ --vnet-name stateset-sandbox-vnet \ --name aks-subnet \ --query id -o tsv) DB_SUBNET_ID=$(az network vnet subnet show \ --resource-group $RESOURCE_GROUP \ --vnet-name stateset-sandbox-vnet \ --name db-subnet \ --query id -o tsv) ``` ## Step 3: Create Azure Container Registry ```bash theme={null} # Create ACR az acr create \ --resource-group $RESOURCE_GROUP \ --name statesetsandboxacr \ --sku Premium \ --admin-enabled false # Get ACR ID ACR_ID=$(az acr show --name statesetsandboxacr --query id -o tsv) ``` ## Step 4: Create AKS Cluster ```bash theme={null} # Create managed identity for AKS az identity create \ --resource-group $RESOURCE_GROUP \ --name stateset-sandbox-identity IDENTITY_ID=$(az identity show \ --resource-group $RESOURCE_GROUP \ --name stateset-sandbox-identity \ --query id -o tsv) IDENTITY_CLIENT_ID=$(az identity show \ --resource-group $RESOURCE_GROUP \ --name stateset-sandbox-identity \ --query clientId -o tsv) # Create AKS cluster az aks create \ --resource-group $RESOURCE_GROUP \ --name $AKS_CLUSTER \ --location $LOCATION \ --kubernetes-version 1.29 \ --node-count 3 \ --node-vm-size Standard_D4s_v3 \ --network-plugin azure \ --network-policy azure \ --vnet-subnet-id $AKS_SUBNET_ID \ --service-cidr 10.1.0.0/16 \ --dns-service-ip 10.1.0.10 \ --enable-managed-identity \ --assign-identity $IDENTITY_ID \ --enable-workload-identity \ --enable-oidc-issuer \ --enable-addons monitoring \ --attach-acr statesetsandboxacr \ --nodepool-name systempool \ --nodepool-labels node-type=controller \ --zones 1 2 3 \ --tier standard # Add sandbox node pool with Kata Containers az aks nodepool add \ --resource-group $RESOURCE_GROUP \ --cluster-name $AKS_CLUSTER \ --name sandboxpool \ --node-count 2 \ --node-vm-size Standard_D8s_v3 \ --node-osdisk-size 100 \ --workload-runtime KataMshvVmIsolation \ --os-sku AzureLinux \ --labels node-type=sandbox isolation=kata \ --node-taints stateset.com/sandbox=true:NoSchedule \ --min-count 0 \ --max-count 20 \ --enable-cluster-autoscaler \ --zones 1 2 3 # Get credentials az aks get-credentials --resource-group $RESOURCE_GROUP --name $AKS_CLUSTER ``` ### Alternative: gVisor Node Pool If you prefer gVisor over Kata: ```bash theme={null} # Add gVisor node pool (requires custom VM image) az aks nodepool add \ --resource-group $RESOURCE_GROUP \ --cluster-name $AKS_CLUSTER \ --name gvisorpool \ --node-count 2 \ --node-vm-size Standard_D8s_v3 \ --labels node-type=sandbox isolation=gvisor \ --node-taints stateset.com/sandbox=true:NoSchedule \ --min-count 0 \ --max-count 20 \ --enable-cluster-autoscaler ``` ## Step 5: Create Azure Database for PostgreSQL ```bash theme={null} # Create private DNS zone az network private-dns zone create \ --resource-group $RESOURCE_GROUP \ --name privatelink.postgres.database.azure.com # Link DNS zone to VNet az network private-dns link vnet create \ --resource-group $RESOURCE_GROUP \ --zone-name privatelink.postgres.database.azure.com \ --name stateset-sandbox-link \ --virtual-network stateset-sandbox-vnet \ --registration-enabled false # Create PostgreSQL Flexible Server az postgres flexible-server create \ --resource-group $RESOURCE_GROUP \ --name stateset-sandbox-pg \ --location $LOCATION \ --admin-user pgadmin \ --admin-password "YOUR_STRONG_PASSWORD" \ --sku-name Standard_D4s_v3 \ --tier GeneralPurpose \ --version 15 \ --storage-size 128 \ --subnet $DB_SUBNET_ID \ --private-dns-zone privatelink.postgres.database.azure.com \ --high-availability ZoneRedundant \ --backup-retention 7 # Create database az postgres flexible-server db create \ --resource-group $RESOURCE_GROUP \ --server-name stateset-sandbox-pg \ --database-name sandbox # Get connection string PG_HOST=$(az postgres flexible-server show \ --resource-group $RESOURCE_GROUP \ --name stateset-sandbox-pg \ --query fullyQualifiedDomainName -o tsv) ``` ## Step 6: Create Azure Cache for Redis ```bash theme={null} # Create Redis cache az redis create \ --resource-group $RESOURCE_GROUP \ --name stateset-sandbox-redis \ --location $LOCATION \ --sku Premium \ --vm-size P1 \ --subnet-id /subscriptions/$SUBSCRIPTION_ID/resourceGroups/$RESOURCE_GROUP/providers/Microsoft.Network/virtualNetworks/stateset-sandbox-vnet/subnets/redis-subnet \ --enable-non-ssl-port false \ --minimum-tls-version 1.2 \ --zones 1 2 3 # Get Redis connection info REDIS_HOST=$(az redis show \ --resource-group $RESOURCE_GROUP \ --name stateset-sandbox-redis \ --query hostName -o tsv) REDIS_KEY=$(az redis list-keys \ --resource-group $RESOURCE_GROUP \ --name stateset-sandbox-redis \ --query primaryKey -o tsv) ``` ## Step 7: Create Azure Blob Storage ```bash theme={null} # Create storage account az storage account create \ --resource-group $RESOURCE_GROUP \ --name statesetsandboxstorage \ --location $LOCATION \ --sku Standard_ZRS \ --kind StorageV2 \ --min-tls-version TLS1_2 \ --allow-blob-public-access false \ --https-only true # Create container for artifacts az storage container create \ --account-name statesetsandboxstorage \ --name artifacts \ --auth-mode login # Enable versioning az storage account blob-service-properties update \ --resource-group $RESOURCE_GROUP \ --account-name statesetsandboxstorage \ --enable-versioning true # Set lifecycle management az storage account management-policy create \ --resource-group $RESOURCE_GROUP \ --account-name statesetsandboxstorage \ --policy '{ "rules": [{ "name": "deleteOldArtifacts", "enabled": true, "type": "Lifecycle", "definition": { "filters": {"blobTypes": ["blockBlob"], "prefixMatch": ["artifacts/"]}, "actions": { "baseBlob": {"delete": {"daysAfterModificationGreaterThan": 90}}, "version": {"delete": {"daysAfterCreationGreaterThan": 30}} } } }] }' ``` ## Step 8: Create Azure Key Vault ```bash theme={null} # Create Key Vault az keyvault create \ --resource-group $RESOURCE_GROUP \ --name stateset-sandbox-kv \ --location $LOCATION \ --sku premium \ --enable-rbac-authorization # Get Key Vault ID KEYVAULT_ID=$(az keyvault show --name stateset-sandbox-kv --query id -o tsv) # Add secrets az keyvault secret set \ --vault-name stateset-sandbox-kv \ --name jwt-secret \ --value "$(openssl rand -base64 32)" az keyvault secret set \ --vault-name stateset-sandbox-kv \ --name database-encryption-key \ --value "$(openssl rand -hex 32)" az keyvault secret set \ --vault-name stateset-sandbox-kv \ --name stripe-secret-key \ --value "your-stripe-secret-key" az keyvault secret set \ --vault-name stateset-sandbox-kv \ --name redis-password \ --value "$REDIS_KEY" # Grant AKS identity access to Key Vault az role assignment create \ --role "Key Vault Secrets User" \ --assignee $IDENTITY_CLIENT_ID \ --scope $KEYVAULT_ID ``` ## Step 9: Configure Workload Identity ```bash theme={null} # Get OIDC issuer URL OIDC_ISSUER=$(az aks show \ --resource-group $RESOURCE_GROUP \ --name $AKS_CLUSTER \ --query oidcIssuerProfile.issuerUrl -o tsv) # Create user-assigned identity for the controller az identity create \ --resource-group $RESOURCE_GROUP \ --name sandbox-controller-identity CONTROLLER_IDENTITY_CLIENT_ID=$(az identity show \ --resource-group $RESOURCE_GROUP \ --name sandbox-controller-identity \ --query clientId -o tsv) # Create federated credential az identity federated-credential create \ --name sandbox-controller-fed \ --identity-name sandbox-controller-identity \ --resource-group $RESOURCE_GROUP \ --issuer $OIDC_ISSUER \ --subject system:serviceaccount:stateset-sandbox:sandbox-controller # Grant permissions az role assignment create \ --role "Key Vault Secrets User" \ --assignee $CONTROLLER_IDENTITY_CLIENT_ID \ --scope $KEYVAULT_ID az role assignment create \ --role "Storage Blob Data Contributor" \ --assignee $CONTROLLER_IDENTITY_CLIENT_ID \ --scope /subscriptions/$SUBSCRIPTION_ID/resourceGroups/$RESOURCE_GROUP/providers/Microsoft.Storage/storageAccounts/statesetsandboxstorage ``` ## Step 10: Deploy StateSet Sandbox ### Create Namespace ```bash theme={null} kubectl apply -f k8s/namespace.yaml ``` ### Configure Service Account with Workload Identity ```yaml theme={null} # azure-service-account.yaml apiVersion: v1 kind: ServiceAccount metadata: name: sandbox-controller namespace: stateset-sandbox annotations: azure.workload.identity/client-id: "YOUR_CONTROLLER_IDENTITY_CLIENT_ID" labels: azure.workload.identity/use: "true" ``` ### Install CSI Secrets Store Driver ```bash theme={null} # Install Secrets Store CSI driver helm repo add csi-secrets-store-provider-azure https://azure.github.io/secrets-store-csi-driver-provider-azure/charts helm install csi-secrets-store-provider-azure csi-secrets-store-provider-azure/csi-secrets-store-provider-azure \ --namespace kube-system ``` ### Create SecretProviderClass ```yaml theme={null} # azure-secret-provider.yaml apiVersion: secrets-store.csi.x-k8s.io/v1 kind: SecretProviderClass metadata: name: azure-keyvault-secrets namespace: stateset-sandbox spec: provider: azure parameters: usePodIdentity: "false" useVMManagedIdentity: "false" clientID: "YOUR_CONTROLLER_IDENTITY_CLIENT_ID" keyvaultName: "stateset-sandbox-kv" tenantId: "YOUR_TENANT_ID" objects: | array: - | objectName: jwt-secret objectType: secret - | objectName: database-encryption-key objectType: secret - | objectName: stripe-secret-key objectType: secret - | objectName: redis-password objectType: secret secretObjects: - secretName: sandbox-controller-auth type: Opaque data: - objectName: jwt-secret key: jwt-secret - objectName: database-encryption-key key: database-encryption-key - objectName: stripe-secret-key key: stripe-secret-key - objectName: redis-password key: redis-password ``` ### Create ConfigMap ```yaml theme={null} # azure-configmap.yaml apiVersion: v1 kind: ConfigMap metadata: name: sandbox-controller-config namespace: stateset-sandbox data: # Azure-specific configuration SANDBOX_IMAGE: "statesetsandboxacr.azurecr.io/stateset-sandbox:latest" # Database (Azure PostgreSQL) DATABASE_URL: "postgres://pgadmin:PASSWORD@stateset-sandbox-pg.postgres.database.azure.com:5432/sandbox?sslmode=require" # Redis (Azure Cache) REDIS_HOST: "stateset-sandbox-redis.redis.cache.windows.net" REDIS_PORT: "6380" REDIS_TLS: "true" # Storage (Azure Blob) STORAGE_PROVIDER: "azure" STORAGE_BUCKET: "artifacts" AZURE_STORAGE_ACCOUNT: "statesetsandboxstorage" # AZURE_STORAGE_SAS_TOKEN is injected via secret in the Azure kustomize overlay. # AZURE_STORAGE_ENDPOINT: "https://statesetsandboxstorage.blob.core.windows.net" # Isolation (Kata Containers on AKS) DEFAULT_ISOLATION: "kata" SANDBOX_RUNTIME_CLASS: "kata-mshv-vm-isolation" # Warm pool WARM_POOL_ENABLED: "true" WARM_POOL_SIZE: "5" # CORS CORS_ORIGIN: "https://sandbox.yourdomain.com" APP_URL: "https://api.sandbox.yourdomain.com" # Logging LOG_LEVEL: "info" NODE_ENV: "production" ``` ### Update Deployment for Azure ```yaml theme={null} # azure-deployment-patch.yaml spec: template: metadata: labels: azure.workload.identity/use: "true" spec: serviceAccountName: sandbox-controller containers: - name: sandbox-controller volumeMounts: - name: secrets-store mountPath: "/mnt/secrets-store" readOnly: true volumes: - name: secrets-store csi: driver: secrets-store.csi.k8s.io readOnly: true volumeAttributes: secretProviderClass: azure-keyvault-secrets ``` ### Deploy Components ```bash theme={null} kubectl apply -f azure-service-account.yaml kubectl apply -f azure-secret-provider.yaml kubectl apply -f azure-configmap.yaml kubectl apply -f k8s/rbac.yaml kubectl apply -f k8s/deployment.yaml kubectl apply -f k8s/service.yaml kubectl apply -f k8s/network-policy.yaml kubectl apply -f k8s/resource-quota.yaml ``` ### Create Kata RuntimeClass ```yaml theme={null} # kata-runtimeclass.yaml apiVersion: node.k8s.io/v1 kind: RuntimeClass metadata: name: kata-mshv-vm-isolation handler: kata-mshv-vm-isolation scheduling: nodeSelector: kubernetes.azure.com/kata-mshv-vm-isolation: "true" tolerations: - key: stateset.com/sandbox operator: Exists effect: NoSchedule ``` ## Step 11: Configure Application Gateway Ingress ### Option A: Application Gateway Ingress Controller (AGIC) ```bash theme={null} # Enable AGIC addon az aks addon update \ --resource-group $RESOURCE_GROUP \ --name $AKS_CLUSTER \ --addon ingress-appgw \ --appgw-name stateset-sandbox-agw \ --appgw-subnet-cidr 10.0.20.0/24 ``` ```yaml theme={null} # azure-ingress-agic.yaml apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: sandbox-controller namespace: stateset-sandbox annotations: kubernetes.io/ingress.class: azure/application-gateway appgw.ingress.kubernetes.io/ssl-redirect: "true" appgw.ingress.kubernetes.io/backend-protocol: "http" appgw.ingress.kubernetes.io/health-probe-path: "/health" appgw.ingress.kubernetes.io/request-timeout: "300" # WAF policy (optional) # appgw.ingress.kubernetes.io/waf-policy-for-path: "/subscriptions/.../applicationGatewayWebApplicationFirewallPolicies/sandbox-waf" spec: tls: - hosts: - api.sandbox.yourdomain.com secretName: sandbox-tls-secret rules: - host: api.sandbox.yourdomain.com http: paths: - path: / pathType: Prefix backend: service: name: sandbox-controller port: number: 80 ``` ### Option B: NGINX Ingress Controller ```bash theme={null} # Install NGINX ingress controller helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx helm install ingress-nginx ingress-nginx/ingress-nginx \ --namespace ingress-nginx \ --create-namespace \ --set controller.service.annotations."service\.beta\.kubernetes\.io/azure-load-balancer-health-probe-request-path"=/healthz ``` ### Configure TLS Certificate ```bash theme={null} # Install cert-manager helm repo add jetstack https://charts.jetstack.io helm install cert-manager jetstack/cert-manager \ --namespace cert-manager \ --create-namespace \ --set installCRDs=true # Create ClusterIssuer cat <=5" \ --window-size 5m \ --evaluation-frequency 1m \ --action sandbox-alerts \ --description "High error rate in StateSet Sandbox" ``` ### Prometheus & Grafana (Optional) ```bash theme={null} # Install Prometheus stack helm repo add prometheus-community https://prometheus-community.github.io/helm-charts helm install prometheus prometheus-community/kube-prometheus-stack \ --namespace monitoring \ --create-namespace \ --set prometheus.prometheusSpec.serviceMonitorSelectorNilUsesHelmValues=false # Apply ServiceMonitor kubectl apply -f k8s/prometheus/service-monitor.yaml kubectl apply -f k8s/prometheus/alerts.yaml ``` ## Step 13: Configure Autoscaling ### Cluster Autoscaler Already enabled on sandbox node pool. Configure behavior: ```bash theme={null} # Update autoscaler profile az aks update \ --resource-group $RESOURCE_GROUP \ --name $AKS_CLUSTER \ --cluster-autoscaler-profile \ scale-down-delay-after-add=10m \ scale-down-unneeded-time=10m \ max-graceful-termination-sec=600 ``` ### Horizontal Pod Autoscaler ```yaml theme={null} # hpa.yaml apiVersion: autoscaling/v2 kind: HorizontalPodAutoscaler metadata: name: sandbox-controller namespace: stateset-sandbox spec: scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: sandbox-controller minReplicas: 3 maxReplicas: 10 metrics: - type: Resource resource: name: cpu target: type: Utilization averageUtilization: 70 - type: Resource resource: name: memory target: type: Utilization averageUtilization: 80 ``` ### KEDA (Event-Driven Autoscaling) ```bash theme={null} # Install KEDA helm repo add kedacore https://kedacore.github.io/charts helm install keda kedacore/keda --namespace keda --create-namespace ``` ```yaml theme={null} # keda-scaledobject.yaml apiVersion: keda.sh/v1alpha1 kind: ScaledObject metadata: name: sandbox-controller-scaler namespace: stateset-sandbox spec: scaleTargetRef: name: sandbox-controller minReplicaCount: 3 maxReplicaCount: 20 triggers: - type: prometheus metadata: serverAddress: http://prometheus-server.monitoring.svc.cluster.local metricName: stateset_sandbox_active_sandboxes query: stateset_sandbox_active_sandboxes threshold: "50" ``` ## Security Hardening ### Azure Policy for AKS ```bash theme={null} # Enable Azure Policy addon az aks enable-addons \ --resource-group $RESOURCE_GROUP \ --name $AKS_CLUSTER \ --addons azure-policy # Assign built-in policies az policy assignment create \ --name "deny-privileged-containers" \ --policy "95edb821-ddaf-4404-9732-666045e056b4" \ --scope "/subscriptions/$SUBSCRIPTION_ID/resourceGroups/$RESOURCE_GROUP/providers/Microsoft.ContainerService/managedClusters/$AKS_CLUSTER" ``` ### Microsoft Defender for Containers ```bash theme={null} # Enable Defender for Containers az security pricing create \ --name Containers \ --tier Standard # Enable Defender for AKS cluster az aks update \ --resource-group $RESOURCE_GROUP \ --name $AKS_CLUSTER \ --enable-defender ``` ### Private Endpoints ```bash theme={null} # Create private endpoint for PostgreSQL az network private-endpoint create \ --resource-group $RESOURCE_GROUP \ --name pg-private-endpoint \ --vnet-name stateset-sandbox-vnet \ --subnet aks-subnet \ --private-connection-resource-id $(az postgres flexible-server show --resource-group $RESOURCE_GROUP --name stateset-sandbox-pg --query id -o tsv) \ --group-id postgresqlServer \ --connection-name pg-connection # Create private endpoint for Storage az network private-endpoint create \ --resource-group $RESOURCE_GROUP \ --name storage-private-endpoint \ --vnet-name stateset-sandbox-vnet \ --subnet aks-subnet \ --private-connection-resource-id $(az storage account show --resource-group $RESOURCE_GROUP --name statesetsandboxstorage --query id -o tsv) \ --group-id blob \ --connection-name storage-connection ``` ## Confidential Computing (Optional) For maximum isolation with Confidential VMs: ```bash theme={null} # Add confidential computing node pool az aks nodepool add \ --resource-group $RESOURCE_GROUP \ --cluster-name $AKS_CLUSTER \ --name confpool \ --node-count 2 \ --node-vm-size Standard_DC8s_v3 \ --os-sku AzureLinux \ --labels node-type=sandbox isolation=confidential \ --node-taints stateset.com/sandbox=true:NoSchedule \ --min-count 0 \ --max-count 10 \ --enable-cluster-autoscaler \ --enable-sgxquotehelper ``` ## Cost Optimization ### Azure Reserved Instances Purchase reserved capacity for predictable workloads: ```bash theme={null} az reservations reservation-order purchase \ --reservation-order-id "00000000-0000-0000-0000-000000000000" \ --sku Standard_D4s_v3 \ --location eastus \ --billing-scope-id "/subscriptions/$SUBSCRIPTION_ID" \ --term P1Y \ --billing-plan Monthly \ --quantity 3 ``` ### Spot VMs for Sandbox Nodes ```bash theme={null} # Add spot node pool az aks nodepool add \ --resource-group $RESOURCE_GROUP \ --cluster-name $AKS_CLUSTER \ --name spotpool \ --priority Spot \ --eviction-policy Delete \ --spot-max-price -1 \ --node-vm-size Standard_D8s_v3 \ --labels node-type=sandbox lifecycle=spot \ --node-taints stateset.com/sandbox=true:NoSchedule,kubernetes.azure.com/scalesetpriority=spot:NoSchedule \ --min-count 0 \ --max-count 20 \ --enable-cluster-autoscaler ``` ## Estimated Monthly Costs | Resource | Configuration | Est. Cost | | ------------------- | ------------------------------ | ------------------------- | | AKS Management | Free tier | \$0 | | System Node Pool | 3x Standard\_D4s\_v3 | \~\$350 | | Sandbox Node Pool | 2-10x Standard\_D8s\_v3 (Spot) | \~$150-$750 | | Azure PostgreSQL | Standard\_D4s\_v3 HA | \~\$450 | | Azure Cache Redis | Premium P1 | \~\$300 | | Application Gateway | WAF v2 | \~\$280 | | Blob Storage | 100GB + transactions | \~\$25 | | Egress | \~500GB/month | \~\$45 | | Key Vault | Operations | \~\$5 | | **Total** | | **\~$1,600-$2,200/month** | ## Troubleshooting ### Common Issues **Pod identity not working** ```bash theme={null} # Check workload identity configuration kubectl describe serviceaccount sandbox-controller -n stateset-sandbox # Verify federated credential az identity federated-credential show \ --name sandbox-controller-fed \ --identity-name sandbox-controller-identity \ --resource-group $RESOURCE_GROUP ``` **Database connection issues** ```bash theme={null} # Test connection from a pod kubectl run -it --rm debug --image=postgres:15 --restart=Never -- \ psql "postgres://pgadmin:pass@stateset-sandbox-pg.postgres.database.azure.com:5432/sandbox?sslmode=require" ``` **Kata containers not working** ```bash theme={null} # Check node pool configuration az aks nodepool show \ --resource-group $RESOURCE_GROUP \ --cluster-name $AKS_CLUSTER \ --name sandboxpool # Check RuntimeClass kubectl get runtimeclass kata-mshv-vm-isolation -o yaml ``` ### Useful Commands ```bash theme={null} # View AKS diagnostics az aks show \ --resource-group $RESOURCE_GROUP \ --name $AKS_CLUSTER \ --query "agentPoolProfiles[].{name:name, count:count, vmSize:vmSize}" # Stream logs kubectl logs -f -l app=sandbox-controller -n stateset-sandbox # Check node status kubectl get nodes -o wide # View resource usage kubectl top nodes kubectl top pods -n stateset-sandbox ``` ### Support Resources * [AKS Documentation](https://docs.microsoft.com/azure/aks/) * [Azure PostgreSQL Documentation](https://docs.microsoft.com/azure/postgresql/) * [StateSet Sandbox Issues](https://github.com/stateset/sandbox/issues) * [Azure Support](https://azure.microsoft.com/support/)