AWS Deployment Guide (EKS)

Complete guide for deploying StateSet Sandbox on Amazon Elastic Kubernetes Service (EKS).

Prerequisites

AWS CLI configured with appropriate permissions
kubectl installed
eksctl installed (recommended)
Helm 3.x installed
Domain name for API endpoint

Quick Start (Kustomize Overlay)

Use the AWS-specific overlay to apply the correct ingress/controller defaults:

kubectl apply -k k8s/overlays/aws
# or
./scripts/deploy-k8s.sh aws
# or (CLI)
stateset deploy aws
stateset deploy aws --preflight --wait --smoke

For smoke checks, export credentials before running deploy:

export STATESET_API_KEY="your_api_key"
export STATESET_API_URL="https://api.sandbox.stateset.com/api/v1"

Before applying the overlay, provision the required controller auth secret:

read -rsp "Admin password: " ADMIN_PASSWORD; echo
bash scripts/create-controller-secret.sh \
  --admin-username admin \
  --admin-password "$ADMIN_PASSWORD"
unset ADMIN_PASSWORD

Architecture Overview

┌─────────────────────────────────────────────────────────────────────────────────┐
│                              AWS Cloud                                           │
│  ┌───────────────────────────────────────────────────────────────────────────┐  │
│  │                          VPC (10.0.0.0/16)                                 │  │
│  │  ┌────────────────────┐  ┌────────────────────┐  ┌────────────────────┐   │  │
│  │  │  Public Subnet 1   │  │  Public Subnet 2   │  │  Public Subnet 3   │   │  │
│  │  │  10.0.1.0/24       │  │  10.0.2.0/24       │  │  10.0.3.0/24       │   │  │
│  │  │  ┌──────────────┐  │  │  ┌──────────────┐  │  │  ┌──────────────┐  │   │  │
│  │  │  │ NAT Gateway  │  │  │  │ NAT Gateway  │  │  │  │ NAT Gateway  │  │   │  │
│  │  │  └──────────────┘  │  │  └──────────────┘  │  │  └──────────────┘  │   │  │
│  │  └────────────────────┘  └────────────────────┘  └────────────────────┘   │  │
│  │                                                                            │  │
│  │  ┌────────────────────┐  ┌────────────────────┐  ┌────────────────────┐   │  │
│  │  │ Private Subnet 1   │  │ Private Subnet 2   │  │ Private Subnet 3   │   │  │
│  │  │ 10.0.11.0/24       │  │ 10.0.12.0/24       │  │ 10.0.13.0/24       │   │  │
│  │  │ ┌────────────────┐ │  │ ┌────────────────┐ │  │ ┌────────────────┐ │   │  │
│  │  │ │ EKS Node Group │ │  │ │ EKS Node Group │ │  │ │ EKS Node Group │ │   │  │
│  │  │ │  Controller    │ │  │ │  Sandboxes     │ │  │ │  Sandboxes     │ │   │  │
│  │  │ └────────────────┘ │  │ └────────────────┘ │  │ └────────────────┘ │   │  │
│  │  └────────────────────┘  └────────────────────┘  └────────────────────┘   │  │
│  └───────────────────────────────────────────────────────────────────────────┘  │
│                                                                                  │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐  ┌──────────────────────┐ │
│  │  Amazon RDS  │  │  ElastiCache │  │     S3       │  │  AWS Secrets Manager │ │
│  │  PostgreSQL  │  │  Redis       │  │  Artifacts   │  │  API Keys / Secrets  │ │
│  └──────────────┘  └──────────────┘  └──────────────┘  └──────────────────────┘ │
│                                                                                  │
│  ┌──────────────────────────────────────────────────────────────────────────┐   │
│  │                      AWS Load Balancer (ALB)                              │   │
│  │                     api.sandbox.yourdomain.com                            │   │
│  └──────────────────────────────────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────────────────────────────┘

Step 1: Create EKS Cluster

Option A: Using eksctl (Recommended)

Create a cluster configuration file:

# cluster.yaml
apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig

metadata:
  name: stateset-sandbox
  region: us-east-1
  version: "1.29"

vpc:
  cidr: 10.0.0.0/16
  nat:
    gateway: HighlyAvailable

iam:
  withOIDC: true

managedNodeGroups:
  # Controller nodes
  - name: controller
    instanceType: t3.large
    desiredCapacity: 3
    minSize: 2
    maxSize: 5
    volumeSize: 50
    labels:
      node-type: controller
    tags:
      environment: production
    iam:
      withAddonPolicies:
        imageBuilder: true
        autoScaler: true
        ebs: true
        albIngress: true
        cloudWatch: true

  # Sandbox nodes (gVisor-enabled)
  - name: sandbox-gvisor
    instanceType: c5.2xlarge
    desiredCapacity: 3
    minSize: 1
    maxSize: 20
    volumeSize: 100
    labels:
      node-type: sandbox
      isolation: gvisor
    taints:
      - key: stateset.com/sandbox
        value: "true"
        effect: NoSchedule
    preBootstrapCommands:
      # Install gVisor
      - |
        set -o errexit
        set -o nounset
        set -o pipefail
        ARCH=$(uname -m)
        URL="https://storage.googleapis.com/gvisor/releases/release/latest/${ARCH}"
        wget "${URL}/runsc" "${URL}/containerd-shim-runsc-v1" -P /tmp
        chmod a+rx /tmp/runsc /tmp/containerd-shim-runsc-v1
        mv /tmp/runsc /tmp/containerd-shim-runsc-v1 /usr/local/bin/
        cat <<EOF | tee /etc/containerd/config.toml
        [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runsc]
          runtime_type = "io.containerd.runsc.v1"
        EOF
        systemctl restart containerd

addons:
  - name: vpc-cni
    version: latest
  - name: coredns
    version: latest
  - name: kube-proxy
    version: latest
  - name: aws-ebs-csi-driver
    version: latest
    attachPolicyARNs:
      - arn:aws:iam::aws:policy/service-role/AmazonEBSCSIDriverPolicy

cloudWatch:
  clusterLogging:
    enableTypes: ["api", "audit", "authenticator", "controllerManager", "scheduler"]

Create the cluster:

eksctl create cluster -f cluster.yaml

Option B: Using Terraform

See our Terraform module for infrastructure-as-code deployment.

Step 2: Install Required Components

AWS Load Balancer Controller

# Create IAM OIDC provider
eksctl utils associate-iam-oidc-provider \
  --cluster stateset-sandbox \
  --approve

# Create IAM policy
curl -o iam_policy.json https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/v2.7.0/docs/install/iam_policy.json

aws iam create-policy \
  --policy-name AWSLoadBalancerControllerIAMPolicy \
  --policy-document file://iam_policy.json

# Create service account
eksctl create iamserviceaccount \
  --cluster=stateset-sandbox \
  --namespace=kube-system \
  --name=aws-load-balancer-controller \
  --attach-policy-arn=arn:aws:iam::$(aws sts get-caller-identity --query Account --output text):policy/AWSLoadBalancerControllerIAMPolicy \
  --override-existing-serviceaccounts \
  --approve

# Install controller
helm repo add eks https://aws.github.io/eks-charts
helm repo update

helm install aws-load-balancer-controller eks/aws-load-balancer-controller \
  -n kube-system \
  --set clusterName=stateset-sandbox \
  --set serviceAccount.create=false \
  --set serviceAccount.name=aws-load-balancer-controller

External DNS (Optional)

# Create IAM policy
cat <<EOF > external-dns-policy.json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["route53:ChangeResourceRecordSets"],
      "Resource": ["arn:aws:route53:::hostedzone/*"]
    },
    {
      "Effect": "Allow",
      "Action": ["route53:ListHostedZones", "route53:ListResourceRecordSets"],
      "Resource": ["*"]
    }
  ]
}
EOF

aws iam create-policy \
  --policy-name ExternalDNSPolicy \
  --policy-document file://external-dns-policy.json

# Install external-dns
helm repo add external-dns https://kubernetes-sigs.github.io/external-dns/
helm install external-dns external-dns/external-dns \
  --namespace kube-system \
  --set provider=aws \
  --set aws.zoneType=public \
  --set txtOwnerId=stateset-sandbox \
  --set policy=sync

cert-manager

helm repo add jetstack https://charts.jetstack.io
helm repo update

helm install cert-manager jetstack/cert-manager \
  --namespace cert-manager \
  --create-namespace \
  --set installCRDs=true

# Create ClusterIssuer for Let's Encrypt
cat <<EOF | kubectl apply -f -
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-production
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: your-email@example.com
    privateKeySecretRef:
      name: letsencrypt-production
    solvers:
      - http01:
          ingress:
            class: alb
EOF

gVisor RuntimeClass

kubectl apply -f - <<EOF
apiVersion: node.k8s.io/v1
kind: RuntimeClass
metadata:
  name: gvisor
handler: runsc
scheduling:
  nodeSelector:
    isolation: gvisor
  tolerations:
    - key: stateset.com/sandbox
      operator: Exists
      effect: NoSchedule
EOF

Step 3: Create AWS Resources

RDS PostgreSQL

# Create RDS subnet group
aws rds create-db-subnet-group \
  --db-subnet-group-name stateset-sandbox \
  --db-subnet-group-description "StateSet Sandbox DB subnet group" \
  --subnet-ids subnet-xxx subnet-yyy subnet-zzz

# Create RDS instance
aws rds create-db-instance \
  --db-instance-identifier stateset-sandbox \
  --db-instance-class db.r6g.large \
  --engine postgres \
  --engine-version 15.4 \
  --master-username postgres \
  --master-user-password "YOUR_STRONG_PASSWORD" \
  --allocated-storage 100 \
  --storage-type gp3 \
  --storage-encrypted \
  --db-subnet-group-name stateset-sandbox \
  --vpc-security-group-ids sg-xxx \
  --multi-az \
  --backup-retention-period 7 \
  --deletion-protection

ElastiCache Redis

# Create ElastiCache subnet group
aws elasticache create-cache-subnet-group \
  --cache-subnet-group-name stateset-sandbox \
  --cache-subnet-group-description "StateSet Sandbox Redis subnet group" \
  --subnet-ids subnet-xxx subnet-yyy subnet-zzz

# Create Redis cluster
aws elasticache create-replication-group \
  --replication-group-id stateset-sandbox \
  --replication-group-description "StateSet Sandbox Redis" \
  --engine redis \
  --engine-version 7.0 \
  --cache-node-type cache.r6g.large \
  --num-node-groups 1 \
  --replicas-per-node-group 2 \
  --cache-subnet-group-name stateset-sandbox \
  --security-group-ids sg-xxx \
  --at-rest-encryption-enabled \
  --transit-encryption-enabled \
  --automatic-failover-enabled

S3 Bucket for Artifacts

# Create bucket
aws s3 mb s3://stateset-sandbox-artifacts-YOUR_ACCOUNT_ID --region us-east-1

# Enable versioning
aws s3api put-bucket-versioning \
  --bucket stateset-sandbox-artifacts-YOUR_ACCOUNT_ID \
  --versioning-configuration Status=Enabled

# Enable encryption
aws s3api put-bucket-encryption \
  --bucket stateset-sandbox-artifacts-YOUR_ACCOUNT_ID \
  --server-side-encryption-configuration '{
    "Rules": [{
      "ApplyServerSideEncryptionByDefault": {
        "SSEAlgorithm": "aws:kms"
      }
    }]
  }'

# Create lifecycle rule for artifact expiration
aws s3api put-bucket-lifecycle-configuration \
  --bucket stateset-sandbox-artifacts-YOUR_ACCOUNT_ID \
  --lifecycle-configuration '{
    "Rules": [{
      "ID": "ExpireOldArtifacts",
      "Status": "Enabled",
      "Filter": {"Prefix": "artifacts/"},
      "Expiration": {"Days": 90},
      "NoncurrentVersionExpiration": {"NoncurrentDays": 30}
    }]
  }'

Secrets Manager

# Create secrets
aws secretsmanager create-secret \
  --name stateset-sandbox/jwt-secret \
  --secret-string "$(openssl rand -base64 32)"

aws secretsmanager create-secret \
  --name stateset-sandbox/database-encryption-key \
  --secret-string "$(openssl rand -hex 32)"

aws secretsmanager create-secret \
  --name stateset-sandbox/stripe-secret-key \
  --secret-string "your-stripe-secret-key"

# Create IAM policy for secrets access
cat <<EOF > secrets-policy.json
{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Action": ["secretsmanager:GetSecretValue"],
    "Resource": "arn:aws:secretsmanager:us-east-1:*:secret:stateset-sandbox/*"
  }]
}
EOF

aws iam create-policy \
  --policy-name StateSetSandboxSecretsPolicy \
  --policy-document file://secrets-policy.json

Step 4: Install External Secrets Operator

helm repo add external-secrets https://charts.external-secrets.io
helm install external-secrets external-secrets/external-secrets \
  --namespace external-secrets \
  --create-namespace

# Create SecretStore
cat <<EOF | kubectl apply -f -
apiVersion: external-secrets.io/v1beta1
kind: SecretStore
metadata:
  name: aws-secrets-manager
  namespace: stateset-sandbox
spec:
  provider:
    aws:
      service: SecretsManager
      region: us-east-1
      auth:
        jwt:
          serviceAccountRef:
            name: sandbox-controller
EOF

# Create ExternalSecret
cat <<EOF | kubectl apply -f -
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: sandbox-controller-secrets
  namespace: stateset-sandbox
spec:
  refreshInterval: 1h
  secretStoreRef:
    name: aws-secrets-manager
    kind: SecretStore
  target:
    name: sandbox-controller-auth
  data:
    - secretKey: jwt-secret
      remoteRef:
        key: stateset-sandbox/jwt-secret
    - secretKey: database-encryption-key
      remoteRef:
        key: stateset-sandbox/database-encryption-key
    - secretKey: stripe-secret-key
      remoteRef:
        key: stateset-sandbox/stripe-secret-key
EOF

Step 5: Deploy StateSet Sandbox

Create Namespace

kubectl apply -f k8s/namespace.yaml

Create ConfigMap

# aws-configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: sandbox-controller-config
  namespace: stateset-sandbox
data:
  # AWS-specific configuration
  SANDBOX_IMAGE: "YOUR_ACCOUNT_ID.dkr.ecr.us-east-1.amazonaws.com/stateset-sandbox:latest"

  # Database (RDS endpoint)
  DATABASE_URL: "postgres://postgres:PASSWORD@stateset-sandbox.xxx.us-east-1.rds.amazonaws.com:5432/sandbox?sslmode=require"

  # Redis (ElastiCache endpoint)
  REDIS_HOST: "stateset-sandbox.xxx.use1.cache.amazonaws.com"
  REDIS_PORT: "6379"
  REDIS_TLS: "true"

  # Storage (S3)
  STORAGE_PROVIDER: "s3"
  STORAGE_BUCKET: "stateset-sandbox-artifacts-YOUR_ACCOUNT_ID"
  STORAGE_REGION: "us-east-1"

  # Isolation
  DEFAULT_ISOLATION: "gvisor"
  SANDBOX_RUNTIME_CLASS: "gvisor"

  # Warm pool
  WARM_POOL_ENABLED: "true"
  WARM_POOL_SIZE: "5"

  # CORS
  CORS_ORIGIN: "https://sandbox.yourdomain.com"
  APP_URL: "https://api.sandbox.yourdomain.com"

  # Logging
  LOG_LEVEL: "info"
  NODE_ENV: "production"

kubectl apply -f aws-configmap.yaml

Deploy Controller

kubectl apply -f k8s/rbac.yaml
kubectl apply -f k8s/deployment.yaml
kubectl apply -f k8s/service.yaml
kubectl apply -f k8s/network-policy.yaml
kubectl apply -f k8s/resource-quota.yaml

Create Ingress with ALB

# aws-ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: sandbox-controller
  namespace: stateset-sandbox
  annotations:
    kubernetes.io/ingress.class: alb
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: ip
    alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:us-east-1:YOUR_ACCOUNT:certificate/xxx
    alb.ingress.kubernetes.io/ssl-redirect: "443"
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS": 443}]'
    alb.ingress.kubernetes.io/healthcheck-path: /health
    alb.ingress.kubernetes.io/healthcheck-interval-seconds: "15"
    alb.ingress.kubernetes.io/healthcheck-timeout-seconds: "5"
    alb.ingress.kubernetes.io/healthy-threshold-count: "2"
    alb.ingress.kubernetes.io/unhealthy-threshold-count: "2"
    # WAF integration (optional)
    # alb.ingress.kubernetes.io/wafv2-acl-arn: arn:aws:wafv2:...
spec:
  rules:
    - host: api.sandbox.yourdomain.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: sandbox-controller
                port:
                  number: 80

kubectl apply -f aws-ingress.yaml

Step 6: Set Up Monitoring

CloudWatch Container Insights

# Install CloudWatch agent
aws eks create-addon \
  --cluster-name stateset-sandbox \
  --addon-name amazon-cloudwatch-observability \
  --configuration-values '{
    "containerLogs": {
      "enabled": true
    },
    "agent": {
      "config": {
        "logs": {
          "metrics_collected": {
            "kubernetes": {
              "cluster_name": "stateset-sandbox"
            }
          }
        }
      }
    }
  }'

Prometheus (Optional)

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm install prometheus prometheus-community/kube-prometheus-stack \
  --namespace monitoring \
  --create-namespace \
  --set prometheus.prometheusSpec.serviceMonitorSelectorNilUsesHelmValues=false

# Apply ServiceMonitor
kubectl apply -f k8s/prometheus/service-monitor.yaml

# Apply alert rules
kubectl apply -f k8s/prometheus/alerts.yaml

Step 7: Configure Autoscaling

Cluster Autoscaler

# Create IAM policy
cat <<EOF > cluster-autoscaler-policy.json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "autoscaling:DescribeAutoScalingGroups",
        "autoscaling:DescribeAutoScalingInstances",
        "autoscaling:DescribeLaunchConfigurations",
        "autoscaling:DescribeTags",
        "autoscaling:SetDesiredCapacity",
        "autoscaling:TerminateInstanceInAutoScalingGroup",
        "ec2:DescribeLaunchTemplateVersions"
      ],
      "Resource": "*"
    }
  ]
}
EOF

aws iam create-policy \
  --policy-name ClusterAutoscalerPolicy \
  --policy-document file://cluster-autoscaler-policy.json

# Install autoscaler
helm repo add autoscaler https://kubernetes.github.io/autoscaler
helm install cluster-autoscaler autoscaler/cluster-autoscaler \
  --namespace kube-system \
  --set autoDiscovery.clusterName=stateset-sandbox \
  --set awsRegion=us-east-1

Horizontal Pod Autoscaler

# hpa.yaml
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: sandbox-controller
  namespace: stateset-sandbox
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: sandbox-controller
  minReplicas: 3
  maxReplicas: 10
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70
    - type: Resource
      resource:
        name: memory
        target:
          type: Utilization
          averageUtilization: 80

Security Hardening

VPC Security Groups

# Create security group for controller
aws ec2 create-security-group \
  --group-name stateset-sandbox-controller \
  --description "StateSet Sandbox Controller" \
  --vpc-id vpc-xxx

# Allow inbound from ALB
aws ec2 authorize-security-group-ingress \
  --group-id sg-xxx \
  --protocol tcp \
  --port 8080 \
  --source-group sg-alb-xxx

# Create security group for sandboxes
aws ec2 create-security-group \
  --group-name stateset-sandbox-pods \
  --description "StateSet Sandbox Pods" \
  --vpc-id vpc-xxx

# Allow inbound from controller only
aws ec2 authorize-security-group-ingress \
  --group-id sg-yyy \
  --protocol tcp \
  --port 0-65535 \
  --source-group sg-controller-xxx

KMS Encryption

# Create KMS key for secrets
aws kms create-key \
  --description "StateSet Sandbox secrets encryption" \
  --tags TagKey=Application,TagValue=stateset-sandbox

# Enable EKS secrets encryption
aws eks associate-encryption-config \
  --cluster-name stateset-sandbox \
  --encryption-config '[{
    "resources": ["secrets"],
    "provider": {"keyArn": "arn:aws:kms:us-east-1:xxx:key/xxx"}
  }]'

Cost Optimization

Spot Instances for Sandbox Nodes

# Add to cluster.yaml
managedNodeGroups:
  - name: sandbox-spot
    instanceTypes:
      - c5.2xlarge
      - c5a.2xlarge
      - c5n.2xlarge
    spot: true
    desiredCapacity: 3
    labels:
      node-type: sandbox
      lifecycle: spot
    taints:
      - key: stateset.com/sandbox
        value: "true"
        effect: NoSchedule

Savings Plans

Consider purchasing Compute Savings Plans for the controller nodes (always running) and using Spot for sandbox nodes (variable load).

Estimated Monthly Costs

Resource	Configuration	Est. Cost
EKS Cluster	1 cluster	$73
Controller Nodes	3x t3.large (on-demand)	~$175
Sandbox Nodes	3-10x c5.2xlarge (spot)	~ $200-$ 800
RDS PostgreSQL	db.r6g.large Multi-AZ	~$350
ElastiCache Redis	cache.r6g.large 2 replicas	~$400
ALB	1 ALB + traffic	~$50
S3	Storage + requests	~$25
Data Transfer	~500GB/month	~$45
Total		~ $1,300-$ 1,900/month

Troubleshooting

Common Issues

Pod stuck in Pending

kubectl describe pod <pod-name> -n stateset-sandbox
# Check for resource constraints or node affinity issues

ALB not routing traffic

kubectl describe ingress sandbox-controller -n stateset-sandbox
# Check ALB controller logs
kubectl logs -n kube-system -l app.kubernetes.io/name=aws-load-balancer-controller

gVisor pods failing

# Check node has gVisor installed
kubectl get nodes -l isolation=gvisor
kubectl debug node/<node-name> -it --image=busybox -- ls /usr/local/bin/runsc

​AWS Deployment Guide (EKS)

​Prerequisites

​Quick Start (Kustomize Overlay)

​Architecture Overview

​Step 1: Create EKS Cluster

​Option A: Using eksctl (Recommended)

​Option B: Using Terraform

​Step 2: Install Required Components

​AWS Load Balancer Controller

​External DNS (Optional)

​cert-manager

​gVisor RuntimeClass

​Step 3: Create AWS Resources

​RDS PostgreSQL

​ElastiCache Redis

​S3 Bucket for Artifacts

​Secrets Manager

​Step 4: Install External Secrets Operator

​Step 5: Deploy StateSet Sandbox

​Create Namespace

​Create ConfigMap

​Deploy Controller

​Create Ingress with ALB

​Step 6: Set Up Monitoring

​CloudWatch Container Insights

​Prometheus (Optional)

​Step 7: Configure Autoscaling

​Cluster Autoscaler

​Horizontal Pod Autoscaler

​Security Hardening

​VPC Security Groups

​KMS Encryption

​Cost Optimization

​Spot Instances for Sandbox Nodes

​Savings Plans

​Estimated Monthly Costs

​Troubleshooting

​Common Issues

​Support Resources