Skip to main content

AWS EKS Deployment

AWS-specific instructions for deploying Surface Security on Amazon Elastic Kubernetes Service (EKS) with internet-facing telemetry ingestion from remote browser extensions.

Purchased via AWS Marketplace? If you subscribed to Surface Security through the AWS Marketplace, see the AWS Marketplace Deployment guide first to obtain your license key, then return here for infrastructure setup.

Architecture Overview

+--------------------------------------+
| Route 53 / ACM / CloudFront |
| surfacesec.yourdomain.com |
+----------+---------------+-----------+
| |
HTTPS (admin) | | mTLS (extensions)
| |
+----------v-----------+ +v------------------+
| NGINX Ingress / ALB | | NLB (L4) |
| (TLS termination) | | -> Envoy Service |
+------+---------------+ +------+-----------+
| |
+-------------v------+ +-------------v----------+
| Admin API / SCIM | | Envoy (mTLS term) |
| Frontend Dashboard | | Client cert validation |
+-------------+------+ +--------+---------------+
| |
+------v-------------------v---------+
| surfacesec-api Service |
+------------------+-----------------+
|
+------------+------------+------------+-------------+
| | | | |
+-----v-----+ +---v----+ +----v----+ +----v---+ +--------v------+
| RDS | |ClickHs | |Redpanda | | S3 | | Ingestion |
| PostgreSQL | | (pod) | | (pod) | | bucket | | Workers |
+-----------+ +--------+ +---------+ +--------+ +---------------+

Two separate entry points:

  • NGINX Ingress or ALB (HTTPS): Admin dashboard, SCIM provisioning, enrollment — standard TLS
  • Envoy NLB (mTLS): Extension API telemetry — mutual TLS with per-device client certificates

Prerequisites

Before starting, ensure you have the following installed and configured:

ToolMinimum VersionPurpose
AWS CLI2.xAWS resource management
eksctl or Pulumi CLIeksctl 0.190+ / Pulumi 3.xCluster creation
kubectl1.31+Cluster operations
Helm3.15+Chart deployment
aws-iam-authenticatorlatestEKS auth (bundled with AWS CLI v2)

You also need:

  • An AWS account with permissions to create EKS clusters, RDS instances, ECR repositories, IAM roles, and VPCs
  • A domain name with DNS you can manage (Route 53 or external)
  • Your Surface Security license key (provided by your sales contact)
# Verify tooling
aws --version
eksctl version
kubectl version --client
helm version

# Configure AWS credentials
aws configure
# or use AWS SSO: aws sso login --profile <profile>

Step 1: Cluster Creation

Choose one of the two provisioning paths below. The Pulumi path is recommended for production; eksctl is faster for pilots and evaluations.

Option A: eksctl

Create a cluster.yaml file (adjust region, instance types, and node counts to match your sizing profile — see the Cost Estimates table):

# cluster.yaml
apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig

metadata:
name: surfacesec
region: us-east-1
version: "1.31"

availabilityZones:
- us-east-1a
- us-east-1b
- us-east-1c

iam:
withOIDC: true # required for IRSA (ECR pull, RDS auth)

managedNodeGroups:
- name: general
instanceType: m5.xlarge # adjust per sizing profile
minSize: 3
maxSize: 10
desiredCapacity: 3
amiFamily: AmazonLinux2023
labels:
role: general
tags:
k8s.io/cluster-autoscaler/enabled: "true"
k8s.io/cluster-autoscaler/surfacesec: "owned"
iam:
attachPolicyARNs:
- arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy
- arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy
- arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly
- arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore
privateNetworking: true # nodes in private subnets

addons:
- name: vpc-cni
version: latest
- name: coredns
version: latest
- name: kube-proxy
version: latest
- name: aws-ebs-csi-driver
version: latest
wellKnownPolicies:
ebsCSIController: true

cloudWatch:
clusterLogging:
enableTypes:
- api
- audit
- authenticator

Deploy the cluster:

eksctl create cluster -f cluster.yaml

# Merge kubeconfig
aws eks update-kubeconfig --region us-east-1 --name surfacesec

# Verify
kubectl get nodes

Option B: Pulumi

The infra/pulumi/ directory contains full IaC for both Azure and AWS. Use the pre-built AWS stack configurations:

cd infra/pulumi

# Install Go dependencies
go mod download

# Pilot deployment
pulumi stack select aws-pilot
pulumi config set surfacesec:domain surfacesec.yourdomain.com
pulumi up

# Production deployment
pulumi stack select aws-prod
pulumi config set surfacesec:domain surfacesec.yourdomain.com
pulumi config set surfacesec:awsRegion us-east-1
pulumi up

Key Pulumi config values for AWS (Pulumi.aws-prod.yaml):

KeyDefaultNotes
surfacesec:cloudProviderawsMust be aws
surfacesec:awsRegionus-east-1Target AWS region
surfacesec:awsNodeSizem6i.2xlargeEC2 instance type for nodes
surfacesec:awsMinNodes5Cluster autoscaler minimum
surfacesec:awsMaxNodes20Cluster autoscaler maximum
surfacesec:awsDBInstanceClassdb.r6g.xlargeRDS instance class
surfacesec:kubernetesVersion1.31EKS Kubernetes version

After pulumi up, retrieve the kubeconfig:

aws eks update-kubeconfig --region us-east-1 --name surfacesec

Step 2: RDS PostgreSQL

Surface Security requires PostgreSQL 16+. Use Amazon RDS for managed, HA-capable PostgreSQL.

Provision via AWS CLI

REGION="us-east-1"
PG_INSTANCE="surfacesec-pg"
PG_PASSWORD="$(openssl rand -base64 32)"
VPC_ID=$(aws eks describe-cluster --name surfacesec --region $REGION \
--query "cluster.resourcesVpcConfig.vpcId" --output text)

# Create a security group that allows PostgreSQL access from EKS nodes
SG_ID=$(aws ec2 create-security-group \
--region $REGION \
--group-name surfacesec-rds-sg \
--description "Surface Security RDS access" \
--vpc-id $VPC_ID \
--query GroupId --output text)

# Allow port 5432 from the VPC CIDR
VPC_CIDR=$(aws ec2 describe-vpcs --vpc-ids $VPC_ID --region $REGION \
--query "Vpcs[0].CidrBlock" --output text)
aws ec2 authorize-security-group-ingress \
--region $REGION \
--group-id $SG_ID \
--protocol tcp \
--port 5432 \
--cidr $VPC_CIDR

# Get private subnet IDs from the EKS cluster VPC
SUBNET_IDS=$(aws ec2 describe-subnets \
--region $REGION \
--filters "Name=vpc-id,Values=$VPC_ID" "Name=tag:aws:cloudformation:logical-id,Values=SubnetPrivate*" \
--query "Subnets[*].SubnetId" --output text | tr '\t' ',')

# Create DB subnet group
aws rds create-db-subnet-group \
--region $REGION \
--db-subnet-group-name surfacesec-pg-subnets \
--db-subnet-group-description "Surface Security RDS subnets" \
--subnet-ids $(echo $SUBNET_IDS | tr ',' ' ')

# Create the RDS instance
aws rds create-db-instance \
--region $REGION \
--db-instance-identifier $PG_INSTANCE \
--db-instance-class db.m5.large \
--engine postgres \
--engine-version "16.6" \
--master-username surfacesec_admin \
--master-user-password "$PG_PASSWORD" \
--db-name surfacesec \
--db-subnet-group-name surfacesec-pg-subnets \
--vpc-security-group-ids $SG_ID \
--storage-type gp3 \
--allocated-storage 100 \
--storage-encrypted \
--backup-retention-period 7 \
--no-publicly-accessible \
--deletion-protection

# Wait for the instance to become available (~10 minutes)
aws rds wait db-instance-available --region $REGION --db-instance-identifier $PG_INSTANCE

# Get the endpoint
PG_HOST=$(aws rds describe-db-instances \
--region $REGION \
--db-instance-identifier $PG_INSTANCE \
--query "DBInstances[0].Endpoint.Address" --output text)
echo "RDS endpoint: $PG_HOST"

For Multi-AZ (required for Medium/Large profiles), add --multi-az to the create-db-instance call.

Recommendation: Use a private subnet and VPC security groups. Never expose RDS publicly. For production, consider Aurora PostgreSQL-compatible for automatic failover and read replicas.

Store the Connection String as a Kubernetes Secret

kubectl create namespace surfacesec

kubectl create secret generic surfacesec-db-credentials \
--namespace=surfacesec \
--from-literal=host="$PG_HOST" \
--from-literal=database="surfacesec" \
--from-literal=username="surfacesec_admin" \
--from-literal=password="$PG_PASSWORD"

Step 3: ECR Image Pull

Images are published to GitHub Container Registry (ghcr.io/surfacesecurity) by default. For EKS deployments you have two options.

When the EKS node IAM role includes AmazonEC2ContainerRegistryReadOnly (already included in the cluster.yaml above), nodes can pull from ECR without Kubernetes pull secrets. Mirror the images to ECR first:

REGION="us-east-1"
ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)
ECR_REGISTRY="${ACCOUNT_ID}.dkr.ecr.${REGION}.amazonaws.com"

# Create ECR repositories (one per image)
for img in api ingestion preflight frontend; do
aws ecr create-repository \
--region $REGION \
--repository-name surfacesecurity/$img \
--image-scanning-configuration scanOnPush=true
done

# Authenticate Docker to ECR
aws ecr get-login-password --region $REGION \
| docker login --username AWS --password-stdin $ECR_REGISTRY

# Pull from GHCR and push to ECR
VERSION="1.0.0"
for img in api ingestion preflight frontend; do
docker pull ghcr.io/surfacesecurity/surfacesec-${img}:${VERSION}
docker tag ghcr.io/surfacesecurity/surfacesec-${img}:${VERSION} \
${ECR_REGISTRY}/surfacesecurity/${img}:${VERSION}
docker push ${ECR_REGISTRY}/surfacesecurity/${img}:${VERSION}
done

Set the registry in your Helm values (no imagePullSecrets needed):

image:
registry: <account-id>.dkr.ecr.us-east-1.amazonaws.com/surfacesecurity
tag: "1.0.0"
imagePullSecrets: []

Option B: Kubernetes Pull Secret (fallback)

If IRSA is not configured, create a pull secret manually:

# Refresh the ECR token (valid 12 hours — automate with a CronJob in production)
aws ecr get-login-password --region $REGION \
| kubectl create secret docker-registry ecr-pull-secret \
--namespace=surfacesec \
--docker-server=${ECR_REGISTRY} \
--docker-username=AWS \
--docker-password-stdin

Then in values.yaml:

imagePullSecrets:
- name: ecr-pull-secret

Step 4: Ingress Setup

Option A: NGINX Ingress Controller (Default)

NGINX is the recommended ingress for Surface Security and matches the default Helm chart configuration.

# Install NGINX Ingress Controller
helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm repo update

helm install ingress-nginx ingress-nginx/ingress-nginx \
--namespace ingress-nginx \
--create-namespace \
--set controller.service.type=LoadBalancer \
--set controller.service.annotations."service\.beta\.kubernetes\.io/aws-load-balancer-type"="nlb" \
--set controller.service.annotations."service\.beta\.kubernetes\.io/aws-load-balancer-scheme"="internet-facing"

# Wait for the load balancer to be assigned
kubectl get svc -n ingress-nginx ingress-nginx-controller --watch

Install cert-manager with Let's Encrypt

helm repo add jetstack https://charts.jetstack.io
helm repo update

helm install cert-manager jetstack/cert-manager \
--namespace cert-manager \
--create-namespace \
--set crds.enabled=true

# Create a ClusterIssuer for Let's Encrypt (replace email)
kubectl apply -f - <<'EOF'
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt-prod
spec:
acme:
server: https://acme-v02.api.letsencrypt.org/directory
email: ops@yourdomain.com
privateKeySecretRef:
name: letsencrypt-prod
solvers:
- http01:
ingress:
ingressClassName: nginx
EOF

Add the cert-manager annotation to your Helm values:

ingress:
enabled: true
className: nginx
host: surfacesec.yourdomain.com
annotations:
cert-manager.io/cluster-issuer: "letsencrypt-prod"
nginx.ingress.kubernetes.io/ssl-redirect: "true"
tls:
enabled: true
secretName: surfacesec-ingress-tls

Option B: AWS Application Load Balancer (ALB) — Appendix

For organizations that prefer native AWS load balancing with ACM certificates and WAF integration, use the AWS Load Balancer Controller.

Install AWS Load Balancer Controller

# Create an IAM policy for the controller
curl -O https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/main/docs/install/iam_policy.json

aws iam create-policy \
--policy-name AWSLoadBalancerControllerIAMPolicy \
--policy-document file://iam_policy.json

# Create an IRSA service account
ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)
eksctl create iamserviceaccount \
--cluster=surfacesec \
--namespace=kube-system \
--name=aws-load-balancer-controller \
--role-name AmazonEKSLoadBalancerControllerRole \
--attach-policy-arn=arn:aws:iam::${ACCOUNT_ID}:policy/AWSLoadBalancerControllerIAMPolicy \
--approve

# Install the controller via Helm
helm repo add eks https://aws.github.io/eks-charts
helm repo update

helm install aws-load-balancer-controller eks/aws-load-balancer-controller \
--namespace kube-system \
--set clusterName=surfacesec \
--set serviceAccount.create=false \
--set serviceAccount.name=aws-load-balancer-controller

Request an ACM Certificate

# Request a certificate (DNS validation recommended)
aws acm request-certificate \
--domain-name surfacesec.yourdomain.com \
--validation-method DNS \
--region us-east-1

# Note the CertificateArn from the output, then complete DNS validation via Route 53

Configure ALB Ingress

Override ingress values in your Helm install (replace <certificate-arn> and optionally <waf-acl-arn>):

ingress:
enabled: true
className: alb
host: surfacesec.yourdomain.com
annotations:
kubernetes.io/ingress.class: alb
alb.ingress.kubernetes.io/scheme: internet-facing
alb.ingress.kubernetes.io/target-type: ip
alb.ingress.kubernetes.io/certificate-arn: "<certificate-arn>"
alb.ingress.kubernetes.io/ssl-policy: "ELBSecurityPolicy-TLS13-1-2-2021-06"
alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS":443}]'
alb.ingress.kubernetes.io/ssl-redirect: "443"
# Optional WAF integration
alb.ingress.kubernetes.io/wafv2-acl-arn: "<waf-acl-arn>"
tls:
enabled: false # TLS is terminated at ALB; no in-cluster secret needed

Note: The ALB path requires the AWS Load Balancer Controller and ACM. cert-manager is not needed. The NGINX path is simpler for most deployments.

Step 5: Create Remaining Secrets

# ClickHouse credentials (ClickHouse runs as a pod in the cluster)
CH_PASSWORD="$(openssl rand -base64 24)"
kubectl create secret generic surfacesec-clickhouse-credentials \
--namespace=surfacesec \
--from-literal=username="surfacesec" \
--from-literal=password="$CH_PASSWORD"

# S3 credentials for screenshot storage
# Option A: IRSA (attach an IAM policy to the node role or a pod service account)
# Option B: IAM access key
AWS_ACCESS_KEY="<iam-user-access-key>"
AWS_SECRET_KEY="<iam-user-secret-key>"
S3_BUCKET="surfacesec-screenshots-$(aws sts get-caller-identity --query Account --output text)"

aws s3api create-bucket \
--bucket $S3_BUCKET \
--region us-east-1

kubectl create secret generic surfacesec-minio-credentials \
--namespace=surfacesec \
--from-literal=endpoint="s3.amazonaws.com" \
--from-literal=access-key="$AWS_ACCESS_KEY" \
--from-literal=secret-key="$AWS_SECRET_KEY"

# Surface Security license
kubectl create secret generic surfacesec-surface-license \
--namespace=surfacesec \
--from-literal=license-key="YOUR_SIGNED_LICENSE_KEY"

Step 6: Helm Install

Add Helm Repository Dependencies

helm repo add bitnami https://charts.bitnami.com/bitnami
helm repo add redpanda https://charts.redpanda.com
helm repo update

Create an EKS-Specific Values Override File

Save the following as eks-values.yaml and fill in your account-specific values:

# eks-values.yaml — EKS-specific overrides for Surface Security

image:
# Replace with your ECR registry if using Option A from Step 3
# Leave as ghcr.io/surfacesecurity if using GHCR pull secrets
registry: ghcr.io/surfacesecurity
tag: "1.0.0"

# Leave empty if using IRSA; add ecr-pull-secret if using Option B
imagePullSecrets: []

ingress:
enabled: true
className: nginx # change to alb if using the ALB appendix
host: surfacesec.yourdomain.com
annotations:
cert-manager.io/cluster-issuer: "letsencrypt-prod"
nginx.ingress.kubernetes.io/ssl-redirect: "true"
nginx.ingress.kubernetes.io/proxy-body-size: "50m"
nginx.ingress.kubernetes.io/proxy-read-timeout: "60"
nginx.ingress.kubernetes.io/proxy-send-timeout: "60"
tls:
enabled: true
secretName: surfacesec-ingress-tls

# RDS PostgreSQL (managed externally — disable embedded chart)
postgresql:
enabled: false
maxConns: 50
minConns: 10
ingestionMaxConns: 30
ingestionMinConns: 5

config:
# Update with your actual ClickHouse, Redpanda, and MinIO/S3 endpoints
clickhouseHosts: "clickhouse:9000"
kafkaBrokers: "redpanda:9092"
redpandaAdminUrl: "http://redpanda:9644"
surfacePortalUrl: "https://portal.surface-security.com"
heartbeatInterval: "300"
surfacePublicKey: "" # provided by Surface Security

# Use EKS gp3 storage class for all PVCs
opensearch:
storageSize: 50Gi

networkPolicies:
enabled: true # EKS with VPC CNI supports NetworkPolicy
airGapped: false

Run Helm Install

helm install surfacesec ./infra/helm/surfacesec \
--namespace surfacesec \
--create-namespace \
--values eks-values.yaml \
--timeout 10m \
--wait

# Check the rollout
kubectl get pods -n surfacesec
kubectl get ingress -n surfacesec

Upgrade

helm upgrade surfacesec ./infra/helm/surfacesec \
--namespace surfacesec \
--values eks-values.yaml \
--timeout 10m \
--wait

Step 7: DNS Configuration

After the Helm install, retrieve external endpoints:

# NGINX Ingress IP / hostname (admin dashboard)
INGRESS_HOSTNAME=$(kubectl get svc -n ingress-nginx ingress-nginx-controller \
-o jsonpath='{.status.loadBalancer.ingress[0].hostname}')
echo "Dashboard LB: $INGRESS_HOSTNAME"

# Envoy mTLS hostname (extension API)
ENVOY_HOSTNAME=$(kubectl get svc -n surfacesec surfacesec-envoy \
-o jsonpath='{.status.loadBalancer.ingress[0].hostname}')
echo "Extension API LB: $ENVOY_HOSTNAME"

Create DNS records:

RecordTypeValuePurpose
surfacesec.yourdomain.comCNAME$INGRESS_HOSTNAMEAdmin dashboard
api.surfacesec.yourdomain.comCNAME$ENVOY_HOSTNAMEExtension mTLS API

Note: EKS NLBs provide a hostname (not an IP). Use CNAME records (or Route 53 Alias records if your zone is in Route 53).

Cost Estimates

The following estimates are for us-east-1 (2026 on-demand pricing). Reserved Instance pricing reduces EC2/RDS costs by ~35-40%.

ProfileExtensionsEKS NodesEC2 TypeRDS ClassEst. Monthly
Pilotup to 1,0001 nodet3.large (2 vCPU / 8 GiB)db.t3.medium~$200
Small1,000–5,0003 nodesm5.xlarge (4 vCPU / 16 GiB)db.m5.large~$800
Medium5,000–25,0005–8 nodesm5.xlarge (4 vCPU / 16 GiB)db.r5.xlarge~$2,500
Large25,000–60,00010–20 nodesm5.2xlarge (8 vCPU / 32 GiB)db.r5.2xlarge Multi-AZ~$6,000+

Cost components included in estimates:

  • EC2 instances (EKS managed node group)
  • RDS PostgreSQL (single-AZ for Pilot/Small, Multi-AZ for Medium/Large)
  • EKS control plane ($0.10/hr)
  • NAT Gateway (~$45/mo per AZ)
  • NLB for Envoy mTLS ingress
  • S3 storage (minimal for screenshots)

Cost components not included (vary by usage):

  • Data transfer egress
  • CloudWatch logs/metrics
  • ACM (free for AWS-issued certificates)
  • WAF (if enabled via ALB appendix)
  • OpenSearch (if using external managed service instead of in-cluster pod)

Tip: For Pilot deployments, a single t3.large spot instance with postgresql.enabled: true and redpanda.enabled: true reduces cost to under $100/mo for evaluation purposes. Do not use spot instances in production.

Step 8: Verification

Health Checks

# All pods running
kubectl get pods -n surfacesec

# Expected output: all pods in Running state, 0 restarts
# NAME READY STATUS RESTARTS AGE
# surfacesec-api-xxxxx 2/2 Running 0 5m
# surfacesec-ingestion-xxxxx 1/1 Running 0 5m
# surfacesec-frontend-xxxxx 1/1 Running 0 5m

# HPA status
kubectl get hpa -n surfacesec

# Ingress with TLS
kubectl get ingress -n surfacesec

# HTTP health endpoint
curl -f https://surfacesec.yourdomain.com/health
# Expected: {"status":"ok","version":"1.0.0"}

# API readiness (internal)
kubectl exec -n surfacesec deploy/surfacesec-api -- \
wget -qO- http://localhost:8080/healthz
# Expected: {"status":"ok"}

# Envoy mTLS endpoint (requires a device client certificate)
curl --cert device.crt --key device.key --cacert ca.crt \
https://api.surfacesec.yourdomain.com/health

# License verification
kubectl logs -n surfacesec -l component=api --tail=20 | grep -E "license|ready"
# Expected: level=info msg="license verified" ...

# Certificate status (if using cert-manager)
kubectl get certificate -n surfacesec
# Expected: READY=True

Troubleshooting

SymptomLikely CauseResolution
Pods in ImagePullBackOffPull secret missing or ECR token expiredVerify imagePullSecrets or refresh ECR login token
CrashLoopBackOff on APIMissing Kubernetes secretsRun kubectl describe pod <name> -n surfacesec and check for missing env vars; verify all secrets from Step 2 and Step 5 exist
Ingress returns 404DNS not propagated or cert-manager still provisioningCheck kubectl get certificate -n surfacesec; allow up to 5 minutes for Let's Encrypt validation
RDS connection refusedSecurity group blocking port 5432Verify the RDS security group allows inbound TCP 5432 from the VPC CIDR
Envoy pod not healthyTLS bootstrap job failedCheck kubectl logs -n surfacesec job/surfacesec-tls-bootstrap
HPA not scalingMetrics server missingInstall kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml
# General debugging commands
kubectl describe pod <pod-name> -n surfacesec
kubectl logs <pod-name> -n surfacesec --previous
kubectl get events -n surfacesec --sort-by='.lastTimestamp'

# Check secret existence
kubectl get secrets -n surfacesec

# Verify RDS connectivity from a pod
kubectl run pg-test --rm -it --image=postgres:16 --namespace=surfacesec \
--env="PGPASSWORD=$PG_PASSWORD" -- \
psql -h $PG_HOST -U surfacesec_admin -d surfacesec -c "SELECT version();"

Configure Extension Enrollment

  1. Log in to the admin dashboard at https://surfacesec.yourdomain.com
  2. Navigate to Onboarding
  3. Set the Extension API Endpoint to https://api.surfacesec.yourdomain.com
  4. Generate enrollment tokens for new devices
  5. Deploy the extension to browsers (see Extension Deployment)