Skip to main content

Troubleshooting

Common issues and their resolutions for Surface Security deployments.

Backend Issues​

Backend Won't Start​

Error: failed to connect to postgres

  • Ensure the PostgreSQL container is running: docker compose ps
  • Check connection string environment variables (PG_HOST, PG_PORT, PG_USER, PG_PASSWORD)
  • Verify database initialization completed: docker compose logs postgres

Error: failed to connect to clickhouse

  • ClickHouse can take 30-60 seconds to start after container launch
  • Check ClickHouse logs: docker compose logs clickhouse
  • Verify port 9000 is not in use by another process: netstat -an | grep 9000

Database Connection Issues​

# Test PostgreSQL connection
docker compose exec postgres pg_isready -U surfacesec

# Test ClickHouse connection
docker compose exec clickhouse clickhouse-client --query "SELECT 1"

Frontend Issues​

Frontend Build Fails​

Error: Module not found

  • Delete node_modules and reinstall: rm -rf node_modules && npm install
  • Clear the Next.js cache: rm -rf .next

Extension Issues​

Extension Not Collecting Events​

If no events appear in the dashboard:

  1. Check the service worker console for errors (navigate to chrome://extensions/, click "Service Worker" link)
  2. Verify enrollment was successful and the device certificate is installed
  3. Confirm API endpoint is configured correctly in extension settings
  4. Check certificate validity -- the device certificate must not be expired or revoked
  5. Verify E2E encryption is enabled in tenant settings (Settings > General)
  6. Confirm E2E channel key is available (synced via the policy endpoint)
  7. Re-enroll if needed -- devices enrolled before E2E was enabled must re-enroll

Extension Not Installing (Enterprise)​

  1. Check Intune deployment status in the Intune portal
  2. Verify the extension ID is correct in the managed policy
  3. Check browser policies are applied: chrome://policy or edge://policy
  4. Confirm the update URL is reachable from the device

Native Host Connection Failed​

  1. Verify registry entries (Windows): 
    Get-ItemProperty "HKLM:\SOFTWARE\Google\Chrome\NativeMessagingHosts\com.surfacesec.agent"
  2. Check the manifest JSON is valid and points to the correct executable path
  3. Verify the native host executable exists at the path specified in the manifest
  4. Check Windows Event Viewer or macOS Console for errors

Auto-Enrollment Fails​

  1. Check if the user exists in Surface Security (must be synced via SCIM first)
  2. Verify the UPN/email matches between Azure AD and Surface Security
  3. Check backend logs for enrollment errors: 
    kubectl logs -l app=surfacesec-api -n surfacesec | grep "auto-enroll"
  4. Verify managed storage is configured correctly: open chrome://policy and look for the extension's managed storage values

SCIM Provisioning Issues​

  1. Check Azure AD provisioning logs in the Enterprise App
  2. Verify the OAuth token is valid and not expired
  3. Test the SCIM endpoint directly: 
    curl https://your-surfacesec.example.com/scim/v2/Users \
      -H "Authorization: Bearer YOUR_ACCESS_TOKEN"

Performance Optimization​

Slow Dashboard Queries​

  • Check ClickHouse table sizes: 
    SELECT table, formatReadableSize(sum(bytes))
    FROM system.parts
    GROUP BY table
  • Verify PostgreSQL indexes exist for frequently filtered columns
  • Consider ClickHouse materialized views for complex aggregations

High Memory Usage​

# Check container memory usage
docker stats
  • ClickHouse: Increase memory limits in docker-compose.yml under deploy.resources.limits.memory
  • Backend: Tune garbage collection via the GOGC environment variable

Kubernetes Debugging​

When pods are failing (CrashLoopBackOff, PodInitializing, etc.), use kubectl to get the actual error details.

Essential Debugging Commands​

# See all pods and their status
kubectl get pods -n surfacesec -o wide

# Get container logs (the actual error messages)
kubectl logs <pod-name> -n surfacesec --tail=50

# Get logs from a specific container in a multi-container pod
kubectl logs <pod-name> -n surfacesec -c <container-name> --tail=50

# Get init container logs
kubectl logs <pod-name> -n surfacesec -c <init-container-name> --tail=20

# Get logs from the previous crashed container
kubectl logs <pod-name> -n surfacesec --previous --tail=50

# Describe pod for events (image pull errors, volume mount failures, probe failures)
kubectl describe pod <pod-name> -n surfacesec | tail -30

# Check events in the namespace
kubectl get events -n surfacesec --sort-by='.lastTimestamp' | tail -20

Common Error Patterns​

SymptomCommandWhat to Look For
CrashLoopBackOff (exit code 1)kubectl logs <pod>Fatal error in application startup
CrashLoopBackOff (exit code 0)kubectl describe pod <pod>Liveness probe failure causing clean shutdown
PodInitializingkubectl describe pod <pod>Image pull failure or volume mount error
CreateContainerConfigErrorkubectl describe pod <pod>Missing Secret or ConfigMap reference

"Install the Surface extension" persists after install: switch back to the original landing tab. The page waits for a content-script handshake from the extension; the handshake is sent only when the extension loads on this tab. Refresh once if the page hasn't auto-advanced.

"Browser not supported" on Firefox or Safari: magic-link enrollment is currently Chrome / Edge only. For Firefox / Safari users, use the MDM rollout path or self-install with a per-user token.

"Enrollment timed out waiting for the identity provider": the landing page polls for up to 30 seconds after the SSO callback. If the IdP returned an error or the callback never fired, click the magic link again to retry. Repeated failures: check that the IdP's redirect URI registration matches the one shown at Settings > General > Deployment Identifiers.

"Magic link is no longer active": the link was revoked or has expired. Generate a new one from Onboarding > Magic Link.

Getting Help​

  1. Check service logs: docker compose logs -f or kubectl logs
  2. Review the health endpoint: curl http://localhost:8080/health
  3. Contact support: support@surface-security.com