VPC Connectivity

VPC connectivity issues are among the most common problems when setting up OpenSearch UI with VPC-based domains. This guide walks through diagnosing and resolving each type of connectivity failure.

Quick diagnosis flowchart

Start here to identify your issue:

SymptomLikely CauseJump To
Data source shows "Unavailable"VPC endpoint not authorizedEndpoint authorization
Data source shows "Unhealthy"Security group blocking trafficSecurity group issues
Application can't reach domainDNS resolution failureDNS resolution
Intermittent connectivitySecurity group or endpoint instabilityIntermittent issues
Lambda can't reach OpenSearch UILambda VPC configurationLambda in VPC

Endpoint not authorized

The most common VPC issue is forgetting to authorize the OpenSearch UI service to access the VPC endpoint.

Diagnosis

Check if the authorization exists:

aws opensearch list-vpc-endpoint-access \
    --domain-name my-vpc-domain \
    --region us-east-1

If the response doesn't include application.opensearchservice.amazonaws.com, the endpoint is not authorized.

Solution

Authorize the endpoint:

aws opensearch authorize-vpc-endpoint-access \
    --domain-name my-vpc-domain \
    --service application.opensearchservice.amazonaws.com \
    --region us-east-1

Wait 5-10 minutes for the authorization to propagate, then check the data source health in your application.

For serverless collections

Serverless collections use network policies instead. Verify the policy:

aws opensearchserverless list-security-policies \
    --type network \
    --region us-east-1

Check that the policy includes:

{
  "SourceServices": [
    "application.opensearchservice.amazonaws.com"
  ]
}

If missing, create or update the network policy:

aws opensearchserverless create-security-policy \
    --type network \
    --name opensearch-ui-access \
    --policy '[{
      "Description": "Allow OpenSearch UI access",
      "Rules": [{
        "ResourceType": "collection",
        "Resource": ["collection/my-collection"]
      }],
      "SourceServices": ["application.opensearchservice.amazonaws.com"],
      "AllowFromPublic": false
    }]' \
    --region us-east-1

Security group misconfiguration

Even with endpoint authorization, the domain's security group must allow inbound traffic on port 443.

Diagnosis

  1. Find the security group attached to your domain:

    aws opensearch describe-domain \
        --domain-name my-vpc-domain \
        --region us-east-1 \
        --query 'DomainStatus.VPCOptions.SecurityGroupIds'
  2. Check the inbound rules:

    aws ec2 describe-security-groups \
        --group-ids sg-abc123 \
        --region us-east-1 \
        --query 'SecurityGroups[0].IpPermissions'
  3. Look for a rule allowing TCP 443 inbound.

Solution

Add an inbound rule for TCP 443:

aws ec2 authorize-security-group-ingress \
    --group-id sg-abc123 \
    --protocol tcp \
    --port 443 \
    --cidr 10.0.0.0/16 \
    --region us-east-1

Replace 10.0.0.0/16 with your VPC CIDR block. For tighter security, use the security group of the VPC endpoint as the source:

aws ec2 authorize-security-group-ingress \
    --group-id sg-abc123 \
    --protocol tcp \
    --port 443 \
    --source-group sg-endpoint-xyz \
    --region us-east-1

Common security group mistakes

MistakeWhy It FailsFix
Only allowing port 9200OpenSearch UI uses HTTPS (443), not 9200Add TCP 443 inbound
Restricting to specific IPThe VPC endpoint IP may changeUse CIDR block or security group reference
No outbound rulesResponses can't reach the endpointAllow all outbound (default)
Wrong VPC CIDRSource CIDR doesn't match endpoint subnetUse the correct VPC CIDR

DNS resolution problems

VPC endpoints rely on DNS to route traffic. If DNS resolution is misconfigured, the application can't find the domain.

Diagnosis

Check if the VPC has DNS resolution enabled:

aws ec2 describe-vpc-attribute \
    --vpc-id vpc-abc123 \
    --attribute enableDnsSupport \
    --region us-east-1
 
aws ec2 describe-vpc-attribute \
    --vpc-id vpc-abc123 \
    --attribute enableDnsHostnames \
    --region us-east-1

Both should return true.

Solution

Enable DNS support and hostnames:

aws ec2 modify-vpc-attribute \
    --vpc-id vpc-abc123 \
    --enable-dns-support '{"Value": true}' \
    --region us-east-1
 
aws ec2 modify-vpc-attribute \
    --vpc-id vpc-abc123 \
    --enable-dns-hostnames '{"Value": true}' \
    --region us-east-1

Private hosted zones

If you're using Route 53 private hosted zones, ensure they don't override the VPC endpoint DNS entries. Custom DNS configurations can interfere with the automatic DNS resolution that VPC endpoints rely on.

NAT gateway issues

If your OpenSearch domain is in a private subnet and needs to communicate with AWS services (like the OpenSearch UI service), a NAT gateway may be required.

When NAT is needed

  • The domain is in a private subnet (no internet gateway route)
  • The VPC endpoint for OpenSearch is not configured
  • The domain needs to reach AWS APIs for management operations

Diagnosis

Check the route table for the domain's subnet:

aws ec2 describe-route-tables \
    --filters "Name=association.subnet-id,Values=subnet-abc123" \
    --region us-east-1 \
    --query 'RouteTables[0].Routes'

Look for a route to 0.0.0.0/0 pointing to a NAT gateway (nat-xxx).

Solution

If no NAT gateway exists and one is needed:

  1. Create a NAT gateway in a public subnet:

    aws ec2 create-nat-gateway \
        --subnet-id subnet-public-123 \
        --allocation-id eipalloc-abc123 \
        --region us-east-1
  2. Add a route in the private subnet's route table:

    aws ec2 create-route \
        --route-table-id rtb-private-456 \
        --destination-cidr-block 0.0.0.0/0 \
        --nat-gateway-id nat-xyz789 \
        --region us-east-1

For OpenSearch UI connectivity specifically, VPC endpoint authorization is the preferred approach over NAT gateways. NAT gateways add cost and complexity.

Lambda in VPC can't reach OpenSearch UI

If you have Lambda functions in a VPC that need to interact with the OpenSearch UI API or the underlying domains, they need proper network configuration.

Common scenario

A Lambda function in a VPC tries to call the OpenSearch UI application URL or the domain endpoint and gets a timeout.

Diagnosis

  1. Check if the Lambda function is in a VPC:

    aws lambda get-function-configuration \
        --function-name my-function \
        --query 'VpcConfig'
  2. Verify the Lambda's subnet has a route to the internet (via NAT) or to the VPC endpoint.

  3. Check the Lambda's security group allows outbound traffic on port 443.

Solution: VPC endpoint approach (recommended)

Create a VPC endpoint for the OpenSearch service:

aws ec2 create-vpc-endpoint \
    --vpc-id vpc-abc123 \
    --service-name com.amazonaws.us-east-1.es \
    --vpc-endpoint-type Interface \
    --subnet-ids subnet-abc123 subnet-def456 \
    --security-group-ids sg-lambda-123 \
    --region us-east-1

Solution: NAT gateway approach

If the Lambda needs to reach the public OpenSearch UI URL:

  1. Place the Lambda in a private subnet
  2. Ensure the private subnet routes to a NAT gateway in a public subnet
  3. The Lambda's security group must allow outbound TCP 443

Lambda security group checklist

RuleDirectionProtocolPortSource/Destination
Outbound to OpenSearchOutboundTCP443Domain security group or VPC CIDR
Outbound to internetOutboundTCP4430.0.0.0/0 (via NAT)

Intermittent connectivity

If the connection works sometimes but not always, the issue is usually related to transient network conditions.

Common causes

  1. Domain is processing a configuration change — During blue/green deployments, connectivity may be briefly interrupted
  2. Security group was modified — Someone may have temporarily changed the rules
  3. VPC endpoint is being recreated — After certain domain changes, the endpoint may be reprovisioned
  4. Resource constraints on the domain — High CPU or memory pressure can cause timeouts

Diagnosis

Check domain events:

aws opensearch describe-domain \
    --domain-name my-vpc-domain \
    --query 'DomainStatus.Processing'

If Processing is true, the domain is undergoing a change. Wait for it to complete.

Check CloudWatch metrics:

aws cloudwatch get-metric-statistics \
    --namespace AWS/ES \
    --metric-name CPUUtilization \
    --dimensions Name=DomainName,Value=my-vpc-domain Name=ClientId,Value=123456789012 \
    --start-time $(date -u -d '1 hour ago' +%Y-%m-%dT%H:%M:%SZ) \
    --end-time $(date -u +%Y-%m-%dT%H:%M:%SZ) \
    --period 300 \
    --statistics Average

Complete VPC troubleshooting checklist

Run through each item when debugging VPC connectivity:

  • VPC endpoint authorization is in place (list-vpc-endpoint-access)
  • Security group allows inbound TCP 443
  • Security group allows outbound traffic
  • VPC has DNS support enabled
  • VPC has DNS hostnames enabled
  • Domain is in Active state (not Processing)
  • Domain access policy allows the OpenSearch UI service
  • (Serverless) Network policy includes SourceServices
  • (Serverless) Data access policy grants read permissions
  • (Lambda) Lambda subnet has route to NAT or VPC endpoint
  • (Lambda) Lambda security group allows outbound 443
  • No conflicting Route 53 private hosted zones

Getting help

If you've exhausted this checklist and still have connectivity issues:

  1. Collect the domain name, VPC ID, subnet IDs, and security group IDs
  2. Run describe-domain and save the full output
  3. Check CloudTrail for any recent AuthorizeVpcEndpointAccess or UpdateDomainConfig events
  4. Open a support case with AWS Support including all collected information