AWS on Nelson Figueroa

Proxying GoatCounter Requests Through CloudFront to Bypass Ad Blockers

Thu, 09 Apr 2026 00:00:00 +0000

This blog is a Hugo-generated static site hosted on AWS using S3 and CloudFront. I’ve been running GoatCounter on my site using the provided script to see who views my blog posts. Every time someone visits my site, a request goes out to GoatCounter. The problem is that adblockers like uBlock Origin block it (understandably).

To get around this, I set up proxying so that the GoatCounter requests go to an endpoint under my domain nelson.cloud/gc/count, and then from there CloudFront handles it and sends it to GoatCounter. Most ad blockers work based on domain and GoatCounter is on the blocklists. Since the browser is now sending requests to the same domain as my site, it shouldn’t trigger any ad blockers. This post explains how I did it in case it’s useful for anyone else.

It’s possible to self-host GoatCounter, but my approach was easier to do and less infrastructure to maintain. Perhaps in the future.

On Analytics and Privacy

I know I’m bypassing a user’s preference to not be tracked, even if it’s (in my opinion) a harmless analytics tool. I just want to see who reads my stuff, that’s all.

Read the GoatCounter developer’s take if you want another opinion: Analytics on personal websites.

Managing Infrastructure with Pulumi

Clicking through the AWS console to configure CloudFront distributions is a pain in the ass. I took the time to finally get the infrastructure for my blog managed as infrastructure-as-code with Pulumi and Python. So while you can click around the console and do all of this, I will be showing how to configure everything with Pulumi.

If you don’t want to use IaC, you can still find all of these options/settings in AWS itself.

Setting it up

To set up GoatCounter proxying via CloudFront, we’ll need to

Create a new CloudFront function resource
Add a second origin to the distribution
Add an ordered cache behavior to the distribution (which references the CloudFront function using its ARN)
Update the GoatCounter script to point to this new endpoint

CloudFront Function

CloudFront functions are JavaScript scripts that run before a request reaches a CloudFront distribution’s origin. In this case, the function strips the /gc from nelson.cloud/gc/count.

We need to strip /gc for two reasons:

I chose to proxy requests that hit the /gc/count endpoint on my site to make sure there’s no collision with post titles/slugs. I’ll never use the /gc/* path for posts.
GoatCounter accepts requests under /count, not /gc/count

Here is the code for the function:

function handler(event) {
    var request = event.request;
    request.uri = request.uri.replace(/^\/gc/, '');
    if (request.uri === '') request.uri = '/';
    return request;
}

And here is the CloudFront function resource defined in Pulumi (using Python) that includes the JavaScript from above. This is a new resource defined in the same Python file where my existing distribution already exists:

goatcounter_rewrite = aws.cloudfront.Function("goatcounter-rewrite",
    name="goatcounter-rewrite",
    runtime="cloudfront-js-2.0",
    code="""\
function handler(event) {
    var request = event.request;
    request.uri = request.uri.replace(/^\\/gc/, '');
    if (request.uri === '') request.uri = '/';
    return request;
}
""",
)

CloudFront Distribution Origin and Cache Behavior

Here is my existing CloudFront distribution being updated with a new origin and cache behavior in Pulumi code.

Note:
At the time of writing CloudFront only allows allowed_methods to be a list of HTTP methods in specific combinations. The value must be one of these:

[HEAD, GET]

[HEAD, GET, OPTIONS]

[HEAD, DELETE, POST, GET, OPTIONS, PUT, PATCH]

Since the GoatCounter JavaScript sends a POST request, and the third option is the only one that includes POST, we’re forced to use all HTTP verbs. It should be harmless though.

nelson_cloud_distribution = aws.cloudfront.Distribution("nelson-cloud",
    aliases=["nelson.cloud"],
    ###
    # other configuration
    ###
    # route /gc/* requests to GoatCounter, stripping the /gc prefix via CloudFront function
    ordered_cache_behaviors=[{
        "path_pattern": "/gc/*", # the path gets matched here
        "allowed_methods": ["GET", "HEAD", "OPTIONS", "PUT", "PATCH", "POST", "DELETE"],
        "cached_methods": ["GET", "HEAD"],
        "target_origin_id": "goatcounter",
        "viewer_protocol_policy": "redirect-to-https",
        "compress": False,
        "cache_policy_id": "4135ea2d-6df8-44a3-9df3-4b5a84be39ad",  # CachingDisabled
        "origin_request_policy_id": "b689b0a8-53d0-40ab-baf2-68738e2966ac",  # AllViewerExceptHostHeader
        "function_associations": [{
            "event_type": "viewer-request",
            "function_arn": goatcounter_rewrite.arn, # CloudFront function from above
        }],
    }],
    origins=[
        ###
        # other existing origins
        ###
        # a new origin to proxy GoatCounter requests
        {
            "custom_origin_config": {
                "http_port": 80,
                "https_port": 443,
                "origin_protocol_policy": "https-only",
                "origin_ssl_protocols": ["TLSv1.2"],
            },
            "domain_name": "nelsonfigueroa.goatcounter.com",
            "origin_id": "goatcounter",
        },
    ],
    ###
    # rest of configuration
    ###
    opts = pulumi.ResourceOptions(protect=True))

Now that my Pulumi code has both the CloudFront function defined and the CloudFront distribution has been updated, I ran pulumi up to apply changes.

Update the GoatCounter Script

Finally, I updated goatcounter.js to use the new endpoint. So instead of goatcounter.com I changed it to my own domain nelson.cloud/gc/count at the very top of the snippet:

1
2
3

<script data-goatcounter="https://nelson.cloud/gc/count">
/* rest of script */
</script>

After this, I built my site with Hugo and deployed it on S3/CloudFront by updating the freshly built HTML/CSS/JS in my S3 Bucket and then invalidating the existing CloudFront cache.

Verifying that it Works

Now, GoatCounter should no longer be blocked by uBlock Origin. I tested by loading my site on an incognito browser window and checked that uBlock Origin was no longer blocking anything on my domain.

And for further proof, checking the network tab shows a successful POST request to the /gc/count endpoint on my domain along with response headers from AWS/CloudFront:

Everything looks good!

Support GoatCounter

If you’re using GoatCounter you should consider sponsoring the developer. It’s a great project.

References

GitHub Actions for Pulumi with an AWS S3 Backend

Thu, 11 Dec 2025 00:00:00 +0000

Introduction

This is a quick guide to set up GitHub Actions for Pulumi with an AWS S3 backend. It builds on a previous guide I wrote: How to Use an AWS S3 Bucket as a Pulumi State Backend.

This guide assumes you have the following:

An AWS S3 Bucket created and ready to be used with Pulumi
An IAM User that has permissions to read/write to the S3 bucket
The Access Key and Secret Access Key for the IAM User to use for authenticating to AWS within GitHub Actions
A passphrase of your choosing that will be used to encrypt secrets in the pulumi stack
A GitHub repository

Setting up Repository Secrets

First, set up secrets on your GitHub repository. These will be filled in by GitHub Actions once we create a workflow YAML. You’ll need to create 3 secrets. You can name them whatever you want, but I’ll be naming them:

AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY
PULUMI_CONFIG_PASSPHRASE

You can create these by browsing to your GitHub repository > Settings > Secrets and variables > Actions. There are “Environment secrets” and “Repository secrets”. In this case go with “Repository secrets”.

Create the three secrets and fill in their respective values. PULUMI_CONFIG_PASSPHRASE can be whatever you want and doesn’t come from AWS.

Warning:
Make sure you don’t change this passphrase after the fact, or your Pulumi state may break.

The end result should look like this:

Defining the GitHub Actions Workflow

Next, clone the repository locally with git clone. We’ll need to create a few files for a minimum viable Pulumi program. Run pulumi new and it’ll guide you through the creation of a basic Pulumi program. The language you choose doesn’t matter.

Then we can create the YAML file to set up GitHub Actions. Create a YAML file under /.github/workflows/. I’ll call it preview.yml in my example. Fill it in with the following YAML, changing values as needed.

name: Pulumi
on:
  push:
    branches:
      - main # change this if you're using a different branch name

jobs:
  preview:
    name: Preview
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Pulumi Preview
        uses: pulumi/actions@v6.6.1
        with:
          command: preview
          stack-name: dev # change this to your stack's name (or whatever you want it to be if it doesn't exist)
          cloud-url: s3://nelson-test-bucket-for-github-actions # change this to your bucket name to be used as a pulumi backend
          upsert: true # creates a stack if it doesn't exist (along with Pulumi..yaml)
        env:
          AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
          AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          PULUMI_CONFIG_PASSPHRASE: ${{ secrets.PULUMI_CONFIG_PASSPHRASE }}
          AWS_REGION: us-east-1 # change the region as needed

This runs a pulumi preview so it’ll verify that everything is set up correctly without actually deploying anything.

Testing the Workflow

Now push your code to GitHub and see if the GitHub Action workflow ran successfully. You should see output from a successful pulumi preview.

If this works, you should be good to go. You can update your Pulumi code and change command: preview to command: up in the GitHub Actions YAML file and run it again to actually deploy some infrastructure.

References

https://github.com/pulumi/actions

How to Use an AWS S3 Bucket as a Pulumi State Backend

Sat, 20 Sep 2025 00:00:00 +0000

I’ll be starting from scratch and creating an IAM user with access to an S3 bucket that will be used to store the Pulumi state file. If you’re working in an enterprise setting, your authentication methods may vary.

tl;dr:
You can run this command (replacing placeholders as needed) if you already have an S3 bucket and AWS credentials configured on your machine:
1
pulumi login 's3://?region=&awssdk=v2&profile='

This post assumes you have the Pulumi CLI installed. Check out the following guide if you don’t have it installed: Download & install Pulumi.

Creating an S3 Bucket

First, we need to create the S3 bucket where the Pulumi state file will be stored. I created a bucket called nelsons-pulumi-state-backend and left all the default settings as-is.

Creating an IAM User

Then we need to create an IAM user in AWS that the Pulumi CLI can use. This IAM user needs permissions to access the S3 bucket we just created.

I go to IAM and create a new user. I just called it pulumi:

Then in the next step, I selected “Attach policies directly” and selected the AWS-managed “AdministratorAccess” policy just to keep things simple. You can provide more fine-grained access depending on your needs. Then click “Next” at the bottom.

In the next screen, double check everything and then click on “Create user”.

Now that we have a user with the appropriate permissions, we’ll need to get an AWS access key and secret to use with the Pulumi CLI.

Go to your IAM user and click on “Create access key” on the right side.

In the next screen, select “Command Line Interface (CLI)”. Check the box at the bottom, then click “Next”.

The next screen will ask for setting a description tag. This is optional. I chose to skip it and clicked on “Create access key”.

We finally have our Access key and Secret access key. Save these somewhere safe and click “Done”. (Don’t worry, the credentials in the screenshot are fake.)

Setting Up AWS Credentials for the Pulumi CLI

Now we can try using these credentials to tell the Pulumi CLI to use the S3 bucket as a backend.

Note:
Note that you do NOT need the AWS CLI installed. Pulumi just needs the AWS credentials.

Create the file ~/.aws/credentials if you don’t have it. Then add in your credentials there under the [default] profile. (You can add more profiles, but this is beyond the scope of this post.)

[default]
aws_access_key_id = 
aws_secret_access_key =

You’ll need the bucket’s region and your local AWS profile name to use S3 as a backend.

The command formula looks like this:

`1`	`pulumi login 's3://?region=&awssdk=v2&profile='`

In my case, the command looks like this (make sure to edit for your needs):

`1`	`pulumi login 's3://nelsons-pulumi-state-backend?region=us-west-1&awssdk=v2&profile=default'`

A successful login shows the following message:

Logged in to 0x6E.local as nelson (s3://nelsons-pulumi-state-backend?region=us-west-1&awssdk=v2&profile=default)

Alternatively, you can add your backend to your Pulumi.yaml file. This is useful if you’re working on multiple Pulumi projects that each have different backends. You won’t need to run pulumi login all the time. Just add a backend key and a nested url key:

name: my-pulumi-project
description: a pulumi project for testing
runtime: nodejs

# add this section
backend:
  url: s3://nelsons-pulumi-state-backend?region=us-west-1&awssdk=v2&profile=default

More information here: Pulumi project file reference.

Testing The Setup

Finally, it’s time to test this out.

To demonstrate, I created a simple Pulumi program by running:

`1`	`pulumi new aws-python`

You can choose whatever language you want though.

This is the main Pulumi code that is generated. It’s code for creating an S3 bucket:

"""An AWS Python Pulumi program"""

import pulumi
from pulumi_aws import s3

# Create an AWS resource (S3 Bucket)
bucket = s3.Bucket('my-bucket')

# Export the name of the bucket
pulumi.export('bucket_name', bucket.id)

Then I ran pulumi up -y and it worked!

And just to double check, I can see that my previously empty S3 bucket now has contents created by the Pulumi CLI:

Everything works!

If you’re interested in setting up CI/CD with this setup, I wrote a post showing how to do so: GitHub Actions for Pulumi with an AWS S3 Backend.

References

Fix "Efficient Cache Policy" Warning on PageSpeed Insights When Using Amazon CloudFront

Thu, 02 Jan 2025 00:00:00 +0000

I ran into this warning recently on PageSpeed Insights: “Serve static assets with an efficient cache policy”. The warning highlighted three assets that had no “Cache TTL” defined:

To resolve this warning, I added a Cache-Control header with the value max-age=31536000 to the HTTP responses of my domain (31536000 is the number of seconds in a year).

I host nelson.cloud on Amazon CloudFront. I added this header to the CloudFront Distribution that the nelson.cloud domain points to. This can be done by adding a “Response headers policy” in the Behavior of the Distribution. I wrote a post demonstrating how to do this in detail: How to Add a Custom Response Header to an Amazon CloudFront Distribution.

After the header was configured, I checked PageSpeed Insights again and the warning had gone away for the assets under my domain nelson.cloud:

(The remaining asset is not under my control so I can’t fix that one.)

Note:
You may be able to use a different value smaller than 31536000 for your header. I chose a year in seconds to be on the safe side for PageSpeed Insights.

References

How to Add a Custom Response Header to an Amazon CloudFront Distribution

Wed, 01 Jan 2025 00:00:00 +0000

This post assumes you already have an Amazon CloudFront Distribution deployed and properly configured. You’ll also need knowledge about HTTP headers if you found your way here because your boss told you to do this and “figure it out”.

Go to the AWS Console and click into your CloudFront Distribution. Then click on the “Behaviors” tab. If you already have a behavior, you can edit the existing one. Otherwise, create a behavior.

I won’t cover the other Behavior settings as that is out of the scope of this post, but to add a custom response header look for the “Response headers policy - optional” field which is under “Cache key and origin requests”.

Click on the “Create response headers policy” link. On the next page fill in the “Name” field at the very top. Then, scroll a bit down and you should see the “Custom headers - optional” field.

Click on the “Add header” button and add the custom header you’d like. In my case, I added the Cache-Control header with a value of max-age=31536000. I also checked the Origin override box because I want this header to take precedence over other headers that may be set in the origin.

Then click the “Create” button on the bottom right. You’ll be redirected to CloudFront > Policies. You can see your newly created response header under the “Response headers” tab. In my case it’s called cache-control-header.

Now that this custom header policy is created, browse back to your Distribution and click on the “Behaviors” tab. Edit an existing Behavior or create one. Scroll down to the “Response headers policy - optional” field again. Then, select your new response header policy.

Then save your changes.

Wait a few minutes and your distribution will be updated with the new header. There is no need to create an Invalidation.

You can test this out by running curl -IX GET . Your output should look something like this. Look for your custom header in the output. In my case, cache-control shows up at the very bottom.

$ curl -IX GET https://abcdefghijklmn.cloudfront.net

HTTP/2 301
content-length: 0
location: https://nelson.cloud/index.html
date: Wed, 01 Jan 2025 08:12:11 GMT
server: AmazonS3
x-cache: Hit from cloudfront
via: 1.1 98167d64569fd17ca63a5b7db2edfe28.cloudfront.net (CloudFront)
x-amz-cf-pop: LAX54-P1
x-amz-cf-id: l5n0bciPQfswC27jGnhcrkuldHWFDgZx0JdYo406qkoKKKP87i_dsA==
age: 2292
cache-control: max-age=31536000

Done!

References

AWS SSM: Run a Shell Command Against Multiple EC2 Instances

Thu, 18 Jan 2024 00:00:00 +0000

In AWS, you can use AWS Systems Manager to run commands on any EC2 Instances that have the AWS Systems Manager Agent installed. AWS Systems Manager has a lot more functionality than what I’m demonstrating here, but this is a specific use case.

Shell Command Setup

To get started running a shell command on multiple EC2 Instances, head over to AWS Systems Manager via the AWS Console. Then, on the left side under “Node Management”, click on “Run Command”.

On the next screen, click on the orange “Run a Command” button on the right.

Next, under “Command document” search for “shell” to find AWS-RunShellScript and select it.

Then, under “Command parameters” type in the shell command you want to run on your EC2 instances. In my case, I want to run security updates on my instances.

Then select EC2 Instances by your preferred method under “Target Selection”. I chose to manually select my instances but you can also select based on tags or resource groups. Make sure you are using the same region your EC2 instances are in!

This part is optional. If you want log outputs of the commands that are run on every instance you can save logs to a S3 Bucket under “Output options”. Make sure the Instance Profile IAM Role has permissions to write to this bucket!

I left all other settings with their default values.

Now we can click the orange “Run” button on the bottom of the page to execute our command.

After Running a Command

After running a command you’ll see the following screen. You can see the status of the command execution on each EC2 instance to ensure everything went fine. You can also see the Command ID here which will come in handy if you enabled S3 logging.

Viewing Logs

If you enabled logging to S3, you can browse to the bucket you selected and view logs. The output log file will be several directories deep under ///.... It’s easier if you refer to the screenshot below:

I downloaded the stdout file locally and viewed it via the command line to confirm that my command executed properly on this particular EC2 instance:

$ cat stdout

Last metadata expiration check: 0:15:25 ago on Thu Jan 18 21:10:06 2024.
Dependencies resolved.
Nothing to do.
Complete!

Troubleshooting

I personally ran into some issues when trying to do this for the first time. Here are a few troubleshooting solutions.

EC2 Instances Don’t Show Up in Systems Manager

If your EC2 Instances do not show up under “Target selection” they may not have the SSM agent installed. AWS has documentation to guide you through the installation process: Manually installing SSM Agent on EC2 instances for Linux.

If your EC2 Instances do have the SSM agent installed but still aren’t showing up, they may not have an IAM instance profile. You can confirm this by clicking on an EC2 Instance and then checking under the “IAM Role” field. If this field is empty, assign an IAM Role and ensure it has proper permissions. In my case, I used two AWS managed policies AmazonSSMManagedInstanceCore and AmazonSSMPatchAssociation.

(The inline policy in the screenshot is to grant this Role S3 permissions for logging purposes which I cover in the next section.)

No Log Output to S3

If you see no log output in your S3 bucket, you may need to add proper permissions to the instance profile IAM Role of the EC2 instances you selected.

Here is the policy I attached to the IAM instance profile role that got logging to work for me. Change the bucket name my-bucket accordingly!

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19


{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:PutObject",
        "s3:GetBucketAcl",
        "s3:GetBucketLocation",
        "s3:GetObject",
        "s3:ListBucket"
      ],
      "Resource": [
        "arn:aws:s3:::my-bucket",
        "arn:aws:s3:::my-bucket/*"
      ]
    }
  ]
}

Getting EC2 Instance Metadata Using IMDSv2

Fri, 01 Dec 2023 00:00:00 +0000

The EC2 Instance Metadata Service (IMDS) allows us to make an API call within an EC2 instance to retrieve instance metadata, such as a local IP address. There are 2 versions of IMDS.

Using IMDSv1, all we needed to do was to hit the http://169.254.169.254/latest/meta-data/ endpoint to retrieve metadata. I previously created a post with some examples: Getting EC2 Instance Metadata Using IMDSv1.

Using IMDSv2, we now need to make an API call to http://169.254.169.254/latest/api/token to retrieve a token, then include that token in a X-aws-ec2-metadata-token header to hit the metadata endpoint http://169.254.169.254/latest/meta-data/.

During EC2 creation you can configure your instance to use IMDSv2. This is what the setting looks like under “Advanced details” when creating an EC2 Instance. As of December 2023, it defaulted to “V2 only (token required)”:

Once IMDSv2 is enabled on an instance, you can SSH into your instance and start making API calls from within.

Useful Examples

Here are some examples I’ve found useful in my career.

View all available categories of metadata:

TOKEN=`curl -s -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600"` \
&& curl -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/

#---- output ----#
ami-id
ami-launch-index
ami-manifest-path
block-device-mapping/
events/
hostname
iam/
identity-credentials/
instance-action
instance-id
instance-life-cycle
instance-type
local-hostname
local-ipv4
mac
metrics/
network/
placement/
profile
public-hostname
public-ipv4
reservation-id
security-groups
services/

Get instance AMI ID:

TOKEN=`curl -s -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600"` \
&& curl -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/ami-id

#---- output ----#
ami-0230bd60aa48260c6

Get hostname:

TOKEN=`curl -s -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600"` \
&& curl -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/hostname

#---- output ----#
ip-172-31-42-147.ec2.internal

Get instance ID:

TOKEN=`curl -s -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600"` \
&& curl -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/instance-id

#---- output ----#
i-0bca16d8b48523ac7

Get instance type:

TOKEN=`curl -s -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600"` \
&& curl -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/instance-type

#---- output ----#
t2.micro

Get local IPv4 address:

TOKEN=`curl -s -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600"` \
&& curl -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/local-ipv4

#---- output ----#
172.31.42.147

Get AWS account ID:

TOKEN=`curl -s -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600"` \
&& curl -s -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/identity-credentials/ec2/info | grep "AccountId" | awk -F\" '{print $4}'

#---- output ----#
123456789012

Get MAC address:

TOKEN=`curl -s -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600"` \
&& curl -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/mac

#---- output ----#
0e:53:26:7a:45:0b

Get availability zone:

TOKEN=`curl -s -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600"` \
&& curl -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/placement/availability-zone

#---- output ----#
us-east-1c

Get availability zone ID:

TOKEN=`curl -s -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600"` \
&& curl -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/placement/availability-zone-id

#---- output ----#
use1-az6

Get security groups associated with the instance:

TOKEN=`curl -s -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600"` \
&& curl -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/security-groups

#---- output ----#
my-security-group-1
my-security-group-2
my-security-group-3

Bash Script

Copy and paste this Bash snippet and use the assigned values as needed:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23


TOKEN=$(curl -s -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600")

ACCOUNT_ID=$(curl -s -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/identity-credentials/ec2/info | grep "AccountId" | awk -F\" '{print $4}')
AMI_ID=$(curl -s -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/ami-id)
AVAILABILITY_ZONE=$(curl -s -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/placement/availability-zone)
AVAILABILITY_ZONE_ID=$(curl -s -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/placement/availability-zone-id)
HOSTNAME=$(curl -s -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/hostname)
INSTANCE_ID=$(curl -s -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/instance-id)
INSTANCE_TYPE=$(curl -s -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/instance-type)
LOCAL_IPV4=$(curl -s -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/local-ipv4)
MAC_ADDRESS=$(curl -s -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/mac)
SECURITY_GROUPS=$(curl -s -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/security-groups)

echo $ACCOUNT_ID
echo $AMI_ID
echo $AVAILABILITY_ZONE
echo $AVAILABILITY_ZONE_ID
echo $HOSTNAME
echo $INSTANCE_ID
echo $INSTANCE_TYPE
echo $LOCAL_IPV4
echo $MAC_ADDRESS
echo $SECURITY_GROUPS

References


https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-retrieval.html

Redirect One Domain to Another Using AWS S3 and CloudFront

Thu, 16 Feb 2023 00:00:00 +0000

With AWS S3, it’s possible to redirect one domain to another. This is useful when migrating domains.

I recently migrated my domain from nelsonfigueroa.dev to nelson.cloud, so I’ll be using those domains as an example.

This guide assumes that you already have ownership or control of the domain you want to redirect.

Redirecting The Old Domain Using HTTP

This section covers the redirecting of the HTTP version of the old domain. In my case, this section covers how I redirected http://nelsonfigueroa.dev to https://nelson.cloud. There is an additional section you can read if you need to redirect the HTTPS version of the old domain.

Configuring a S3 Bucket

To begin redirecting one domain to another, we need to create an S3 bucket.

Go to Amazon S3 and click the “Create bucket” button. I prefer to name the bucket with the same name as the domain being redirected, but you can choose any name you want.

Click into the bucket and then click on the “Properties” tab. Scroll to the bottom to the “Static website hosting” section and click the “Edit” button.

Set the following configuration:

Under “Static website hosting” click the “Enable” radio button
Under “Hosting type” select “Host a static website”
Under “Index document” type in “index.html”, even though we won’t need an index.html file.

Finally, under the “Redirection rules” text box add the following (change the domain for your needs):

[
    {
        "Redirect": {
            "HostName": "nelson.cloud",
            "Protocol": "https"
        }
    }
]

This rule will redirect to the new domain and append paths as well. So if a user goes to nelsonfigueroa.dev/about they will be redirected to https://nelson.cloud/about.

Tip:
If you want to redirect to HTTP instead of HTTPS, you can change "Protocol" to "http".

Then click the “Save changes” button at the very bottom of the page.

Make a note of the “Static website endpoint” URL at the bottom of the “Properties” tab. Next, we’ll need to point the old domain to this endpoint.

Setting Up DNS for the Old Domain

Next, we’ll need to create a CNAME record that points the old domain to the bucket website endpoint. We can use Route 53 to do this.

Creating a CNAME Record in Route 53

On Amazon Web Services, browse to the Route 53 console.

First, make sure you have a public hosted zone in Route 53. If not, create one and specify the old domain name:

You will need to update the DNS nameservers for your domain specifying the nameservers that AWS provides to you in the hosted zone. This varies depending on the registrar so I will let you do some Googling for this part. It’s pretty easy though :).

Next, create a CNAME record in the new hosted zone that points the old domain to the S3 Bucket Website endpoint.

Tip:
In Route 53 specifically you can create an A record, enable the “Alias” toggle, and then select the S3 Bucket website endpoint. This is an alternative solution.

After the CNAME record is created (and changes have propagated), all requests going to the old domain will be redirected to the new domain.

Warning:
This will only work with HTTP requests to the old domain and not HTTPS requests.

In my case, http://nelsonfigueroa.dev will be redirected to https://nelson.cloud, but https://nelsonfigueroa.dev will time out. This is because there is nothing redirecting the HTTPS version of the old domain.

If this isn’t a problem to you, you are free to stop here. Otherwise, read on.

Redirecting the Old Domain Using HTTPS

To redirect the old domain when using HTTPS we’ll need an ACM SSL Certificate and a CloudFront Distribution. This will allow the HTTPS version of the old domain to be redirected.

Requesting an ACM SSL Certificate

Browse to the AWS Certificate Manager console. Make sure you are in the us-east-1 region by looking at the region on the top-right corner of the screen. Only ACM certificates from the us-east-1 region will work with a CloudFront Distribution.

Once you are sure you’re in the us-east-1 region, click on “Request certificate” on the left-hand sidebar.

For the “Certificate type”, select “Request a public certificate”. Then click the “Next” button:

In the following form, do the following:

For “Fully qualified domain name”, enter the old domain (nelsonfigueroa.dev in my case)
For “Validation method”, select “DNS validation”
For “Key algorithm”, it’s okay to leave it at RSA 2048. You can select a different algorithm if you’d like.

Then click the “Request” button on the bottom of the page.

After the certificate is created we’ll need to validate the certificate (we need to prove to AWS that we own this domain by creating a CNAME record in the hosted zone for this domain). Click into the certificate and scroll down to the “Domains” section. There will be a table with two columns named “CNAME name” and “CNAME value”. You will need to create a CNAME record in Route 53 with this name/value combination following the same procedure as before.

After the CNAME record is created, wait around 15 minutes and the certificate will be validated and you should see “Success” under the “Status” and “Renewal status” columns:

Now we can create a CloudFront Distribution and associate this certificate to it.

Creating a CloudFront Distribution

Browse to the CloudFront console and click on the “Create distribution” button on the top right.

Fill out the form with the following:

For “Origin domain” select the S3 bucket you created for your old domain. Or you can paste in the S3 bucket website endpoint from earlier (this is what I did). Either one of these work for our case.

For “Viewer protocol policy”, select “Redirect HTTP to HTTPS”:

Under “Alternate domain name (CNAME)”, click the “Add item” button and enter the old domain in the field that appears (nelsonfigueroa.dev in my case):

Under “Custom SSL certificate”, select the ACM certificate that you created earlier.

The rest of the settings can remain as is. Click the “Create distribution” button on the bottom of the page. Wait a few minutes for the distribution to deploy out. You’ll know deployment is done when the “Last modified” field displays a date. Make a note of the “Domain name” column for your distribution. You’ll need this value for the next step.

Updating DNS to use the new CloudFront Distribution

The final step is to update the previous record so that it points to the distribution instead of the bucket website endpoint.

Go to Route 53. Click into the hosted zone for the old domain. Then edit the record that was previously created. Instead of entering the S3 Bucket website endpoint in the “Value” field, enter the CloudFront Distribution domain name. Then click “Save”.

Now both http://nelsonfigueroa.dev and https://nelsonfigueroa.dev/ will redirect to the new domain.

Testing the Redirect

You can test this out by entering the old domain in a browser prefixed with https://. Or you can test this out in a terminal with curl.

The HTTP version of the domain should give us a location header redirecting us to https://nelsonfigueroa.dev:

$ curl -IX GET http://nelsonfigueroa.dev

HTTP/1.1 301 Moved Permanently
Server: CloudFront
Date: Thu, 16 Feb 2023 05:55:35 GMT
Content-Type: text/html
Content-Length: 167
Connection: keep-alive
Location: https://nelsonfigueroa.dev/

And the HTTPS version of the domain should then redirect to the new domain https://nelson.cloud:

$ curl -IX GET https://nelsonfigueroa.dev

HTTP/2 301
content-length: 0
location: https://nelson.cloud/index.html
date: Thu, 16 Feb 2023 05:42:44 GMT
server: AmazonS3
x-cache: Miss from cloudfront

As a final test, we can make sure that paths are also redirected to the new domain. In other words, https://nelsonfigueroa.dev/test-path/ should redirect to https://nelson.cloud/test-path/, and we should see this value in the location header:

$ curl -IX GET https://nelsonfigueroa.dev/test-path/

HTTP/2 301
content-length: 0
location: https://nelson.cloud/test-path/
date: Thu, 16 Feb 2023 05:58:52 GMT
server: AmazonS3
x-cache: Miss from cloudfront

It works, we are done!

Hosting a Static Website on AWS Using S3 and CloudFront

Fri, 30 Dec 2022 00:00:00 +0000

Overview

Using Amazon Web Services, we can easily set up a static site worldwide without the need to maintain servers and load balancers.

In this guide, I’ll go over how I set up this site on AWS. I used the following AWS services:

I recently migrated my domain from nelsonfigueroa.dev to nelson.cloud, so I’ll be using this new domain as an example.

This guide assumes the following:

You have already purchased a domain from a registrar like Namecheap, Gandi, or Porkbun.
You have already signed up for an Amazon Web Services account.

Creating an S3 Bucket

First, we’ll need to create an S3 bucket. This is where static files (HTML/CSS/JavaScript/Images, etc) will go.

In the AWS console, browse to S3. Create a new bucket. I named my bucket with the same name as my domain.

Then click on the bucket and go to the “properties” tab. Scroll all the way to the bottom to the “Static website hosting” section. Click the “Edit” button. Now configure this bucket to host a static site:

Under “Static website hosting” select “Enable”
Under “Hosting type” select “Host a static website”
In the “Index document” field, write in “index.html” (unless you want a different root page for your site)
Then click “Save changes”

Then go to the “Permissions” tab in the bucket. Click the “Edit” button under the “Block public access (bucket settings)” section. Uncheck the “Block all public access” checkbox. We need this bucket to have public access so the site is viewable across the internet. Save changes.

Next, still under the “Permissions tab”, click the “Edit” button on the “Bucket policy” section. We need to add permissions for public access. Here is what the policy should look like. Change the bucket name after Resource to your bucket name:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "PublicReadGetObject",
            "Effect": "Allow",
            "Principal": "*",
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::nelson.cloud/*"
        }
    ]
}

Then save changes.

At this point “Block all public access” should be off, and the bucket policy should show under the section:

To verify that everything works, we can upload some HTML files to test out the site.

Go to the “Objects” tab on the bucket and drop in your static files. Even a single index.html file will do. In my case I have a lot more files since my site has blog posts and other things:

Then go to the “Properties” tab and at the very bottom you should see a “Bucket website endpoint”. Click on this link and your site should open in your browser. If you see the contents of your HTML file(s) then you are all done here with S3.

The AWS docs provide more information about S3 permissions for static hosting if you’re curious:

https://docs.aws.amazon.com/AmazonS3/latest/userguide/WebsiteAccessPermissionsReqd.html

Creating an ACM SSL Certificate

Next we need to request an SSL certificate in ACM that we’ll use with CloudFront. This will allow us to set up our site with HTTPS.

In AWS, go to “AWS Certificate Manager (ACM)”.

Warning:
SSL certificates used with CloudFront must be in the us-east-1 region. Check the upper right corner and make sure you are in the us-east-1 region before proceeding.

Then click the “Request” button.

For “Certificate type” select “Request a public certificate”. Then click Next.

Then, under “Fully qualified domain name” write in the name of your domain I also like to add in support for all subdomains. If you want to do this, add in the wildcard subdomain *.yourdomain.com

Under “Validation method” select DNS validation. Under “Key algorithm” select “RSA 2048” or higher if you’d like

Then click the “Request” button.

You’ll see that the new certificate says “Pending validation” under the “Status” column. Click into your certificate.

Under “Domains” you should see the columns “CNAME name” and “CNAME value”. You’ll have to create a CNAME record in Route 53 to validate this certificate.

Note that in my case the CNAME names and values are repeated. I only had to create the record once to validate both domains.

Info:
The CNAME record(s) can also be created on your registrar, but I chose to do it all on AWS to keep things in one place.

Go to Route 53 and create a public hosted zone for your domain (note that at the time of this writing hosted zones are 50 cents per month).

Once the zone is created, go back to your certificate and click on the upper right button that says “Create records in Route 53”. AWS will add the records for you without any manual work on your part.

After the record(s) are created, wait some time until the certificate is validated. In my case, it took around 10 minutes.

This page from the AWS documentation elaborates more on DNS validation for ACM SSL Certificates:

https://docs.aws.amazon.com/acm/latest/userguide/dns-validation.html

Creating a CloudFront Distribution

Next, we need to create a CloudFront distribution. The distribution will use the bucket and ACM certificate we created in order to host our site.

CloudFront is Amazon’s content delivery network (CDN). By deploying our site on CloudFront, our site will be available worldwide with low latency.

First, we’ll need the S3 bucket website endpoint that we clicked on earlier.

Go to the S3 bucket you created. Then, under the “Properties” tab, scroll all the way down and copy the value of “Bucket website endpoint”

Then go to the CloudFront service in the AWS console. Click the “Create distribution” button on the top right.

Under “Origin domain” paste the value of “Bucket website endpoint” that you copied. The value should be something like http://mybucket.s3-website-us-east-1.amazonaws.com

Under “Viewer protocol policy” I like to select “Redirect HTTP to HTTPS”, essentially disallowing HTTP connections.

Under “Alternate domain name (CNAME)”, add in your domain.

Under “Custom SSL certificate” select the ACM certificate that you created earlier. It should show up in the drop-down menu.

Under “default root object” write in index.html or whatever you want your root to be. This is the page that gets loaded by default when hitting your domain without a path (https://nelson.cloud/ in this case).

The rest of the settings can stay as is. Click the “Create distribution” button. Allow some time for the distribution to deploy. Once the “Last modified” field doesn’t say “Deploying” and displays a date, it’s done.

I chose to use the S3 website endpoint as the Origin, but you can also use the S3 bucket itself. I chose the website endpoint because my site has multiple index.html templates which can give errors in CloudFront.

More on Origins in the AWS documentation:

https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/DownloadDistS3AndCustomOrigins.html

Adding a 404 Page to the CloudFront Distribution (Optional)

CloudFront Distributions have barebones and unstylized error pages like this 404 page:

If we want to use our own custom error pages, we need to create a custom error response.

Click into your newly created CloudFront Distribution. Click on the “Error pages” tab and then click the “Create custom error response” button.

Select the following settings:

“HTTP error code” should be “404: Not Found”
Error caching minimum TTL can stay as is (10)
Select “Yes” for “Customize error response”
The “Response page path” should be “/404.html” (or whatever you want your 404 page file name to be).
- You’ll need to create a 404.html template and place it in your S3 Bucket.
Select “404: Not Found” for the “HTTP Response code” field
Click the “Create custom error response” button

Give your distribution a few minutes to update and you’re done here.

There are several other error codes you can account for using the same process. I only added a 404 page since it’s one of the most common.

Pointing a Custom Domain to the CloudFront Distribution

The last step is to point your domain to the cloudfront distribution. For this step, I once again used Route 53, but you can also create a CNAME or ALIAS record in your domain’s registrar and get the same results.

Go to Route 53 again. Click on the Hosted Zone you created earlier. Create a record with the following settings:

Leave the subdomain blank, unless you want your site to be accessible under a subdomain like blog.mysite.com.
For Record type, select “A - Routes traffic to an IPv4 address and some AWS resources.”
Click the “Alias” toggle to enable it.
Under “Route traffic to” select “Alias to CloudFront distribution”
Under the “Choose distribution” field select the distribution you created.
The routing policy can stay as is.

Then click the “Create records” button.

It will take some time for these changes to propagate across DNS servers. In my experience, it’s never been more than around 15 minutes. Usually faster.

Viewing The Live Site

After adding the DNS record and waiting some time, you should be able to go to your domain on a browser and see your site.

You can also test out your custom error page(s). In my case, I browsed to https://nelson.cloud/test to see my custom 404 page:

Congrats! Your site is now live.

Finding AWS Resources by IP Address

Sun, 12 Jun 2022 00:00:00 +0000

Finding EC2 Instances by IP Address

In the EC2 Console

In the AWS EC2 Management Console, you can search for EC2 instances using a private or public IP address. Filter by either Private IP address or Public IPv4 address in the search field:

Using the AWS CLI

The AWS CLI can be used to find EC2 instances by either private or public IP address.

Using a Private IP Address

To find EC2 instances by private IP address, the command looks like this (Replace --region with your region if it’s not set by default. Replace Values with the IP address):

`1`	`aws ec2 describe-instances --region=us-west-1 --filter Name=private-ip-address,Values=10.0.0.1`

Using a Public IP Address

To find EC2 instances by public IP address, the Name filter changes to ip-address but otherwise the command is the same as the one from above:

`1`	`aws ec2 describe-instances --region=us-west-1 --filter Name=ip-address,Values=54.123.45.67`

Specifying Multiple IP Addresses

For either of these commands, you can specify several IP addresses by adding them to the Values filter as such:

For public IP addresses:

`1`	`aws ec2 describe-instances --region=us-west-1 --filter Name=ip-address,Values=54.123.45.67,54.123.45.68,54.123.45.69`

For private IP addresses:

`1`	`aws ec2 describe-instances --region=us-west-1 --filter Name=private-ip-address,Values=10.0.0.1,10.0.0.2,10.0.0.3`

Example Output

The output for these commands will look something like this:

{
    "Reservations": [
        {
            "Groups": [],
            "Instances": [
                {
                    "AmiLaunchIndex": 0,
                    "ImageId": "ami-1234abcd",
                    "InstanceId": "i-001122334455abcd",
                    "InstanceType": "t3.micro",
                    "KeyName": "key-name",
                    "LaunchTime": "2022-01-01T00:00:00.000Z",
                    "Monitoring": {
                        "State": "enabled"
                    },
                    "Placement": {
                        "AvailabilityZone": "us-west-1a",
                        "GroupName": "",
                        "Tenancy": "default"
                    },
                    "PrivateDnsName": "ip-10-0-0-1.ec2.internal",
                    "PrivateIpAddress": "10.0.0.1",
                    "ProductCodes": [],
                    "PublicDnsName": "",
                    "State": {
                        "Code": 16,
                        "Name": "running"
                    },
                    "StateTransitionReason": "",
                    "SubnetId": "subnet-1234abcd",
                    "VpcId": "vpc-1234abcd",
                    "Architecture": "x86_64",
                    "BlockDeviceMappings": [
                        {
                            "DeviceName": "/dev/xvda",
                            "Ebs": {
                                "AttachTime": "2022-01-01T00:00:00.000Z",
                                "DeleteOnTermination": true,
                                "Status": "attached",
                                "VolumeId": "vol-012345abcdef"
                            }
                        }
                    ],
                    "ClientToken": "00000000",
                    "EbsOptimized": false,
                    "Hypervisor": "xen",
                    "IamInstanceProfile": {
                        "Arn": "arn:aws:iam::000000000000:instance-profile/my-instance-profile",
                        "Id": "ABCDEF12345"
                    },
                    "NetworkInterfaces": [
                        {
                            "Attachment": {
                                "AttachTime": "2022-01-01T00:00:00.000Z",
                                "AttachmentId": "eni-attach-abcd1234",
                                "DeleteOnTermination": true,
                                "DeviceIndex": 0,
                                "Status": "attached",
                                "NetworkCardIndex": 0
                            },
                            "Description": "",
                            "Groups": [
                                {
                                    "GroupName": "security-group-1",
                                    "GroupId": "sg-abc123"
                                },
                                {
                                    "GroupName": "security-group-2",
                                    "GroupId": "sg-def456"
                                }
                            ],
                            "Ipv6Addresses": [],
                            "MacAddress": "aa:00:bb:22:cc:33",
                            "NetworkInterfaceId": "eni-abcd1234",
                            "OwnerId": "112233445566",
                            "PrivateDnsName": "ip-10-0-0-1.ec2.internal",
                            "PrivateIpAddress": "10.0.0.1",
                            "PrivateIpAddresses": [
                                {
                                    "Primary": true,
                                    "PrivateDnsName": "ip-10-0-0-1.ec2.internal",
                                    "PrivateIpAddress": "10.0.0.1"
                                }
                            ],
                            "SourceDestCheck": true,
                            "Status": "in-use",
                            "SubnetId": "subnet-abcd1234",
                            "VpcId": "vpc-abcd1234",
                            "InterfaceType": "interface"
                        }
                    ],
                    "RootDeviceName": "/dev/xvda",
                    "RootDeviceType": "ebs",
                    "SecurityGroups": [
                        {
                            "GroupName": "security-group-1",
                            "GroupId": "sg-123abc"
                        },
                        {
                            "GroupName": "security-group-2",
                            "GroupId": "sg-456def"
                        }
                    ],
                    "SourceDestCheck": true,
                    "Tags": [

                        {
                            "Key": "App",
                            "Value": "Testing App"
                        }
                    ],
                    "VirtualizationType": "hvm",
                    "CpuOptions": {
                        "CoreCount": 1,
                        "ThreadsPerCore": 2
                    },
                    "CapacityReservationSpecification": {
                        "CapacityReservationPreference": "open"
                    },
                    "HibernationOptions": {
                        "Configured": false
                    },
                    "MetadataOptions": {
                        "State": "applied",
                        "HttpTokens": "optional",
                        "HttpPutResponseHopLimit": 1,
                        "HttpEndpoint": "enabled",
                        "HttpProtocolIpv6": "disabled",
                        "InstanceMetadataTags": "disabled"
                    },
                    "EnclaveOptions": {
                        "Enabled": false
                    },
                    "PlatformDetails": "Linux/UNIX",
                    "UsageOperation": "RunInstances",
                    "UsageOperationUpdateTime": "2022-01-01T00:00:00.000Z",
                    "PrivateDnsNameOptions": {},
                    "MaintenanceOptions": {
                        "AutoRecovery": "default"
                    }
                }
            ],
            "OwnerId": "112233445566",
            "RequesterId": "112233445566",
            "ReservationId": "r-012345abcdef"
        }
    ]
}

Finding Other Resources by IP Address

In the AWS Console

To identify other AWS resources (such as Lambdas) based on IP address, you can search for the corresponding ENI. In the AWS Console, browse to the EC2 console and click on Network Interfaces on the left hand side. Then search by “Primary private IPv4 address” (or “Public IPv4 address” if you want to search by a public IP address).

You can then poke around through the ENI details to figure out what resource is associated with the IP address.

Using the AWS CLI

This can also be done using the AWS CLI with the following command, replacing --region and Values as needed:

`1`	`aws ec2 describe-network-interfaces --region=us-west-1 --filters Name=addresses.private-ip-address,Values=10.0.0.1`

Here’s what the output looks like

{
    "NetworkInterfaces": [
        {
            "Attachment": {
                "AttachmentId": "ela-attach-12345abcde",
                "DeleteOnTermination": false,
                "DeviceIndex": 1,
                "InstanceOwnerId": "amazon-aws",
                "Status": "attached"
            },
            "AvailabilityZone": "us-west-1a",
            "Description": "Test AWS Lambda",
            "Groups": [
                {
                    "GroupName": "test-group",
                    "GroupId": "sg-01234abcde"
                }
            ],
            "InterfaceType": "lambda",
            "Ipv6Addresses": [],
            "MacAddress": "00:aa:11:bb:22:cc",
            "NetworkInterfaceId": "eni-012345abcdef",
            "OwnerId": "112233445566",
            "PrivateDnsName": "ip-10-0-0-1.ec2.internal",
            "PrivateIpAddress": "10.0.0.1",
            "PrivateIpAddresses": [
                {
                    "Primary": true,
                    "PrivateDnsName": "ip-10-0-0-1.ec2.internal",
                    "PrivateIpAddress": "10.0.0.1"
                }
            ],
            "RequesterId": "AAABBBCCCDDDEEEFFF:112233445566",
            "RequesterManaged": false,
            "SourceDestCheck": true,
            "Status": "in-use",
            "SubnetId": "subnet-1234abcd",
            "TagSet": [
                {
                    "Key": "App",
                    "Value": "Lambda Test"
                }
            ],
            "VpcId": "vpc-1234abcd"
        }
    ]
}

Check the value of InterfaceType for clues as to what resource is using the ENI. In this case, it’s a Lambda function.

References

Replacing AWS ACM SSL Certificates With No Downtime

Sun, 12 Jun 2022 00:00:00 +0000

I recently learned that it is possible to replace an existing ACM SSL certificate on any AWS resource with no downtime. The key is that the old certificate must exist while making SSL certificate updates to a resource. AWS does not allow users to delete a certificate that has resources associated. That means there is no risk of an in-use SSL certificate from being missing.

Here’s what a resource associated to an ACM SSL certificate looks like. In this case it’s a CloudFront Distribution:

If I wanted to replace the SSL certificate on the CloudFront Distribution above, I would create a new certificate and associate it with the Distribution. The updating process is then handled automatically by AWS. Since the old certificate still exists, there would be no downtime during the update. After updates are finished, I would be free to delete the old certificate.

This applies to any AWS resource that can be associated with an ACM SSL certificate.

Invoking Amazon API Gateway with an API Key

Mon, 06 Jun 2022 00:00:00 +0000

To invoke an Amazon API Gateway with an API Key we need to pass in the API key in a x-api-key header. For the following examples, assume the invoke URL is https://12abcde45.execute-api.us-west-1.amazonaws.com/prod/create and the API key is abc123.

Invoking with curl

To invoke this API with curl it would look like this:

1
2

# GET request
curl --header "x-api-key: abc123" https://12abcde45.execute-api.us-west-1.amazonaws.com/prod/create

1
2

# POST request with data
curl -d "key1=value1&key2=value2" --header "x-api-key: abc123" -X POST https://12abcde45.execute-api.us-west-1.amazonaws.com/prod/create

Invoking with HTTPie

To invoke the API with HTTPie:

1
2

# GET request
http https://12abcde45.execute-api.us-west-1.amazonaws.com/prod/create x-api-key:abc123

1
2

# POST request with data
http post https://12abcde45.execute-api.us-west-1.amazonaws.com/prod/create key1=value1 x-api-key:abc123

Invoking with Python

GET request:

import requests

url = "https://12abcde45.execute-api.us-west-1.amazonaws.com/prod/create"

# API key specified as a header
# Key hardcoded for demonstrational purposes. Do not push/commit plaintext keys!
headers = {"x-api-key": "abc123"}

# GET request with custom header
response = requests.get(url, headers=headers)
print(response.status_code)

POST request with data:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14


import requests

url = "https://12abcde45.execute-api.us-west-1.amazonaws.com/prod/create"

# API key specified as a header
# Key hardcoded for demonstrational purposes. Do not push/commit plaintext keys!
headers = {"x-api-key": "abc123"}

# Data to be sent
data = {"key1": "value1"}

# POST request with custom header and data
response = requests.post(url, json=data, headers=headers)
print(response.status_code)

Getting EC2 Instance Metadata Using IMDSv1

Tue, 08 Feb 2022 00:00:00 +0000

Sometimes I need to get information about an EC2 instance within a script but the AWS documentation doesn’t provide many useful examples. Instead of poking around through all the available metadata endpoints, I made this list of curl commands to retrieve commonly used EC2 information. Run any of these commands within an EC2 instance. These commands have been tested on an Amazon Linux 2 instance.

These are not all the possible values you can retrieve from the metadata service, these are only the ones I found most useful.

Warning:
This post covers Instance Metadata Service Version 1 (IMDSv1). For security purposes, it is recommended that you use IMDSv2. I created a separate post covering IMDSv2 here: Getting EC2 Instance Metadata Using IMDSv2

If you want a bash script that you can copy and paste, scroll down to the bottom of this article.

Metadata Retrieval with curl

View all available categories of metadata:

$ curl -s http://169.254.169.254/latest/meta-data/

ami-id
ami-launch-index
ami-manifest-path
block-device-mapping/
events/
hibernation/
hostname
iam/
identity-credentials/
instance-action
instance-id
instance-life-cycle
instance-type
local-hostname
local-ipv4
mac
metrics/
network/
placement/
profile
public-keys/
reservation-id
security-groups
services/

Get instance AMI ID:

1
2
3

$ curl -s http://169.254.169.254/latest/meta-data/ami-id

ami-0d43f810ac49e9511

Get hostname:

1
2
3

$ curl -s http://169.254.169.254/latest/meta-data/hostname

ip-10-128-128-128.us-west-1.compute.internal

Get AWS account ID:

1
2
3

$ curl -s http://169.254.169.254/latest/meta-data/identity-credentials/ec2/info | grep "AccountId" | awk -F\" '{print $4}'

123456789012

Get instance ID:

1
2
3

$ curl -s http://169.254.169.254/latest/meta-data/instance-id

i-01234567890f1234b

Get instance type:

1
2
3

$ curl -s http://169.254.169.254/latest/meta-data/instance-type

t2.medium

Get IPv4 address:

1
2
3

$ curl -s http://169.254.169.254/latest/meta-data/local-ipv4

10.128.128.128

Get MAC address:

1
2
3

$ curl -s http://169.254.169.254/latest/meta-data/mac

05:5f:bd:1a:4c:77

Get availability zone:

1
2
3

$ curl -s http://169.254.169.254/latest/meta-data/placement/availability-zone

us-west-1a

Get availability zone ID:

1
2
3

$ curl -s http://169.254.169.254/latest/meta-data/placement/availability-zone-id

usw1-az1

Get security groups associated with the instance:

$ curl -s http://169.254.169.254/latest/meta-data/security-groups

my-security-group-1
my-security-group-2
my-security-group-3

Bash Script

Copy and paste this bash snippet and use values as needed:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21


AMI_ID=$(curl -s http://169.254.169.254/latest/meta-data/ami-id)
HOSTNAME=$(curl -s http://169.254.169.254/latest/meta-data/hostname)
ACCOUNT_ID=$(curl -s http://169.254.169.254/latest/meta-data/identity-credentials/ec2/info | grep "AccountId" | awk -F\" '{print $4}')
INSTANCE_ID=$(curl -s http://169.254.169.254/latest/meta-data/instance-id)
INSTANCE_TYPE=$(curl -s http://169.254.169.254/latest/meta-data/instance-type)
LOCAL_IPV4=$(curl -s http://169.254.169.254/latest/meta-data/local-ipv4)
MAC_ADDRESS=$(curl -s http://169.254.169.254/latest/meta-data/mac)
AVAILABILITY_ZONE=$(curl -s http://169.254.169.254/latest/meta-data/placement/availability-zone)
AVAILABILITY_ZONE_ID=$(curl -s http://169.254.169.254/latest/meta-data/placement/availability-zone-id)
SECURITY_GROUPS=$(curl -s http://169.254.169.254/latest/meta-data/security-groups)

echo $AMI_ID
echo $HOSTNAME
echo $ACCOUNT_ID
echo $INSTANCE_ID
echo $INSTANCE_TYPE
echo $LOCAL_IPV4
echo $MAC_ADDRESS
echo $AVAILABILITY_ZONE
echo $AVAILABILITY_ZONE_ID
echo $SECURITY_GROUPS

Resolving AWS CloudFront Access Denied Errors

Sun, 25 Jul 2021 00:00:00 +0000

Info:
Note that Origin Access Identity has since been deprecated by AWS. The replacement is Origin Access Control. I have no idea if that fixes the issue I talk about below, but it’s something to be aware of.

If you’re seeing “Access Denied” errors on CloudFront and the official troubleshooting docs aren’t helping, you might be running into the following issue.

I discovered that “Access Denied” errors may show up when a CloudFront Distribution is set up under the following conditions:

A private S3 Bucket is being used along with a S3 Origin in the CloudFront Distribution
An Origin Access Identity is being used. (This is necessary if the bucket is private. When a user hits the distribution, the Origin Access Identity retrieves files from the bucket and forwards them to the distribution and to the end user. The end user never accesses the bucket directly.)
Multiple index.html templates exist in the bucket

An Origin Access Identity is useful because this allows S3 contents to be accessible only through CloudFront. However, it doesn’t work well when there are multiple index.html templates in the bucket and we end up seeing “Access Denied” errors when accessing any sub-pages. This is unfortunate for those like me that use Hugo to generate static sites. Hugo creates several index.html templates when building a site.

The solution is to either:

Use a public bucket.
Enable static website hosting on the bucket (which means the Distribution will have a Custom Origin).
Avoid using an Origin Access Identity.
Acknowledge that users will be able to access S3 contents directly without HTTPS.

Use a private bucket.
Don’t enable static website hosting on the bucket (the Distribution will have a S3 Origin).
Use an Origin Access Identity so that users cannot access S3 contents directly and can only view contents via CloudFront.
Avoid having multiple index.html templates in the bucket, excluding the root index.html template, which could be a hurdle.

I was surprised that this isn’t documented in Amazon’s own troubleshooting guide. I only learned about this issue through a comment on a forum (which has now been archived) that reads as follows:

Quote:
"…CloudFront provides default root object support as well, but not for any subdirectories. You can solve this by using a custom origin instead. When CloudFront uses the S3 static website URL as the origin, you get the desired functionality."

This seems like a limitation of CloudFront. While this issue won’t affect me too much, there are others who would not want their buckets to be publicly accessible. For security purposes, it’s best if users are only able to access a site through the distribution and not directly from the bucket, especially since S3 static website hosting does not support HTTPS. Hopefully AWS fixes this flaw.

AWS IAM: Allowing a Role to Assume Another Role

Sat, 19 Jun 2021 00:00:00 +0000

To allow an IAM Role to assume another Role, we need to modify the trust relationship of the role that is to be assumed. This process varies depending if the roles exist within the same account or if they’re in separate accounts.

Roles in the Same Account

Let’s say we have two roles, Role_A and Role_B. If we want to allow Role_A to assume Role_B, we need to modify the trust relationship of Role_B with the following:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::111111111111:role/Role_A"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

This is all that’s needed to allow a role to assume another role within the same account.

Note:
Be mindful of the Principal element where we specify the role that we want to give permissions to. In general, the Principal element is used in policies to give users/roles/services access to other AWS resources. However, the Principal element cannot be used in policies attached to Roles. It can only exist in the trust relationships of roles (you’ll get errors if you try to use the Principal element in an IAM Role policy). You can read more about this element in the AWS docs.

Roles in Different Accounts

Let’s say Role_A and Role_B are in different accounts. In this case, the process from above stays the same. Role_B needs to have its trust relationship modified to allow Role_A to assume it. The difference here is that Role_A will need an additional policy with sts:AssumeRole permissions. So the final result is as follows:

The Role_B trust relationship stays the same:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::111111111111:role/Role_A"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

And Role_A needs the following attached as a policy:

{
  "Version": "2012-10-17",
  "Statement": {
    "Effect": "Allow",
    "Action": "sts:AssumeRole",
    "Resource": "arn:aws:iam::222222222222:role/Role_B"
  }
}

Now Role_A will be able to assume Role_B even if they’re in different accounts.