The question CloudTrail should answer, and often doesn’t
There’s an m5.2xlarge in the shared services account called test-2. It has no tags except Name, it’s been running for over a year, and nobody in the platform channel admits to it. Someone suggests checking CloudTrail. Event history comes back empty, because the instance is older than 90 days. Or it comes back with a RunInstances event whose user is AWSReservedSSO_AdministratorAccess_4f1c..., or the Auto Scaling service role, or a role session called botocore-session-1692304152, none of which is a person you can message.
CloudTrail does record who created a resource. Getting from that record to a person you can ask takes a few more steps. This guide covers them in about 30 minutes, using read-only access and the logs you probably already have.
What you need:
- Read-only access to the account that owns the resource. The AWS managed
ReadOnlyAccesspolicy includescloudtrail:LookupEvents. - AWS CLI v2 and
jq, with one named profile per account. - For anything older than 90 days: a CloudTrail trail that delivers to S3 (an organization trail is ideal), read access to that bucket, and Amazon Athena.
ReadOnlyAccessisn’t enough to run Athena: you also needathena:StartQueryExecution, permission to create a table in the Glue Data Catalog, and write access to an Athena query results location.
Nothing in this guide changes, tags or deletes a resource. The only thing it creates is an Athena table definition in Step 4, which points at logs you already store.
Why the console and tags don’t get you there
Most AWS resources have no “created by” field. Here’s where the obvious places run out.
Event history only goes back 90 days, one account and one Region at a time. It’s free and on in every account, but it covers management events for the last 90 days, in the Region where the call happened (global services such as IAM are usually recorded in US East (N. Virginia)). The resources that cause the most confusion are usually older than that.
The identity is often a role, not a person. Most people reach AWS through IAM Identity Center, a SAML identity provider, CI/CD or an assumed role. CloudTrail records the role session that made the call. Sometimes the session name is the person, sometimes it’s a random string.
The caller is often a service. Auto Scaling, CloudFormation, EKS node groups and Karpenter launch instances on someone’s behalf. The RunInstances event is real, but the useful answer is “who configured the thing that launched it”.
aws:createdBy only helps going forward. AWS can apply an aws:createdBy cost allocation tag, but only after someone in the management account activates it, only to resources created after that, only for a listed set of resource types and Regions, and only in billing data. It won’t appear in Tag Editor or on the resource.
CloudTrail Lake isn’t an option for most new readers. AWS closed CloudTrail Lake to new customers on May 31, 2026. If you already have an event data store, you can keep using it (there’s a query for it in Step 4). If you don’t, S3 and Athena are the path.
The 30-minute plan
| Time | Step | Output |
|---|---|---|
| 0 to 5 min | Find the creation time and any tags that name what created it | A creation date, and whether you need event history or Athena |
| 5 to 10 min | Look up the creation event in CloudTrail event history | The userIdentity of the creating call |
| 10 to 20 min | Decode that identity: SSO session, assumed role, service, CI | A person, a pipeline or a stack |
| 20 to 30 min | If it’s older than 90 days, query your trail’s S3 logs with Athena | The same answer, from any date your logs cover |
Step 1: Work out when it was created and what created it (0 to 5 minutes)
Before you search any logs, check two things: how old the resource is, and whether AWS already tagged it with the thing that created it.
For EC2, there’s a catch. LaunchTime is the time the instance was last started, not first launched. Stop and start an instance and LaunchTime moves. AWS’s own API reference says to use the attachment time of the primary network interface for the original launch:
aws ec2 describe-instances --instance-ids i-0abc123def4567890 \
--query "Reservations[].Instances[].{
last_start: LaunchTime,
first_launch: NetworkInterfaces[?Attachment.DeviceIndex==\`0\`].Attachment.AttachTime | [0],
tags: Tags}" \
--output json
Then look at the tags. Even untagged instances often carry tags AWS or a controller added at launch, and these point at the real creator:
| Tag on the resource | What created it | Who to look for next |
|---|---|---|
aws:cloudformation:stack-name | A CloudFormation stack (including CDK and SAM) | Whoever created or last updated that stack |
aws:autoscaling:groupName | An Auto Scaling group | Whoever created the group or its launch template |
aws:ec2launchtemplate:id | A launch template | The template’s creator, and whatever launched from it |
aws:ec2:fleet-id | An EC2 Fleet request | The caller of CreateFleet |
eks:cluster-name, eks:nodegroup-name | An EKS managed node group | The node group’s creator, usually the cluster owner |
karpenter.sh/nodepool | Karpenter, running in an EKS cluster | The cluster owner (eks:eks-cluster-name names the cluster). The CreateFleet call is made by Karpenter’s own role |
If one of these is present, you’re looking for the creator of the stack, group or cluster, not the instance. Those parents usually live longer, and their creation events are easier to find.
Now compare the creation date with today. If it’s within the last 90 days, go to Step 2. If it’s older, go straight to Step 4.
Other resource types have their own creation timestamps: CreateTime on EBS volumes, StartTime on snapshots, InstanceCreateTime on RDS instances, CreatedTime on load balancers, CreationDate on S3 buckets. Treat the S3 date with care: AWS notes that it can change when you change the bucket, for example by editing its bucket policy. Elastic IPs don’t expose an allocation time, so you’ll be relying on CloudTrail.
Step 2: Search CloudTrail event history (5 to 10 minutes)
Event history can filter on one attribute per call. ResourceName with the resource ID is usually the quickest. Run it in the resource’s Region:
aws cloudtrail lookup-events --region us-east-2 \
--lookup-attributes AttributeKey=ResourceName,AttributeValue=i-0abc123def4567890 \
--query "Events[?EventName=='RunInstances'].CloudTrailEvent" \
--output json |
jq -r '.[] | fromjson | {
time: .eventTime,
type: .userIdentity.type,
arn: .userIdentity.arn,
invokedBy: .userIdentity.invokedBy,
sourceIdentity: .userIdentity.sessionContext.sourceIdentity,
accessKeyId: .userIdentity.accessKeyId,
sourceIP: .sourceIPAddress,
userAgent: .userAgent }'
A few things to know about this call:
RunInstanceshappens once per instance, at launch. Starting a stopped instance is a separateStartInstancesevent, so if you seeRunInstances, that’s the launch.- The full event is in
CloudTrailEvent, a JSON string. TheUsernamefield in the summary is a shortcut. For assumed roles it’s the session name, which you’ll want, but the fulluserIdentitytells you much more. - Lookups are limited to two requests per second per account per Region, and each page holds up to 50 events. The CLI paginates for you; on a busy resource, add
--start-timeand--end-timearound the creation date from Step 1. userAgentis a useful hint about how the call was made: a browser string means the console,aws-cli/...means the CLI, and Terraform and SDKs identify themselves too.
The creation event depends on the resource type. These are the ones you’ll look for most:
| Resource | Creation event | Event source |
|---|---|---|
| EC2 instance | RunInstances (or CreateFleet for EC2 Fleet and Karpenter) | ec2.amazonaws.com |
| EBS volume / snapshot | CreateVolume / CreateSnapshot | ec2.amazonaws.com |
| Elastic IP | AllocateAddress | ec2.amazonaws.com |
| NAT gateway | CreateNatGateway | ec2.amazonaws.com |
| Load balancer (ALB/NLB) | CreateLoadBalancer | elasticloadbalancing.amazonaws.com |
| RDS instance | CreateDBInstance | rds.amazonaws.com |
| S3 bucket | CreateBucket | s3.amazonaws.com |
| Auto Scaling group | CreateAutoScalingGroup | autoscaling.amazonaws.com |
| CloudFormation stack | CreateStack | cloudformation.amazonaws.com |
If a ResourceName lookup comes back empty for a resource you know is recent, the creating event may not list it as a resource (this filter only matches what the event records in its resource list). Fall back to the event name, and search the event body for the ID:
aws cloudtrail lookup-events --region us-east-2 \
--lookup-attributes AttributeKey=EventName,AttributeValue=CreateFleet \
--start-time 2026-09-01T00:00:00Z --end-time 2026-09-03T00:00:00Z \
--query "Events[].CloudTrailEvent" --output json |
jq -r '.[] | select(contains("i-0abc123def4567890")) | fromjson |
[.eventTime, .userIdentity.arn] | @tsv'
If you don’t know which Region the resource was created in (an S3 bucket, say), loop over the enabled Regions. The multi-account script later in this guide does that for you.
Step 3: Turn the identity into a person (10 to 20 minutes)
You now have a userIdentity. Use its type, its ARN and invokedBy to work out what you’re looking at:
| What you see | What it means | How to get to a person |
|---|---|---|
type: IAMUser, arn: ...:user/jsmith | A long-lived IAM user | That’s your answer. Check the user still exists and who uses its keys |
AssumedRole, role name starts AWSReservedSSO_ | An IAM Identity Center user through a permission set | The session name, after the last /, is normally their Identity Center username (often an email) |
AssumedRole, any other role, readable session name | A role assumed by a person or a tool that sets a meaningful name | Session name, then sourceIdentity if your roles require it |
AssumedRole, session name like botocore-session-... or a random ID | A role assumed by code or a chain of roles | Find the AssumeRole call that issued the credentials (below) |
invokedBy: autoscaling.amazonaws.com, role AWSServiceRoleForAutoScaling | Auto Scaling launched it | Look up CreateAutoScalingGroup for the group name in the instance’s tags |
invokedBy: cloudformation.amazonaws.com | A CloudFormation stack made the call | Look up CreateStack and UpdateStack for the stack in the instance’s tags |
type: Root | The account’s root user | Ask whoever holds the root credentials. Then treat it as a security finding |
SSO sessions
For an Identity Center user, the ARN looks like this:
arn:aws:sts::111122223333:assumed-role/AWSReservedSSO_PowerUser_4f1c2a9b8e7d6c5a/jsmith@example.com
The role name tells you the permission set (PowerUser) and the session name tells you the user. If your usernames aren’t readable, look the user up in the identity store. You need access to the Identity Center instance, which usually means the management account or a delegated administrator:
aws sso-admin list-instances --query "Instances[].IdentityStoreId" --output text
aws identitystore get-user-id --identity-store-id d-1234567890 \
--alternate-identifier '{"UniqueAttribute":{"AttributePath":"userName","AttributeValue":"jsmith@example.com"}}'
aws identitystore describe-user --identity-store-id d-1234567890 \
--user-id 544894e8-80c1-707f-60e3-3ba6510dfac1 \
--query "{name: DisplayName, user: UserName, emails: Emails[].Value}"
One related change: since July 2025, the sign-in and portal events that Identity Center itself emits identify users by onBehalfOf.userId rather than a username. That doesn’t change the RunInstances event in the member account.
Roles with unhelpful session names
When the session name tells you nothing, find out who obtained the credentials. The creating event has an accessKeyId starting with ASIA: temporary credentials, issued by a call to AssumeRole, AssumeRoleWithSAML or AssumeRoleWithWebIdentity. That issuing call has the same key in its responseElements.
In event history, search the role’s account for AssumeRole calls shortly before the creation time and match the key. For federated sessions, the event name is AssumeRoleWithSAML or AssumeRoleWithWebIdentity instead:
aws cloudtrail lookup-events --region us-east-1 \
--lookup-attributes AttributeKey=EventName,AttributeValue=AssumeRole \
--start-time 2026-09-02T13:00:00Z --end-time 2026-09-02T14:10:00Z \
--query "Events[].CloudTrailEvent" --output json |
jq -r '.[] | fromjson |
select(.responseElements.credentials.accessKeyId == "ASIAEXAMPLEKEYID1234") |
[.eventTime, .userIdentity.type, (.userIdentity.arn // .userIdentity.accountId), .requestParameters.roleSessionName] | @tsv'
Check the Region you search. STS calls are logged in the Region of the STS endpoint that served them, and calls to the global endpoint, sts.amazonaws.com, are logged in US East (N. Virginia). For a cross-account assumption, the call is logged in both the caller’s account (with the caller’s identity) and the role’s account (where the caller often shows only as an account ID). If the caller was itself a role, repeat the step. That’s role chaining, and each hop has its own AssumeRole.
Three patterns come up often:
- CI/CD with OIDC.
AssumeRoleWithWebIdentityevents includesubjectFromWebIdentityTokeninresponseElements. For GitHub Actions, that subject names the repository and the branch or environment, such asrepo:octo-org/octo-repo:ref:refs/heads/main, which tells you which pipeline created the resource. Repositories created after July 15, 2026 also include numeric owner and repository IDs, as inrepo:octo-org@123456/octo-repo@456789:ref:refs/heads/main. - SAML federation without Identity Center.
AssumeRoleWithSAMLevents carry the user’s SAML subject inuserIdentity.userName. sourceIdentity. If your role trust policies require a source identity (allowsts:SetSourceIdentityand set a condition on thests:SourceIdentitykey), the original user’s identity follows every hop of a role chain and appears inuserIdentity.sessionContext.sourceIdentity. If you set one control after reading this guide, make it this one.
Services that launched it on someone’s behalf
For Auto Scaling, CloudFormation and EKS, repeat Step 2 for the parent. Look up CreateAutoScalingGroup with ResourceName set to the group name, or CreateStack with the stack name. CloudFormation also keeps its own record of the stack:
aws cloudformation describe-stacks --stack-name payments-api-prod \
--query "Stacks[].{created: CreationTime, updated: LastUpdatedTime, role: RoleARN, tags: Tags}"
If the stack uses a service role (RoleARN), the resources were created under that role, and the person who ran CreateStack or UpdateStack is the one to ask. If RoleARN is empty, CloudFormation made the calls with the credentials of whoever ran the stack operation, so the ARN on the creation event is already that person or pipeline.
Step 4: Go past 90 days with Athena (20 to 30 minutes)
If the resource is older than 90 days, you need the copy of your CloudTrail logs in S3. First, check that a trail exists and where it writes:
aws cloudtrail describe-trails \
--query "trailList[].[Name, S3BucketName, S3KeyPrefix, IsOrganizationTrail, IsMultiRegionTrail, HomeRegion]" \
--output table
aws cloudtrail get-trail-status --name org-trail --query "{logging: IsLogging, latest: LatestDeliveryTime}"
Then check how long the bucket keeps logs. A lifecycle rule that expires objects after 365 days means you can’t see further back than that:
aws s3api get-bucket-lifecycle-configuration --bucket my-org-cloudtrail-logs
If there’s no trail, or it was created after the resource, the record is gone. Mark it “creator unknown”, fall back to the ownership signals in our unowned resources guide, and create an organization trail now so it doesn’t happen again.
Create the table
Athena’s documentation includes a CREATE TABLE statement for CloudTrail logs that uses partition projection, so you don’t have to load partitions by hand. The version below adapts it for an organization trail, whose objects live under AWSLogs/<org-id>/<account-id>/CloudTrail/<region>/YYYY/MM/DD/, by projecting account and Region as well as date. It keeps only the columns this guide needs, and adds sessioncontext.sourceidentity. Replace the bucket, prefix, organization ID, account list and Region list with your own:
CREATE EXTERNAL TABLE cloudtrail_org (
eventversion STRING,
useridentity STRUCT<
type: STRING, principalid: STRING, arn: STRING, accountid: STRING,
invokedby: STRING, accesskeyid: STRING, username: STRING,
sessioncontext: STRUCT<
attributes: STRUCT<mfaauthenticated: STRING, creationdate: STRING>,
sessionissuer: STRUCT<type: STRING, principalid: STRING, arn: STRING, accountid: STRING, username: STRING>,
sourceidentity: STRING>>,
eventtime STRING,
eventsource STRING,
eventname STRING,
awsregion STRING,
sourceipaddress STRING,
useragent STRING,
errorcode STRING,
requestparameters STRING,
responseelements STRING,
eventid STRING,
readonly STRING,
resources ARRAY<STRUCT<arn: STRING, accountid: STRING, type: STRING>>,
recipientaccountid STRING
)
PARTITIONED BY (account STRING, region STRING, `timestamp` STRING)
ROW FORMAT SERDE 'org.apache.hive.hcatalog.data.JsonSerDe'
STORED AS INPUTFORMAT 'com.amazon.emr.cloudtrail.CloudTrailInputFormat'
OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
LOCATION 's3://my-org-cloudtrail-logs/AWSLogs/o-exampleorgid/'
TBLPROPERTIES (
'projection.enabled'='true',
'projection.account.type'='enum',
'projection.account.values'='111122223333,222233334444,333344445555',
'projection.region.type'='enum',
'projection.region.values'='us-east-1,us-east-2,us-west-2,eu-west-1',
'projection.timestamp.type'='date',
'projection.timestamp.format'='yyyy/MM/dd',
'projection.timestamp.interval'='1',
'projection.timestamp.interval.unit'='DAYS',
'projection.timestamp.range'='2022/01/01,NOW',
'storage.location.template'='s3://my-org-cloudtrail-logs/AWSLogs/o-exampleorgid/${account}/CloudTrail/${region}/${timestamp}'
);
For a single-account trail, drop the organization ID from both paths. If your trail uses a key prefix, add it before AWSLogs.
Query for the creator
Athena charges by the amount of data scanned, so always filter on the partition columns. Start with a window of a few days around the creation date from Step 1:
SELECT eventtime, account, region, eventname,
useridentity.type, useridentity.arn, useridentity.invokedby,
useridentity.sessioncontext.sourceidentity AS source_identity,
useridentity.accesskeyid, useragent
FROM cloudtrail_org
WHERE account = '333344445555'
AND region = 'us-east-2'
AND "timestamp" BETWEEN '2025/06/10' AND '2025/06/14'
AND eventname IN ('RunInstances', 'CreateFleet')
AND responseelements LIKE '%i-0abc123def4567890%'
ORDER BY eventtime;
The same query works for any resource: change the event names and the ID. To follow a temporary key back to the AssumeRole that issued it, as in Step 3, search every Region of the account for that day, since STS calls may be logged in a different Region from the resource:
SELECT eventtime, account, region, eventname, useridentity.type, useridentity.arn,
json_extract_scalar(requestparameters, '$.roleArn') AS role_arn,
json_extract_scalar(requestparameters, '$.roleSessionName') AS session_name
FROM cloudtrail_org
WHERE account = '333344445555'
AND "timestamp" = '2025/06/12'
AND eventname LIKE 'AssumeRole%'
AND responseelements LIKE '%ASIAEXAMPLEKEYID1234%';
You can run these in the Athena console, or from the CLI with aws athena start-query-execution --query-string "..." --work-group primary followed by aws athena get-query-results --query-execution-id <id>.
If you already use CloudTrail Lake
Existing CloudTrail Lake customers can query an event data store directly. AWS publishes a sample query for exactly this. The version below returns the full ARN rather than the principal ID:
aws cloudtrail list-event-data-stores --query "EventDataStores[].[Name, EventDataStoreArn, RetentionPeriod]" --output table
aws cloudtrail start-query --query-statement "
SELECT userIdentity.arn, eventName, eventTime, recipientAccountId, awsRegion
FROM <event-data-store-id>
WHERE eventName = 'RunInstances'
AND element_at(responseElements, 'instancesSet') LIKE '%\"instanceId\":\"i-0abc123def4567890\"%'
AND eventTime > '2025-06-10 00:00:00' AND eventTime < '2025-06-14 00:00:00'"
aws cloudtrail get-query-results --query-id <query-id>
Running it across accounts
When you have a list of resources from several accounts, or you don’t know which Region something was created in, script the event history lookup. This one is read-only. For each profile and each enabled Region, it searches for a resource ID and prints any creation-style events it finds:
#!/usr/bin/env bash
# who-created.sh: read-only CloudTrail event history search for a resource's creator.
# Usage: ./who-created.sh <resource-id-or-name> profile-a profile-b ... > creator.tsv
set -euo pipefail
resource=$1; shift
printf 'account\tregion\ttime\tevent\tidentity_type\tidentity\tinvoked_by\n'
for profile in "$@"; do
account=$(aws sts get-caller-identity --profile "$profile" --query Account --output text)
regions=$(aws ec2 describe-regions --profile "$profile" --region us-east-1 \
--query 'Regions[].RegionName' --output text)
for region in $regions; do
aws cloudtrail lookup-events --profile "$profile" --region "$region" \
--lookup-attributes "AttributeKey=ResourceName,AttributeValue=${resource}" \
--query "Events[].CloudTrailEvent" --output json 2>/dev/null |
jq -r --arg a "$account" --arg r "$region" '.[] | fromjson |
select(.eventName | test("^(Run|Create|Allocate|Request|Import|Copy|Register)")) |
[$a, $r, .eventTime, .eventName, .userIdentity.type,
(.userIdentity.arn // .userIdentity.accountId // "unknown"),
(.userIdentity.invokedBy // "")] | @tsv' ||
echo "warning: lookup failed in $account/$region" >&2
done
done
Event history is throttled at two lookups per second per account and Region, which is fine for a handful of resources. For hundreds, query the organization trail in Athena instead: one query covers every account, and you can join it against a list of IDs.
Safe next steps once you have a name
Finding the creator is a lead, not a verdict. Treat it that way.
- The creator isn’t always the owner. Someone in the platform team may have launched an instance for the data team, or the creator may have moved teams. Ask them who it belongs to now, not just whether they made it.
- Record the answer on the resource. Once someone confirms ownership, add your
Ownertag and note how you know. Next time, nobody has to go back to the logs. - If the creator has left, go to their old team. Their manager or their team’s channel is the next stop. Don’t delete a resource just because its creator is gone. Follow the claim-and-stop process in our unowned resources guide.
- Treat root and unexplained identities as security findings. A resource created by the root user, by an IAM user nobody recognizes or from an unexpected IP address is worth raising with whoever handles security before you touch it.
- Keep the evidence. Save the event ID and the query you ran, in case the answer is disputed later.
Why this doesn’t stay done
You can answer “who created this?” for one resource in half an hour. The trouble is that it comes up again for the next resource, and the trail gets colder every day. Event history drops events after 90 days, S3 lifecycle rules expire logs, people change teams and leave, and roles get renamed. Several things make the next search easier: an organization trail with retention long enough for your oldest resources, sourceIdentity on shared roles, readable session names in CI, and aws:createdBy activated in the management account. Recording the owner on the resource at creation time beats all of them.
That gap is what Cloud Sleuth is being built for. The free local tool will run on your machine or in your VPC as a single binary or Docker container. It will connect to a handful of AWS accounts and Azure subscriptions with read-only access, and apply ownership rules by account, subscription and name pattern, so far fewer resources ever need a CloudTrail search. Built-in policies will flag missing owners, untagged resources and common orphans, and nothing leaves your environment. Cloud Sleuth Cloud, the hosted version for teams, shares the same early-access list.
Both are pre-launch. Get early access and we’ll let you know when you can run them against your own accounts.
Checklist
- Creation time confirmed (for EC2, from the primary network interface’s attach time, not
LaunchTime) - AWS-added tags checked for a stack, Auto Scaling group, fleet, node group or Karpenter node pool
- Creation event found in event history (90 days or less) or in S3 with Athena (older)
-
userIdentitydecoded: SSO session name,sourceIdentity, or theAssumeRolethat issued the key - For service-launched resources, the creator of the parent stack, group or cluster found
- Owner confirmed with a person, and recorded as a tag
- Organization trail and log retention checked, so the next search has data
-
sourceIdentity, readable CI session names andaws:createdByconsidered for the future