SuperAPI AI Directory
Partner with us中文AI newsFavorites
Skillsagentcore-ops-review

agentcore-ops-review

Deployment & operations

View repository

Updated 10-15 17:23

About this skill

Bedrock AgentCore Operational Review

Use this skill when performing an operational review of Amazon Bedrock AgentCore resources, investigating agent runtime failures, auditing memory/knowledge pipeline health, or reviewing gateway resilience and resource hygiene.

Overview

This skill provides procedures for a review of AgentCore resources across four check areas mapped to the AWS Well-Architected Framework:

  • Runtime Resilience (Reliability) — Failed/stuck runtimes and endpoints, single-AZ VPC placement, endpoint version drift, DEFAULT-endpoint-only deployments
  • Gateway Health (Reliability) — Gateway status, target redundancy and health, policy engine attachment, target synchronization freshness
  • Memory & Knowledge Effectiveness (Performance Efficiency) — Extraction pipeline errors, empty long-term memory, high ingestion error rate, provisioned-but-never-populated memories
  • Resource Utilization & Operational Hygiene (Operational Excellence) — Idle resources, consolidation opportunities, low overall utilization
  • Runtime Observability (cross-pillar) — Session counts, invocations, throttles, vCPU-hours, GB-hours per runtime

This is a READ-ONLY review. No modifications are made to any resource, and it never invokes an agent (InvokeAgentRuntime), reads no prompts or responses, and makes no other data-plane calls. One exception: for long-term memories the skill calls bedrock-agentcore:ListMemoryRecords (a data-plane API) solely to obtain a record count. That response can include a content field carrying extracted, potentially PII-bearing memory content; the skill uses only the count and never parses, logs, stores, or reproduces content. See "Memory record data handling" below.

Data Source Boundaries (IMPORTANT — IAM footprint)

The standard AIDevOpsAgentAccessPolicy covers bedrock:* read actions but does NOT include the bedrock-agentcore:* namespace. (Note: bedrock-agentcore-control is the SDK client name, not an IAM prefix — all control-plane actions such as ListAgentRuntimes, GetMemory, and ListGateways authorize under the single service prefix bedrock-agentcore:.) Two modes:

  1. Runtime-observability-only mode — relies exclusively on CloudWatch metrics (cloudwatch:GetMetricData, cloudwatch:ListMetrics) against namespace AWS/Bedrock-AgentCore. Requires no bedrock-agentcore: additions. When an account isn't using AgentCore, the namespace is simply empty ("no activity detected") — this is not a failure.
  2. Full control-plane mode — adds read-only bedrock-agentcore: control-plane List/Get actions and ec2:DescribeSubnets to enable the runtime/gateway/memory/utilization checks. It also uses one data-plane action, bedrock-agentcore:ListMemoryRecords, for the long-term-memory record count only (see "Memory record data handling" below). See references/iam-policy-linked-account.json.

If a required permission is missing, the affected check degrades to a visibility limit (reported as "signal unavailable / check skipped") rather than failing the review or producing a false finding.

Memory record data handling: bedrock-agentcore:ListMemoryRecords is a data-plane API whose MemoryRecordSummary entries include a required content field — the extracted facts/preferences/summaries a long-term memory has stored, which can contain end-user PII. This skill calls it only to count records for AC-MEM-2 and never reads, parses, logs, stores, transforms, or reproduces the content field. If you prefer zero data-plane access, omit this action from the IAM policy: AC-MEM-2 then degrades to a visibility limit while all other memory checks (which use CloudWatch ingestion metrics) continue to work.

Seam with agentcore-observability-setup: This skill assesses operational posture from existing telemetry; it does not configure or validate observability wiring. Where telemetry is absent, this skill reports a visibility limit and defers the configuration gap to agentcore-observability-setup — that skill owns the observability-wiring finding (Transaction Search, OTEL/ADOT, log delivery, X-Ray resource policy), while this skill reports only the posture consequence. A customer running both should not see two overlapping findings on the same resource.

Step 1: Discover Account and Regions

  1. Call sts:GetCallerIdentity to determine the account.
  2. Discover AgentCore-active regions:
    • Query Cost Explorer: ce:GetDimensionValues (dimension SERVICE, last 30 days), match "Amazon Bedrock AgentCore".
    • Then ce:GetCostAndUsage for that service grouped by REGION; select top regions by UnblendedCost.
    • If no AgentCore spend is found, fall back to probing cloudwatch:ListMetrics (namespace AWS/Bedrock-AgentCore) in the customer's primary regions to detect activity.
  3. If no AgentCore activity is detected in any region, stop: "No AgentCore usage detected in the last 30 days."

Step 2: Runtime Resilience Pillar (Reliability)

Discover runtimes and enrich each with endpoints and versions:

  • bedrock-agentcore:ListAgentRuntimes → GetAgentRuntime (status, failureReason, networkConfiguration)
  • bedrock-agentcore:ListAgentRuntimeEndpoints (status, liveVersion, targetVersion, name)
  • bedrock-agentcore:ListAgentRuntimeVersions (to compute latest version + drift)
  • ec2:DescribeSubnets (resolve VPC subnet IDs to AZs — endpoints carry NO AZ data)
Check Rule Severity
AC-RUN-1 Runtime/endpoint in CREATE_FAILED/UPDATE_FAILED, or stuck in CREATING/UPDATING > 1 hour Critical
AC-RUN-2 VPC-mode runtime whose subnets all resolve to a single AZ (PUBLIC mode exempt) High
AC-RUN-3 Live endpoint ≥ 3 versions behind latest runtime version Medium
AC-RUN-4 Actively-updated runtime (2+ versions) served only by the auto-updating DEFAULT endpoint Low

Key fact: Endpoints are a versioning/traffic-routing construct, not a redundancy mechanism. AZ fault tolerance is derived exclusively from the runtime's VPC subnet configuration. If ec2:DescribeSubnets is unavailable, AC-RUN-2 degrades to "AZ distribution unknown".

Step 3: Gateway Health (Reliability)

Discover gateways and their targets:

  • bedrock-agentcore:ListGateways → GetGateway (status, protocolType, authorizerType, policyEngineConfiguration)
  • bedrock-agentcore:ListGatewayTargets → GetGatewayTarget (status, targetConfiguration, lastSynchronizedAt, credential providers)

Target type derived from targetConfiguration.mcp: Lambda, MCP Server, API Gateway, OpenAPI Schema, Smithy Model.

Risk Level Condition
Critical Gateway not READY, zero targets, OR all targets unhealthy
Warning Single target (no redundancy), no policy engine attached, some unhealthy targets, OR stale sync (> 7 days)
Healthy Multiple READY targets, policy engine attached, recent synchronization

Unhealthy target statuses: FAILED, UPDATE_UNSUCCESSFUL, SYNCHRONIZE_UNSUCCESSFUL. Stale sync threshold: lastSynchronizedAt older than 7 days.

Step 4: Memory & Knowledge Effectiveness (Performance Efficiency)

Discover memories and their strategies, then query CloudWatch ingestion metrics:

  • bedrock-agentcore:ListMemories → GetMemory (status, createdAt, configured strategies)
  • bedrock-agentcore:ListMemoryRecords — data-plane call, count only (long-term-strategy memories). Read the returned record count; do not read the content field (see "Memory record data handling")
  • cloudwatch:GetMetricData (namespace AWS/Bedrock-AgentCore): per-memory Invocations/Errors for the Ingestion operation, and Invocations for the CreateEvent operation (30-day window)

All rules are strategy-aware — read GetMemory strategies first. Record-count rules apply ONLY to memories with a long-term strategy; short-term-only memories are never flagged as empty.

Check Rule Severity
AC-MEM-1 Long-term memory with CloudWatch ingestion Errors > 0 High
AC-MEM-2 Long-term memory with < 10 records (escalates to High if 0 records AND > 7 days old) Medium→High
AC-MEM-3 Ingestion error rate > 20% (Errors / Invocations) High
AC-MEM-4 Provisioned but never populated: zero CreateEvent activity AND > 7 days old Medium

Safety rule: CreateEvent activity is queried for EVERY memory (short-term memories receive events too). When the CloudWatch signal is unreadable, event count is None (not 0), so AC-MEM-4 is skipped rather than firing a false "never populated" finding.

Step 5: Resource Utilization & Operational Hygiene (Operational Excellence)

Enumerate all provisionable resource types and collect a 30-day activity signal:

  • Inventory: ListAgentRuntimes, ListMemories, ListGateways, ListBrowsers, ListCodeInterpreters, ListWorkloadIdentities
  • Activity: cloudwatch:GetMetricData (Invocations, AWS/Bedrock-AgentCore) keyed by per-type dimension (AgentRuntimeId, MemoryId, GatewayId)
Check Rule Severity
AC-UTIL-1 Idle resource: zero activity over 30-day window, > 7 days old Medium
AC-UTIL-2 Consolidation: duplicate configs in a region, OR regions holding resources with < 5% of total activity (3+ regions) Informational→Medium
AC-UTIL-3 Overall utilization < 60% (active / assessable, excluding recently-created) Informational

For the AC-UTIL-1 idle signal, prefer the real-time ActiveSessionCount gauge (a currently-running-sessions gauge, filterable by the Service dimension) where available; fall back to the cumulative SessionCount / Invocations counters over the 30-day window when ActiveSessionCount is not present.

Framing: AgentCore runtime billing is consumption-based — idle time is free. Frame findings as operational hygiene and security surface (unmanaged IAM roles, stale config), NOT wasted spend. Exception: memories holding stored long-term records DO accrue storage cost — call this out in the finding.

Classification safety: A resource is only classified Idle when its activity signal is complete for the full window. Types with no queryable per-resource metric (Browser, CodeInterpreter, WorkloadIdentity), and every resource when the CloudWatch call fails, are classified Active (partial) — never Idle. Resources younger than 7 days are RecentlyCreated and excluded from both idle flagging and the utilization ratio.

Step 6: Runtime Observability (cross-pillar signals)

For each runtime, collect CloudWatch metrics from namespace AWS/Bedrock-AgentCore:

Metric Dimensions Period Statistic
CPUUsed-vCPUHours Resource (ARN), Service=AgentCore.Runtime, Name 3600s Sum
MemoryUsed-GBHours Resource (ARN), Service=AgentCore.Runtime, Name 3600s Sum
SessionCount Resource (ARN), Operation=InvokeAgentRuntime, Name 300s Sum
Invocations Resource (ARN), Operation=InvokeAgentRuntime, Name 300s Sum
Throttles Resource (ARN), Operation=InvokeAgentRuntime, Name 300s Sum

SessionCount is a cumulative counter of new sessions per period. Where a real-time view is needed, ActiveSessionCount is a gauge of currently-running sessions (filterable by the Service dimension).

Discover resources first via cloudwatch:ListMetrics (namespace AWS/Bedrock-AgentCore), extract unique Resource ARN + Name dimensions, then batch GetMetricData per resource.

Derived signals:

  • Throttle rate = Throttles / Invocations × 100
  • Health status = "Throttling Detected" when Throttles > 0, else "Healthy"
  • Only report runtimes with any activity (CPU-hours, GB-hours, invocations, or session count > 0).

Step 7: Additional Context

  1. Health Events: health:DescribeEvents filtered for Bedrock/AgentCore service, last 14 days.
  2. Documentation: Cross-reference findings against AgentCore best-practices docs for remediation links.

Step 8: Produce Report and Recommendations

Produce an artifact with the structure defined in references/report-template.md.

For each finding with severity Warning/Medium or higher, create a recommendation with:

  • Title: [Check ID] [Check name] — [Resource identifier]
  • Summary: Current state, expected state, business/operational impact, remediation steps, and a TAM conversation starter.

Related Skills / Boundaries

Boundaries are deliberate and non-overlapping:

  • agentcore-observability-setup — sets up/validates observability wiring. This skill consumes that telemetry for posture assessment and defers observability-configuration gaps to it (see the seam described in Data Source Boundaries).
  • agentcore-runtime-diag (queued submission — referenced for scope, not yet published) — reactive triage of a single failing invocation. This skill is proactive, estate-wide posture; it does not diagnose individual call failures or evaluate authorizer/credential correctness. Gateway checks here are redundancy/staleness/health only, not auth evaluation.
  • aiml-access-diagnostics (published) — generic AI/ML AccessDenied chain via iam:SimulatePrincipalPolicy. Not overlapping — this skill makes no IAM-simulation calls.
  • bedrock-adoption-readiness (published) — foundation-model workload readiness (IAM governance, ZDR, quotas, observability). Different resource surface — this skill reviews the AgentCore agent runtime / memory / gateway layer, not foundation-model inference.

Error Handling

Error Action
AccessDenied on bedrock-agentcore:* Log as visibility limit "signal unavailable — check skipped", continue in observability-only mode
AccessDenied on ec2:DescribeSubnets AC-RUN-2 degrades to "AZ distribution unknown"
CloudWatch returns no data Report "no activity detected" for that resource; classify utilization as Active (partial), never Idle
No AgentCore usage anywhere Stop early: "No AgentCore usage detected"
Throttled by AWS API Retry with exponential backoff (3 attempts)
Check raises an exception Isolate the failure — degrade that check to a visibility limit and continue the others

Important Notes

  • ALL API calls with pagination (NextToken) MUST be paginated to completion.
  • Cost Explorer queries run against us-east-1 (global endpoint).
  • Findings are only produced when the underlying signal is complete — incomplete signals become visibility limits, never false positives.
  • Batch CloudWatch GetMetricData requests where possible.
  • This is a READ-ONLY review — no modifications, and no data-plane calls other than ListMemoryRecords for record count (count only; content is never read).