← Financial Cloud Cloud Cloud Club · AWS Re:cap

Vertex Macro | Financial Cloud Cloud · AWS Re:cap

AWS Re:cap 04: Serverless AR Game with Bedrock AgentCore

Speaker: Cyrus Wong

Session: 04

Session
Summit Dev Lounge2026 Re:cap
01 Encode Architecture as Steering for AI Agents
02 Agent Harness Is the Real Engineering Moat
03 Ask Observability Data in Plain Language
04 Serverless AR Game with Bedrock AgentCore
05 Multi-Agent Quant Backtesting on AgentCore
06 Blog to Slides in Three Minutes with Kiro
AWS Community Day Hong Kong 2025 Re:cap
02 AWS Compliance with Terraform
03 Beginner to Builder An AWeSome Cloud Journey
04 Team-First Serverless Engineering with Laravel & Bref
05 Event Opening - AWS Community Day Hong Kong 2025
06 Agent-to-Agent: Building Interoperable AI on AWS
07 Utilize another telemetry data for faster improvement with AI agent
08 Graduating from Vibe Coding: Spec-Driven Development with Kiro
09 Automated Testing using MCP & AI Agents
10 Modernizing Telecom Security ML Powered Approach
11 Rethinking GenAI Agent: RAG & MCP
12 Disaster and Emergency Response with TAK and AWS
13 Rethinking Serverless Application Workflows from a Testing Perspective
14 Practical AWS FinOps for Cloud Success
15 AI-Powered Global Pure-Alpha Macro Trades on AWS: Revolutionizing Risk-Adjusted Asset Returns
FSI Recap
01 Modern Trade Lifecycle: Trading to Settlement
02 Goldman Sachs: Fast Track your applications onto Cloud - AWS Re:cap Q1/2023
03 Zurich Insurance Group: Building an Effective Log Management Solution on AWS
04 FSI Meetup 2025 Q4 - Brex Database Disaster Recovery
05 FSI Meetup 2025 Q4 - A Graviton Migration Success Story
06 FSI Meetup 2025 Q4 - Stifel Modern Data Platform
07 FSI Meetup 2025 Q4 - Financial Transaction Data Reconciler PayPal
08 FSI Meetup 2025 Q4 - Scaling Resilience
09 Maximizing AI Inference Cost Efficiency: Strategic Adoption of AWS GPU Instances
10 Advanced Agentic AI Design Patterns
11 Build New Modern Apps on AWS
AWS re:Invent 2025
01 Coinbase re:Invent Recap (IND3312)
02 Building the Future Trading Platform Leveraging AI and AWS
03 Trading Innovation: Jefferies' AI Assistant on Amazon Bedrock (IND3315)
04 How FSI Revolutionized HFT Analytics with Agentic AI (GBL302)
05 Improving Distributed Systems with Amazon Time Sync Featuring Nasdaq
06 Amazon Aurora HA and DR Design Patterns for Global Resilience (DAT442)
07 Building Agentic AI: Amazon Nova Act and Strands Agents in Practice (DEV327)
08 Deep Dive into Amazon Aurora and Its Innovations (DAT441)
09 Deep dive on Amazon S3 (STG407)
10 Nasdaq: Build Resilient Infrastructure for Global Financial Services (HMC327)
11 What's New with AWS Lambda (CNS376)
12 Spec-Driven Development with Kiro (DEV314)
13 Amazon's finops: Cloud cost lessons from a global e-commerce giant (AMZ308)
14 Tick to trade latency trading platforms on aws
Government data
01 The AI Era: The Boundary Between Development and Design Is Disappearing
02 On-Device Multimodal AI and Smart-City Practice
03 Large-Model Capability Evaluation and a Method for Landing AI Projects
04 Controlled End-to-End Automation of Government Development with Cloud Agents
05 A New Software Ecosystem for the Agent Era, Seen Through Multi-Agent Systems
06 AI-Driven Macro Quantitative Research and Smart Governance
07 Authorized Operation of Public Data and Smart-Government Practice
08 Putting Data Assetization into Practice: Rights, Compliance, Engineering Governance, and Digital-Government Cases
09 AI for Mental-Health Public Welfare: Governance, Architecture, and Practice of a Trustworthy Platform
Amarathon 2025 Recap
01 A Developer’s Roadmap to Architecting for Agents
02 Amazon Bedrock Data Automation
03 Multi-Agent on AgentCore
04 Building Agentic AI Nova Act and Strands Agents in Practice
04 Accelerating Migration Projects with Kiro using Spec-Driven Development
06 From Matching to Understanding: Personalized AI Search Practice Driven by AgentCore Memory
07 Observe to Optimize – LLM Observability to AIOps Turning real-time insights into intelligent automation
08 Deploying TEAM and Building the Best Engineering Team
09 Five Hard Lessons from Five Years of So-Called Serverless Databases
14 What if AI does my job How Q Developer CLI and Kiro have changed my daily routine
16 Velocity with Vigilance: Security Essentials for Amazon Bedrock Agent Development
26 Run OSS LLMs on a Single H100 Smarter, Cheaper, Faster
28 A Modern Unified Metadata Architecture: New Approaches to Breaking Down Data Silos
29 Serverless MediaOps: Automating Video Workflows with AI on Amazon Web Services
30 Architecting for Efficiency and Reliability with Performance Testing at Scale
31 Connecting the World Through Open Source: Practical Journey of Technology, Community and Global Developer Relations
33 Building Streaming Iceberg Tables for Real-Time Logistics Analytics
34 Accelerating Large-Scale Robot Strategy Training: An Automated Closed-Loop Architecture Based on Kiro, Trainium, and EKS
35 From Vibe to Viable with spec driven development
36 Making Cloud Cost Analysis Smarter: Building FinOps Intelligent Agents with Strands and AgentCore
37 Transform Conversational Agentic AIOps for K8s Using CNCF Kagent, K8sGPT, and Nova Sonic

AWS Hong Kong Summit 2026 · Developer Lounge Recap · Article 4 of 6

AWS Magazine · Field Report from Victoria Harbour


The Scene: Central-District Swagger Meets Serverless Sorcery

In my six decades of tracking technology shifts, I have stood on hundreds of conference floors, but nothing quite matches the electric pulse of the AWS Hong Kong Summit 2026 Developer Lounge.

Perched against the sweeping backdrop of Victoria Harbour, the Developer Lounge was a striking collision of Hong Kong luxury and AWS high-tech craft. Picture this: venture capitalists and startup founders closing seven-figure deals over artisanal drip coffee, international speakers trading architecture diagrams, and private developer meetups humming with high-stakes venture talk. Money was moving in and out, collaborations were forming on the fly, and amidst the neon shimmer and Central-district energy, builders were gathered around screens to witness pure serverless sorcery.

The standout attraction of the lounge? A demo by Cyrus Wong that brought together luxury entertainment and cloud engineering: The Domain Expansion AR Game.

(Not bad! Your reaction is almost as fast as me grabbing a limited-edition bag in Shibuya!)

Strike a Domain Expansion hand sign in front of a camera, and the screen erupts in 3D particle effects while a real (or simulated) humanoid robot adopts a kung-fu stance—all narrated live by an AI commentator speaking sassy, fashion-obsessed local Cantonese.

Here is the deep, hands-on architectural breakdown of how this fan-made application was constructed using AWS Bedrock AgentCore, AWS CDK, and serverless orchestrations.


The Star of the Show: AI Commentator

At the heart of the experience is a sassy AI commentator. Powered by multimodal AI vision, she roasts players' hand gestures, clothes, postures, and messy rooms in real-time.

Multilingual Support & Vision Intelligence

● Languages: Local Cantonese, Taiwanese Mandarin, Japanese, and English.

● Personality: Central-district high fashion meets competitive AR flair.

● Model Stack: moonshotai.kimi-k2.5 running inside a custom ECR container on Bedrock, paired with AWS Polly for high-fidelity neural speech synthesis.

System Architecture Summary

Surface / Function Model & Infrastructure Implementation Key Service Stack
AI Commentator Visual-aware text reasoning via moonshotai.kimi-k2.5 AWS Bedrock AgentCore Runtime
Voice Synthesis (TTS) Neural Cantonese, Mandarin, Japanese, English AWS Polly & Amazon S3
Gesture Tracking 21 3D Landmarks at 60 FPS Google MediaPipe Hands (Client JS)
Robotics Control Hardware & 3D Simulator MQTT stances AWS IoT Core & AWS Lambda
Infrastructure 100% Infrastructure as Code AWS CDK (TypeScript)

Step-by-Step Hands-On Execution Flow

+-----------------------------------------------------------------------------------+
END-TO-END GESTURE TO SPEECH FLOW

1. [ Browser Client ] -- MediaPipe 60FPS --> [ API Gateway ]
2. [ API Gateway ] -- Cognito JWT --> [ Monolithic Lambda ]
3. [ Monolithic Lambda ] -- XML multimodal payload --> [ AgentCore Runtime ]
4. [ AgentCore Runtime ] --> [ AWS Polly ] neural speech
5. [ AWS Polly ] --> [ S3 Audio Bucket ] --> browser playback
+-----------------------------------------------------------------------------------+

Step A: Client-Side Gesture Detection & API Signalling

The browser runs hand_tracker.js using Google’s MediaPipe Hands to analyze 21 3D landmarks at 60 FPS. When a Domain Expansion gesture is verified, battle.js sends a POST request with a Cognito Bearer JWT to Amazon API Gateway:

// Cognito JWT + recognized gesture → API Gateway → Polly audio URL
const response = await fetch("/api/trigger-technique", {
  method: "POST",
  headers: { Authorization: `Bearer ${await getCognitoAccessToken()}` },
  body: JSON.stringify({ technique: techniqueName }),
});
playCommentaryAudio((await response.json()).audioUrl);

Step B: Cognito Authorization & XML Base64 Bypass

When API Gateway validates the token, it forwards the request to a monolithic Flask Lambda (lambda_function.py).

To prevent enterprise network proxies from stripping binary multipart image data during transit, commentary.py embeds webcam snapshots inside custom XML tags in the text payload:

# Hide JPEG inside text so enterprise proxies can't strip multipart binaries
content_block += f"<p1_webcam_base64_jpeg>{image_base64}</p1_webcam_base64_jpeg>"
agent_client.invoke_agent_runtime(
    agentRuntimeArn=os.environ["AGENTCORE_RUNTIME_ARN"],
    payload=json.dumps({"prompt": content_block, "session_id": session_id}).encode(),
)

Step C: Serverless AgentCore Runtime Execution

The AWS Bedrock AgentCore Runtime routes the call via IAM SigV4 to the custom container (domain_commentator_agentcore). The container’s FastAPI service (commentator_agent.py) uses regex to extract the base64 XML block and reconstruct binary multimodal payloads for Bedrock:

# Unwrap XML → rebuild multimodal Bedrock payload
image_b64 = re.search(
    r"<p1_webcam_base64_jpeg>(.*?)</p1_webcam_base64_jpeg>", prompt_text, re.DOTALL
).group(1)
await strands_agent.invoke_async([
    {"image": {"format": "jpeg", "source": {"bytes": base64.b64decode(image_b64)}}},
    {"text": prompt_text},
])

Step D: Hardware Dispatch via AgentCore Tool Gateway

When the model triggers physical actions, the runtime calls the AgentCore Tool Gateway (bedrockagentcore.Gateway), which invokes robot_tool_lambda.py via IAM SigV4 to publish MQTT commands to AWS IoT Core:

# AgentCore tool name → IoT MQTT action
local_tool = event["tool_name"].split("___")[-1]  # "...___robot_wave"
iot_client.publish(topic="robot_1/topic", payload=json.dumps({"action": "wave_hand"}))

Infrastructure as Code: CDK Core Patterns

To keep infrastructure repeatable, modern, and clean, the entire system is modeled in AWS CDK.

1. Centralized Tool Gateway Construct (robot-tool-gateway.ts)

// Robot Lambda behind AgentCore Gateway — IAM SigV4 only
this.gateway = new bedrockagentcore.Gateway(this, "RobotToolGateway", {
  authorizerConfiguration: bedrockagentcore.GatewayAuthorizer.usingAwsIam(),
});
this.gateway.addLambdaTarget("RobotToolLambdaTarget", {
  lambdaFunction: this.robotToolFunction,
});

2. Serverless AgentCore Commentator Runtime (domain-expansion-serverless.ts)

// Cost-aware AgentCore lifecycle: idle sessions die fast
new agentcore.Runtime(this, "Runtime", {
  runtimeName: "domain_commentator_agentcore",
  lifecycleConfiguration: {
    idleRuntimeSessionTimeout: Duration.seconds(120), // $0 when idle
    maxLifetime: Duration.seconds(900),
  },
  environmentVariables: { BEDROCK_MODEL_ID: "moonshotai.kimi-k2.5" },
});

Split-Route API Gateway Security Model

The team implemented a split-route REST model to resolve a classic web conflict: standard browser elements (<img src="...">) cannot pass Bearer headers, but write operations cost LLM money.

// Reads are public (<img> can't send Bearer); LLM writes require Cognito
apiResource.addResource("get-snapshot").addMethod("GET", lambdaIntegration, {
  authorizationType: apigateway.AuthorizationType.NONE,
});
apiResource.addResource("trigger-technique").addMethod("POST", lambdaIntegration, {
  authorizationType: apigateway.AuthorizationType.COGNITO,
  authorizer: restAuthorizer,
});

Observability & Enterprise Tracing

Observability is handled via CloudWatch Vended Logs namespaces and AWS X-Ray:

// AgentCore → CloudWatch Vended Logs (X-Ray maps the full path)
logGroupName: `/aws/vendedlogs/bedrock-agentcore/gateway/APPLICATION_LOGS/${gateway.gatewayId}`

This ensures end-to-end distributed tracing on a single X-Ray map:

Browser Gesture → API Gateway → Lambda Router → AgentCore Container → Strands LLM → AWS Polly TTS.


Smart Client Guards & Character Engineering

1. Cost-Guarding Session Logic (auth-check.js)

To avoid billing leaks from idle browser tabs keeping AgentCore sockets open, the client checks the Cognito JWT every 30 seconds on-device:

Session Expired = currentTime >= jwt.exp

If expired, it closes active WebSockets immediately:

// Expire JWT locally → close sockets → no idle AgentCore bill
if (isTokenExpired(getCognitoToken())) webSocketConnection?.close();
setInterval(checkSessionGuard, 30000); // client-side only

2. Speech Text Sanitization (commentary_tts.py)

Before passing model outputs to AWS Polly, Markdown symbols (**bold**, #) are converted to clean plain text using BeautifulSoup to prevent Polly from reading syntax aloud:

# Stop Polly from reading **bold** / # aloud
BeautifulSoup(markdown.markdown(raw_text), "html.parser").get_text()

3. Soul as Code Configuration

Character identity is fully decoupled into standard Markdown files loaded at runtime:

● IDENTITY.md: Directives on sassy Cantonese styling and demeanor.

● SOUL.md: Combat lore rules (e.g., how to roast rival Domain Expansion users).


Architectural Economics & Pay-As-You-Go

Architecture Choice Cost & Efficiency Impact
Zero Idle Billing $0.00 cost when no matches are active; no continuous EC2/ECS servers.
S3 Lifecycle Rules Webcam snapshots & Polly MP3 audio streams auto-deleted after 7 days.
P2P Video WebRTC Video streams directly peer-to-peer; API Gateway WebSockets handle light telemetry only.

Closing Reflection: The Developer Lounge Vibe

What Cyrus Wong and the team demonstrated at the AWS Hong Kong Summit 2026 Developer Lounge was more than a fun fan game. It was a masterclass in modern serverless design: high fashion, high technology, low latency, and zero idle cost.

As the sun set over Central, illuminating Victoria Harbour, developers were still clustered around the booth, scanning CDK repositories and testing hand signs. That is the real spirit of the AWS Developer Lounge—bringing world-class engineering, vibrant local culture, and serverless power together under one roof.