← Financial Cloud Cloud Cloud Club · AWS Re:cap

Vertex Macro | Financial Cloud Cloud · AWS Re:cap

AWS Re:cap 04: Serverless AR Game with Bedrock AgentCore

Speaker: Cyrus Wong

Session: 04

Session
Summit Dev Lounge2026 Re:cap
01 Encode Architecture as Steering for AI Agents
02 Agent Harness Is the Real Engineering Moat
03 Ask Observability Data in Plain Language
04 Serverless AR Game with Bedrock AgentCore
05 Multi-Agent Quant Backtesting on AgentCore
06 Blog to Slides in Three Minutes with Kiro
AWS Community Day Hong Kong 2025 Re:cap
02 AWS Compliance with Terraform
03 Beginner to Builder An AWeSome Cloud Journey
04 Team-First Serverless Engineering with Laravel & Bref
05 Event Opening - AWS Community Day Hong Kong 2025
06 Agent-to-Agent: Building Interoperable AI on AWS
07 Utilize another telemetry data for faster improvement with AI agent
08 Graduating from Vibe Coding: Spec-Driven Development with Kiro
09 Automated Testing using MCP & AI Agents
10 Modernizing Telecom Security ML Powered Approach
11 Rethinking GenAI Agent: RAG & MCP
12 Disaster and Emergency Response with TAK and AWS
13 Rethinking Serverless Application Workflows from a Testing Perspective
14 Practical AWS FinOps for Cloud Success
15 AI-Powered Global Pure-Alpha Macro Trades on AWS: Revolutionizing Risk-Adjusted Asset Returns
FSI Recap
01 Modern Trade Lifecycle: Trading to Settlement
02 Goldman Sachs: Fast Track your applications onto Cloud - AWS Re:cap Q1/2023
03 Zurich Insurance Group: Building an Effective Log Management Solution on AWS
04 FSI Meetup 2025 Q4 - Brex Database Disaster Recovery
05 FSI Meetup 2025 Q4 - A Graviton Migration Success Story
06 FSI Meetup 2025 Q4 - Stifel Modern Data Platform
07 FSI Meetup 2025 Q4 - Financial Transaction Data Reconciler PayPal
08 FSI Meetup 2025 Q4 - Scaling Resilience
09 Maximizing AI Inference Cost Efficiency: Strategic Adoption of AWS GPU Instances
10 Advanced Agentic AI Design Patterns
11 Build New Modern Apps on AWS
AWS re:Invent 2025
01 Coinbase re:Invent Recap (IND3312)
02 Building the Future Trading Platform Leveraging AI and AWS
03 Trading Innovation: Jefferies' AI Assistant on Amazon Bedrock (IND3315)
04 How FSI Revolutionized HFT Analytics with Agentic AI (GBL302)
05 Improving Distributed Systems with Amazon Time Sync Featuring Nasdaq
06 Amazon Aurora HA and DR Design Patterns for Global Resilience (DAT442)
07 Building Agentic AI: Amazon Nova Act and Strands Agents in Practice (DEV327)
08 Deep Dive into Amazon Aurora and Its Innovations (DAT441)
09 Deep dive on Amazon S3 (STG407)
10 Nasdaq: Build Resilient Infrastructure for Global Financial Services (HMC327)
11 What's New with AWS Lambda (CNS376)
12 Spec-Driven Development with Kiro (DEV314)
13 Amazon's finops: Cloud cost lessons from a global e-commerce giant (AMZ308)
14 Tick to trade latency trading platforms on aws
Government data
01 The AI Era: The Boundary Between Development and Design Is Disappearing
02 On-Device Multimodal AI and Smart-City Practice
03 Large-Model Capability Evaluation and a Method for Landing AI Projects
04 Controlled End-to-End Automation of Government Development with Cloud Agents
05 A New Software Ecosystem for the Agent Era, Seen Through Multi-Agent Systems
06 AI-Driven Macro Quantitative Research and Smart Governance
07 Authorized Operation of Public Data and Smart-Government Practice
08 Putting Data Assetization into Practice: Rights, Compliance, Engineering Governance, and Digital-Government Cases
09 AI for Mental-Health Public Welfare: Governance, Architecture, and Practice of a Trustworthy Platform
Amarathon 2025 Recap
01 A Developer’s Roadmap to Architecting for Agents
02 Amazon Bedrock Data Automation
03 Multi-Agent on AgentCore
04 Building Agentic AI Nova Act and Strands Agents in Practice
04 Accelerating Migration Projects with Kiro using Spec-Driven Development
06 From Matching to Understanding: Personalized AI Search Practice Driven by AgentCore Memory
07 Observe to Optimize – LLM Observability to AIOps Turning real-time insights into intelligent automation
08 Deploying TEAM and Building the Best Engineering Team
09 Five Hard Lessons from Five Years of So-Called Serverless Databases
14 What if AI does my job How Q Developer CLI and Kiro have changed my daily routine
16 Velocity with Vigilance: Security Essentials for Amazon Bedrock Agent Development
26 Run OSS LLMs on a Single H100 Smarter, Cheaper, Faster
28 A Modern Unified Metadata Architecture: New Approaches to Breaking Down Data Silos
29 Serverless MediaOps: Automating Video Workflows with AI on Amazon Web Services
30 Architecting for Efficiency and Reliability with Performance Testing at Scale
31 Connecting the World Through Open Source: Practical Journey of Technology, Community and Global Developer Relations
33 Building Streaming Iceberg Tables for Real-Time Logistics Analytics
34 Accelerating Large-Scale Robot Strategy Training: An Automated Closed-Loop Architecture Based on Kiro, Trainium, and EKS
35 From Vibe to Viable with spec driven development
36 Making Cloud Cost Analysis Smarter: Building FinOps Intelligent Agents with Strands and AgentCore
37 Transform Conversational Agentic AIOps for K8s Using CNCF Kagent, K8sGPT, and Nova Sonic

AWS Hong Kong Summit 2026 · Developer Lounge Recap · Article 4 of 6

AWS Magazine · Field Report from Victoria Harbour


The Scene: Central-District Swagger Meets Serverless Sorcery

In my six decades of tracking technology shifts, I have stood on hundreds of conference floors, but nothing quite matches the electric pulse of the AWS Hong Kong Summit 2026 Developer Lounge.

Perched against the sweeping backdrop of Victoria Harbour, the Developer Lounge was a striking collision of Hong Kong luxury and AWS high-tech craft. Picture this: venture capitalists and startup founders closing seven-figure deals over artisanal drip coffee, international speakers trading architecture diagrams, and private developer meetups humming with high-stakes venture talk. Money was moving in and out, collaborations were forming on the fly, and amidst the neon shimmer and Central-district energy, builders were gathered around screens to witness pure serverless sorcery.

The standout attraction of the lounge? A demo by Cyrus Wong that brought together luxury entertainment and cloud engineering: The Jujutsu Kaisen Domain Expansion AR Game.

「唔錯喎!反應好似我去搶購減價化妝品噉快!當堂順眼好多!」 (Not bad! Your reaction is almost as fast as me grabbing a limited-edition bag in Shibuya!)

Strike Gojo Satoru’s Unlimited Void (無量空處) or Ryomen Sukuna’s Malevolent Shrine (伏魔御廚子) hand sign in front of a camera, and the screen erupts in 3D particle effects while a real (or simulated) humanoid robot adopts a kung-fu stance—all narrated live by an AI commentator speaking sassy, fashion-obsessed local Cantonese.

Here is the deep, hands-on architectural breakdown of how this fan-made application was constructed using AWS Bedrock AgentCore, AWS CDK, and serverless orchestrations.


The Star of the Show: AI Commentator Nobara

At the heart of the experience is an AI commentator modeled after Kugisaki Nobara (釘崎野薔薇). Powered by multimodal AI vision, she roasts players' hand gestures, clothes, postures, and messy rooms in real-time.

Multilingual Support & Vision Intelligence

● Languages: Local Cantonese, Taiwanese Mandarin, Japanese, and English.

● Personality: Central-district high fashion meets anime cursed techniques.

● Model Stack: moonshotai.kimi-k2.5 running inside a custom ECR container on Bedrock, paired with AWS Polly for high-fidelity neural speech synthesis.

System Architecture Summary

Surface / Function Model & Infrastructure Implementation Key Service Stack
AI Commentator (Nobara) Visual-aware text reasoning via moonshotai.kimi-k2.5 AWS Bedrock AgentCore Runtime
Voice Synthesis (TTS) Neural Cantonese, Mandarin, Japanese, English AWS Polly & Amazon S3
Gesture Tracking 21 3D Landmarks at 60 FPS Google MediaPipe Hands (Client JS)
Robotics Control Hardware & 3D Simulator MQTT stances AWS IoT Core & AWS Lambda
Infrastructure 100% Infrastructure as Code AWS CDK (TypeScript)

Step-by-Step Hands-On Execution Flow

+-----------------------------------------------------------------------------------+
END-TO-END GESTURE TO SPEECH FLOW

1. [ Browser Client ] -- MediaPipe 60FPS --> [ API Gateway ]
2. [ API Gateway ] -- Cognito JWT --> [ Monolithic Lambda ]
3. [ Monolithic Lambda ] -- XML multimodal payload --> [ AgentCore Runtime ]
4. [ AgentCore Runtime ] --> [ AWS Polly ] neural speech
5. [ AWS Polly ] --> [ S3 Audio Bucket ] --> browser playback
+-----------------------------------------------------------------------------------+

Step A: Client-Side Gesture Detection & API Signalling

The browser runs hand_tracker.js using Google’s MediaPipe Hands to analyze 21 3D landmarks at 60 FPS. When a gesture (e.g., Malevolent Shrine) is verified, battle.js sends a POST request with a Cognito Bearer JWT to Amazon API Gateway:

// battle.js: Dispatching recognized gesture to API Gateway[cite: 1]
async function triggerCursedTechnique(techniqueName) {
    const token = await getCognitoAccessToken(); // High-security Bearer Token[cite: 1]
    try {
        const response = await fetch("/api/trigger-technique", {
            method: "POST",
            headers: {
                "Content-Type": "application/json",
                "Authorization": `Bearer ${token}`
            },
            body: JSON.stringify({
                technique: techniqueName, // e.g. "malevolent_shrine"[cite: 1]
                robotId: "all",
                sessionId: "mcpserver"
            })
        });
        const result = await response.json();
        playCommentaryAudio(result.audioUrl); // Play resulting neural voice track[cite: 1]
    } catch (err) {
        console.error("Failed to execute server-side technique orchestration:", err); //[cite: 1]
    }
}

Step B: Cognito Authorization & XML Base64 Bypass

When API Gateway validates the token, it forwards the request to a monolithic Flask Lambda (lambda_function.py).

To prevent enterprise network proxies from stripping binary multipart image data during transit, commentary.py embeds webcam snapshots inside custom XML tags in the text payload:

# commentary.py: Packaging snapshots inside XML text tags[cite: 1]
def generate_ai_commentary(content_block, session_id="mcpserver", image_base64_p1=""):
    if image_base64_p1:
        content_block += f"\n<p1_webcam_base64_jpeg>{image_base64_p1}</p1_webcam_base64_jpeg>" #[cite: 1]

    agent_client = boto3.client("bedrock-agentcore", region_name="us-east-1") #[cite: 1]

    payload_dict = {
        "prompt": content_block,
        "session_id": session_id
    }

    response = agent_client.invoke_agent_runtime(
        agentRuntimeArn=os.environ.get("AGENTCORE_RUNTIME_ARN"),
        runtimeSessionId=session_id,
        payload=json.dumps(payload_dict).encode("utf-8")
    )
    return response.get("response").read().decode("utf-8") #[cite: 1]

Step C: Serverless AgentCore Runtime Execution

The AWS Bedrock AgentCore Runtime routes the call via IAM SigV4 to the custom container (domain_commentator_agentcore). The container’s FastAPI service (commentator_agent.py) uses regex to extract the base64 XML block and reconstruct binary multimodal payloads for Bedrock:

# commentator_agent.py: Unwrapping XML image parts inside container[cite: 1]
@app.post("/invocations")
async def invoke_agent(request: Request):
    body = await request.json()
    prompt_text = body.get("prompt", "")

    p1_pattern = re.compile(r"<p1_webcam_base64_jpeg>(.*?)</p1_webcam_base64_jpeg>", re.DOTALL) #[cite: 1]
    p1_match = p1_pattern.search(prompt_text)

    image_b64_p1 = ""
    if p1_match:
        image_b64_p1 = p1_match.group(1).strip()
        prompt_text = p1_pattern.sub("", prompt_text).strip() # Clean text prompt[cite: 1]

    img_bytes_p1 = base64.b64decode(image_b64_p1)
    message_content = [
        {"text": "Player 1 webcam snapshot:"},
        {"image": {"format": "jpeg", "source": {"bytes": img_bytes_p1}}},
        {"text": prompt_text}
    ]

    response = await strands_agent.invoke_async(message_content) #[cite: 1]
    return JSONResponse(content={"response": str(response)}, status_code=200) #[cite: 1]

Step D: Hardware Dispatch via AgentCore Tool Gateway

When the model triggers physical actions, the runtime calls the AgentCore Tool Gateway (bedrockagentcore.Gateway), which invokes robot_tool_lambda.py via IAM SigV4 to publish MQTT commands to AWS IoT Core:

# robot_tool_lambda.py: Stripping tool prefix and invoking IoT Core[cite: 1]
def lambda_handler(event, context):
    full_tool_name = event.get("tool_name", "") # e.g., "robot-only-mcp-lambda___robot_wave"[cite: 1]
    local_tool_name = full_tool_name.split("___")[-1] if "___" in full_tool_name else full_tool_name #[cite: 1]

    if local_tool_name == "robot_wave":
        iot_client = boto3.client("iot-data")
        iot_client.publish(
            topic="arn:aws:iot:us-east-1:123456789012:topic/robot_1/topic",
            qos=1,
            payload=json.dumps({"action": "wave_hand", "timestamp": int(time.time())})
        )
        return {"status": "SUCCESS", "message": "Robot hand wave triggered."} #[cite: 1]


Infrastructure as Code: CDK Core Patterns

To keep infrastructure repeatable, modern, and clean, the entire system is modeled in AWS CDK.

1. Centralized Tool Gateway Construct (robot-tool-gateway.ts)

// Constructing the Tool Gateway with IAM SigV4 security[cite: 1]
export class RobotToolGatewayConstruct extends Construct {
    public readonly robotToolFunction: PythonFunction;
    public readonly gateway: bedrockagentcore.Gateway;

    constructor(scope: Construct, id: string, props: RobotToolGatewayConstructProps) {
        super(scope, id);

        this.robotToolFunction = new PythonFunction(this, "RobotToolFunction", {
            entry: path.join(__dirname, "../../../mcp_server"),
            runtime: SHARED_PYTHON_RUNTIME,
            index: "robot_tool_lambda.py",
            handler: "lambda_handler",
            timeout: Duration.seconds(30),
        });

        this.gateway = new bedrockagentcore.Gateway(this, "RobotToolGateway", {
            description: "AgentCore gateway fronting the robot Lambda tools",
            authorizerConfiguration: bedrockagentcore.GatewayAuthorizer.usingAwsIam(), // SigV4 Auth[cite: 1]
        });

        this.gateway.addLambdaTarget("RobotToolLambdaTarget", {
            gatewayTargetName: "robot-only-mcp-lambda",
            lambdaFunction: this.robotToolFunction,
            toolSchema: bedrockagentcore.ToolSchema.fromLocalAsset(materializeRobotToolSchemaAsset()),
        });
    }
}

2. Serverless AgentCore Commentator Runtime (domain-expansion-serverless.ts)

// Deploying commentator runtime with cost-aware lifecycles[cite: 1]
const runtime = new agentcore.Runtime(this, "Runtime", {
    runtimeName: "domain_commentator_agentcore",
    agentRuntimeArtifact: agentcore.AgentRuntimeArtifact.fromAsset(
        path.join(__dirname, "../../../domain-expansion-commentator-agentcore"),
        { platform: Platform.LINUX_ARM64 }
    ),
    authorizerConfiguration: agentcore.RuntimeAuthorizerConfiguration.usingIAM(),
    lifecycleConfiguration: {
        idleRuntimeSessionTimeout: Duration.seconds(120), // 2 min idle timeout[cite: 1]
        maxLifetime: Duration.seconds(900),              // 15 min max session[cite: 1]
    },
    tracingEnabled: true,
    environmentVariables: {
        BEDROCK_MODEL_ID: "moonshotai.kimi-k2.5",
    },
});


Split-Route API Gateway Security Model

The team implemented a split-route REST model to resolve a classic web conflict: standard browser elements (<img src="...">) cannot pass Bearer headers, but write operations cost LLM money.

// Public vs Protected Endpoint Configuration[cite: 1]
const restApi = new apigateway.RestApi(this, "DomainExpansionRestApi", {
    restApiName: "Domain Expansion Serverless REST API",
});

const restAuthorizer = new apigateway.CognitoUserPoolsAuthorizer(this, "DomainExpansionRestApiAuthorizer", {
    cognitoUserPools: [props.userPool],
});

// 🔓 PUBLIC READ-ONLY ENDPOINTS (No Cognito Token required)[cite: 1]
apiResource.addResource("get-snapshot").addMethod("GET", lambdaIntegration, {
    authorizationType: apigateway.AuthorizationType.NONE, //[cite: 1]
});

// 🔒 PROTECTED WRITE ENDPOINTS (Strict Cognito Validation)[cite: 1]
apiResource.addResource("trigger-technique").addMethod("POST", lambdaIntegration, {
    authorizationType: apigateway.AuthorizationType.COGNITO, // Edge validation[cite: 1]
    authorizer: restAuthorizer,
});


Observability & Enterprise Tracing

Observability is handled via CloudWatch Vended Logs namespaces and AWS X-Ray:

// Configuring Vended Logs namespace for AgentCore[cite: 1]
const applicationLogGroup = new logs.LogGroup(scope, `${id}ApplicationLogGroup`, {
    logGroupName: `/aws/vendedlogs/bedrock-agentcore/gateway/APPLICATION_LOGS/${gateway.gatewayId}`, //[cite: 1]
    retention: logs.RetentionDays.THREE_DAYS,
});

This ensures end-to-end distributed tracing on a single X-Ray map:

Browser Gesture → API Gateway → Lambda Router → AgentCore Container → Strands LLM → AWS Polly TTS.


Smart Client Guards & Character Engineering

1. Cost-Guarding Session Logic (auth-check.js)

To avoid billing leaks from idle browser tabs keeping AgentCore sockets open, the client checks the Cognito JWT every 30 seconds on-device:

Session Expired = currentTime >= jwt.exp

If expired, it closes active WebSockets immediately:

function checkSessionGuard() {
    const token = getCognitoToken();
    if (isTokenExpired(token)) {
        if (webSocketConnection) webSocketConnection.close(); // Prevent billing leaks[cite: 1]
        triggerLocalLogout();
    }
}
setInterval(checkSessionGuard, 30000); // 0 cloud compute overhead[cite: 1]

2. Speech Text Sanitization (commentary_tts.py)

Before passing model outputs to AWS Polly, Markdown symbols (**bold**, #) are converted to clean plain text using BeautifulSoup to prevent Polly from reading syntax aloud:

import markdown
from bs4 import BeautifulSoup

def sanitize_text_for_tts(raw_text):
    html_formatted = markdown.markdown(raw_text)
    return BeautifulSoup(html_formatted, "html.parser").get_text() # Clean plain text[cite: 1]

3. Soul as Code Configuration

Character identity is fully decoupled into standard Markdown files loaded at runtime:

● IDENTITY.md: Directives on sassy Cantonese styling and demeanor.

● SOUL.md: Combat lore rules (e.g., how to roast Gojo Satoru users).


Architectural Economics & Pay-As-You-Go

Architecture Choice Cost & Efficiency Impact
Zero Idle Billing $0.00 cost when no matches are active; no continuous EC2/ECS servers.
S3 Lifecycle Rules Webcam snapshots & Polly MP3 audio streams auto-deleted after 7 days.
P2P Video WebRTC Video streams directly peer-to-peer; API Gateway WebSockets handle light telemetry only.

Closing Reflection: The Developer Lounge Vibe

What Cyrus Wong and the team demonstrated at the AWS Hong Kong Summit 2026 Developer Lounge was more than a fun fan game. It was a masterclass in modern serverless design: high fashion, high technology, low latency, and zero idle cost.

As the sun set over Central, illuminating Victoria Harbour, developers were still clustered around the booth, scanning CDK repositories and testing hand signs. That is the real spirit of the AWS Developer Lounge—bringing world-class engineering, vibrant local culture, and serverless power together under one roof.