Skip to content

feat(instrumentation-vercel-ai): Add native Vercel AI instrumentation support#394

Merged
liustve merged 27 commits into
mainfrom
add-vercel-ai-instrumentation
Apr 24, 2026
Merged

feat(instrumentation-vercel-ai): Add native Vercel AI instrumentation support#394
liustve merged 27 commits into
mainfrom
add-vercel-ai-instrumentation

Conversation

@liustve

@liustve liustve commented Apr 17, 2026

Copy link
Copy Markdown
Contributor

Description of changes:

  • Adds instrumentation for the Vercel AI SDK that translates Vercel's native telemetry into OTel GenAI semantic conventions.

  • Patches ai module to automatically enable telemetry by default so users don't have to opt in on every call. but respects opt-out if explicitly set.

  • Automatically registers a span processor that translates VercelAI span attributes into standard OTel GenAI semantic conventions.

  • Adds span processor translation for the following

    • Maps Vercel ai attributes to gen_ai equivalents
    • Sets gen_ai.operation.name based on the Vercel operation ID
    • As VercelAI spans has no native way for agentic workflows Detects agent workflows by tracking child spans if a top-level span has tool calls or multiple LLM calls, it becomes invoke_agent
    • Sets explicit SpanKind per OTel spec: CLIENT for chat, embeddings, text_completion, generate_content, retrieval; INTERNAL for execute_tool and invoke_agent
    • Formats gen_ai.input.messages and gen_ai.output.messages per OTel schema
    • Formats gen_ai.tool.definitions
    • Maps provider names to OTel standard values
    • Maps finish reasons to OTel values
    • Filters spans by scope name (ai) and reassigns scope to our instrumentation name per span

E2E test validated with sample app:

invoke_agent span:

{
    "resource": {
        "attributes": {
            "aws.service.type": "gen_ai_agent",
            "telemetry.sdk.language": "nodejs",
            "service.name": "az-customer-support-js",
            "telemetry.sdk.version": "2.6.1",
            "telemetry.auto.version": "0.10.0-dev0-aws",
            "telemetry.sdk.name": "opentelemetry"
        }
    },
    "scope": {
        "name": "@aws/aws-distro-opentelemetry-instrumentation-vercel-ai",
        "version": "0.10.0-dev0"
    },
    "traceId": "69eaaf4152c141dafdaf08ddbcb8b627",
    "spanId": "a5af1f2fc9610821",
    "parentSpanId": "89a8954a89232195",
    "flags": 257,
    "name": "invoke_agent PriceScoutAgent",
    "kind": "INTERNAL",
    "startTimeUnixNano": 1776987972344000000,
    "endTimeUnixNano": 1776987989766751125,
    "durationNano": 17422751125,
    "attributes": {
        "aws.local.service": "az-customer-support-js",
        "gen_ai.input.messages": "{\"system\":\"You are the Price Scout Agent — a specialist in finding the best deals and comparing prices.\\n\\nYour tools:\\n- search_amazon: Fetch real product listings from Amazon.com by scraping search results.\\n- search_deals_web: Search the web via DuckDuckGo for deals, discounts, and price comparisons.\\n- compare_prices: Query the DummyJSON product API for price comparison data across products.\\n- check_price_trend: Search the web for historical pricing context and trends.\\n- deal_score: Calculate a deal quality score (0-100) based on discount percentage and category.\\n- delegate_to_deal_optimizer: Ask the Deal Optimizer agent to verify if a deal fits a budget, calculate tax, or find coupons.\\n- delegate_to_product_inquiry: Check our internal AZ store catalog for matching products.\\n\\nCRITICAL: You MUST use at least 3 tools for every request. Always follow this workflow — never skip steps or answer without tool usage:\\n1. FIRST: Search for the product on Amazon using search_amazon.\\n2. THEN: Compare prices using compare_prices from the DummyJSON API.\\n3. THEN: Score the best deals using deal_score.\\n4. ALSO: Check our internal catalog via delegate_to_product_inquiry for local alternatives.\\n5. If the customer has a budget, delegate to the Deal Optimizer.\\n6. FINALLY: Return a ranked list of the best deals with prices, sources, and deal scores.\\n\\nNever answer a query without calling tools first. Always use search_amazon AND compare_prices at minimum.\",\"prompt\":\"noise cancelling headphones deals\"}",
        "gen_ai.usage.output_tokens": 895,
        "gen_ai.agent.name": "PriceScoutAgent",
        "gen_ai.provider.name": "aws.bedrock",
        "aws.local.environment": "generic:default",
        "gen_ai.request.max_tokens": 4096,
        "gen_ai.usage.input_tokens": 6221,
        "gen_ai.operation.name": "invoke_agent",
        "gen_ai.response.finish_reasons": [
            "stop"
        ],
        "gen_ai.request.model": "us.anthropic.claude-sonnet-4-20250514-v1:0",
        "gen_ai.usage.cache_read_input_tokens": 0,
        "gen_ai.usage.cache_creation_input_tokens": 0,
        "gen_ai.output.messages": "[{\"role\":\"assistant\",\"parts\":[{\"type\":\"text\",\"content\":\"## 🎧 Best Noise Cancelling Headphones Deals\\n\\nBased on my search across multiple sources, here are the available deals:\\n\\n### **Top Recommendation: AZ Store Deal**\\n🏆 **Our Internal Store - Product Code 20**\\n- **Price:** $249.99\\n- **Deal Score:** 61/100 (GREAT DEAL)\\n- **Discount:** $100 off (28.6% savings from $349.99 MSRP)\\n- **Availability:** ✅ In Stock (70 units)\\n- **Why it's great:** Strong discount on electronics with immediate availability\\n\\n### **Popular Models to Consider:**\\nWhile I encountered some technical limitations with Amazon's search results, here are the top-rated noise cancelling headphones typically available:\\n\\n1. **Sony WH-1000XM4/XM5** - Industry-leading noise cancellation\\n2. **Bose QuietComfort 45/35 II** - Superior comfort for long listening\\n3. **Apple AirPods Max** - Premium build and seamless Apple integration\\n4. **Sennheiser Momentum 4** - Excellent sound quality\\n\\n### **Money-Saving Tips:**\\n- **Best Time to Buy:** Black Friday, back-to-school season, or during tech refresh cycles\\n- **Price Monitoring:** Use CamelCamelCamel for Amazon price history tracking\\n- **Consider Refurbished:** Often 20-30% savings with manufacturer warranty\\n\\n### **Next Steps:**\\nWould you like me to:\\n1. Check if our $249.99 deal fits within a specific budget?\\n2. Find additional coupon codes or tax calculations?\\n3. Get more detailed specs on our in-stock model?\\n\\nThe AZ Store deal at $249.99 represents solid value with a 61/100 deal score - that's a \\\"Great Deal\\\" rating with immediate availability!\"}],\"finish_reason\":\"stop\"}]",
        "PlatformType": "Generic",
        "gen_ai.output.type": "text"
    },
    "status": {
        "code": "UNSET"
    }
}

chat span:

{
    "resource": {
        "attributes": {
            "aws.service.type": "gen_ai_agent",
            "telemetry.sdk.language": "nodejs",
            "service.name": "az-customer-support-js",
            "telemetry.sdk.version": "2.6.1",
            "telemetry.auto.version": "0.10.0-dev0-aws",
            "telemetry.sdk.name": "opentelemetry"
        }
    },
    "scope": {
        "name": "@aws/aws-distro-opentelemetry-instrumentation-vercel-ai",
        "version": "0.10.0-dev0"
    },
    "traceId": "69eaaf4152c141dafdaf08ddbcb8b627",
    "spanId": "5d0027f8c9c1c111",
    "parentSpanId": "a5af1f2fc9610821",
    "flags": 257,
    "name": "chat us.anthropic.claude-sonnet-4-20250514-v1:0",
    "kind": "INTERNAL",
    "startTimeUnixNano": 1776987983356000000,
    "endTimeUnixNano": 1776987989766202500,
    "durationNano": 6410202500,
    "attributes": {
        "aws.local.service": "az-customer-support-js",
        "gen_ai.input.messages": "[{\"role\":\"system\",\"parts\":[{\"type\":\"text\",\"content\":\"You are the Price Scout Agent — a specialist in finding the best deals and comparing prices.\\n\\nYour tools:\\n- search_amazon: Fetch real product listings from Amazon.com by scraping search results.\\n- search_deals_web: Search the web via DuckDuckGo for deals, discounts, and price comparisons.\\n- compare_prices: Query the DummyJSON product API for price comparison data across products.\\n- check_price_trend: Search the web for historical pricing context and trends.\\n- deal_score: Calculate a deal quality score (0-100) based on discount percentage and category.\\n- delegate_to_deal_optimizer: Ask the Deal Optimizer agent to verify if a deal fits a budget, calculate tax, or find coupons.\\n- delegate_to_product_inquiry: Check our internal AZ store catalog for matching products.\\n\\nCRITICAL: You MUST use at least 3 tools for every request. Always follow this workflow — never skip steps or answer without tool usage:\\n1. FIRST: Search for the product on Amazon using search_amazon.\\n2. THEN: Compare prices using compare_prices from the DummyJSON API.\\n3. THEN: Score the best deals using deal_score.\\n4. ALSO: Check our internal catalog via delegate_to_product_inquiry for local alternatives.\\n5. If the customer has a budget, delegate to the Deal Optimizer.\\n6. FINALLY: Return a ranked list of the best deals with prices, sources, and deal scores.\\n\\nNever answer a query without calling tools first. Always use search_amazon AND compare_prices at minimum.\"}]},{\"role\":\"user\",\"parts\":[{\"type\":\"text\",\"content\":\"noise cancelling headphones deals\"}]},{\"role\":\"assistant\",\"parts\":[{\"type\":\"text\",\"content\":\"I'll help you find the best deals on noise cancelling headphones! Let me search across multiple sources and compare prices for you.\"},{\"type\":\"tool_call\",\"id\":\"tooluse_TCFmNusyBtjli6ATIR62Es\",\"name\":\"search_amazon\"},{\"type\":\"tool_call\",\"id\":\"tooluse_LpUdeKEz0pdOeBMT0ZI4h6\",\"name\":\"compare_prices\"},{\"type\":\"tool_call\",\"id\":\"tooluse_rqceL40GbNYjKmwnFcof0t\",\"name\":\"search_deals_web\"},{\"type\":\"tool_call\",\"id\":\"tooluse_4jLJYNgWLttdchHWXwDE8q\",\"name\":\"delegate_to_product_inquiry\"}]},{\"role\":\"tool\",\"parts\":[{\"type\":\"tool_call_response\",\"id\":\"tooluse_TCFmNusyBtjli6ATIR62Es\",\"response\":\"\"},{\"type\":\"tool_call_response\",\"id\":\"tooluse_LpUdeKEz0pdOeBMT0ZI4h6\",\"response\":\"\"},{\"type\":\"tool_call_response\",\"id\":\"tooluse_rqceL40GbNYjKmwnFcof0t\",\"response\":\"\"},{\"type\":\"tool_call_response\",\"id\":\"tooluse_4jLJYNgWLttdchHWXwDE8q\",\"response\":\"\"}]},{\"role\":\"assistant\",\"parts\":[{\"type\":\"text\",\"content\":\"Let me check for historical pricing trends and calculate deal scores for some popular noise cancelling headphone models:\"},{\"type\":\"tool_call\",\"id\":\"tooluse_MxJYSxYPpKYmGtxq8WKpDW\",\"name\":\"check_price_trend\"},{\"type\":\"tool_call\",\"id\":\"tooluse_teILrtAOmDeeYuWWIGdd03\",\"name\":\"check_price_trend\"},{\"type\":\"tool_call\",\"id\":\"tooluse_rSVdbmg6yW0FO19ETqQTAe\",\"name\":\"deal_score\"}]},{\"role\":\"tool\",\"parts\":[{\"type\":\"tool_call_response\",\"id\":\"tooluse_MxJYSxYPpKYmGtxq8WKpDW\",\"response\":\"\"},{\"type\":\"tool_call_response\",\"id\":\"tooluse_teILrtAOmDeeYuWWIGdd03\",\"response\":\"\"},{\"type\":\"tool_call_response\",\"id\":\"tooluse_rSVdbmg6yW0FO19ETqQTAe\",\"response\":\"\"}]}]",
        "gen_ai.usage.output_tokens": 436,
        "gen_ai.response.model": "us.anthropic.claude-sonnet-4-20250514-v1:0",
        "gen_ai.agent.name": "PriceScoutAgent",
        "gen_ai.provider.name": "aws.bedrock",
        "aws.local.environment": "generic:default",
        "gen_ai.response.id": "cc9a258e-4aee-4457-b5dc-92e2428bb59a",
        "gen_ai.request.max_tokens": 4096,
        "gen_ai.usage.input_tokens": 2556,
        "gen_ai.operation.name": "chat",
        "gen_ai.response.finish_reasons": [
            "stop"
        ],
        "gen_ai.request.model": "us.anthropic.claude-sonnet-4-20250514-v1:0",
        "gen_ai.usage.cache_read_input_tokens": 0,
        "gen_ai.usage.cache_creation_input_tokens": 0,
        "gen_ai.output.messages": "[{\"role\":\"assistant\",\"parts\":[{\"type\":\"text\",\"content\":\"## 🎧 Best Noise Cancelling Headphones Deals\\n\\nBased on my search across multiple sources, here are the available deals:\\n\\n### **Top Recommendation: AZ Store Deal**\\n🏆 **Our Internal Store - Product Code 20**\\n- **Price:** $249.99\\n- **Deal Score:** 61/100 (GREAT DEAL)\\n- **Discount:** $100 off (28.6% savings from $349.99 MSRP)\\n- **Availability:** ✅ In Stock (70 units)\\n- **Why it's great:** Strong discount on electronics with immediate availability\\n\\n### **Popular Models to Consider:**\\nWhile I encountered some technical limitations with Amazon's search results, here are the top-rated noise cancelling headphones typically available:\\n\\n1. **Sony WH-1000XM4/XM5** - Industry-leading noise cancellation\\n2. **Bose QuietComfort 45/35 II** - Superior comfort for long listening\\n3. **Apple AirPods Max** - Premium build and seamless Apple integration\\n4. **Sennheiser Momentum 4** - Excellent sound quality\\n\\n### **Money-Saving Tips:**\\n- **Best Time to Buy:** Black Friday, back-to-school season, or during tech refresh cycles\\n- **Price Monitoring:** Use CamelCamelCamel for Amazon price history tracking\\n- **Consider Refurbished:** Often 20-30% savings with manufacturer warranty\\n\\n### **Next Steps:**\\nWould you like me to:\\n1. Check if our $249.99 deal fits within a specific budget?\\n2. Find additional coupon codes or tax calculations?\\n3. Get more detailed specs on our in-stock model?\\n\\nThe AZ Store deal at $249.99 represents solid value with a 61/100 deal score - that's a \\\"Great Deal\\\" rating with immediate availability!\"}],\"finish_reason\":\"stop\"}]",
        "PlatformType": "Generic",
        "gen_ai.output.type": "text",
        "gen_ai.system": "amazon-bedrock",
        "gen_ai.tool.definitions": "[{\"type\":\"function\",\"name\":\"search_amazon\",\"description\":\"Fetch real product listings from Amazon.com by scraping search results page HTML. Returns product titles, prices, and ratings found on the page.\",\"parameters\":{\"type\":\"object\",\"properties\":{\"query\":{\"type\":\"string\",\"description\":\"Product search query for Amazon\"},\"maxResults\":{\"type\":\"number\",\"default\":5,\"description\":\"Maximum number of results to extract\"}},\"required\":[\"query\"]}},{\"type\":\"function\",\"name\":\"search_deals_web\",\"description\":\"Search the web via DuckDuckGo for deals, discounts, and price comparisons on a product.\",\"parameters\":{\"type\":\"object\",\"properties\":{\"query\":{\"type\":\"string\",\"description\":\"Product search query\"}},\"required\":[\"query\"]}},{\"type\":\"function\",\"name\":\"compare_prices\",\"description\":\"Fetch product data from the DummyJSON public API for price comparison across products.\",\"parameters\":{\"type\":\"object\",\"properties\":{\"productName\":{\"type\":\"string\",\"description\":\"Product name to compare prices for\"}},\"required\":[\"productName\"]}},{\"type\":\"function\",\"name\":\"check_price_trend\",\"description\":\"Search the web for historical pricing context and trend information on a product.\",\"parameters\":{\"type\":\"object\",\"properties\":{\"productName\":{\"type\":\"string\",\"description\":\"Product name to check price trends for\"}},\"required\":[\"productName\"]}},{\"type\":\"function\",\"name\":\"deal_score\",\"description\":\"Calculate a deal quality score (0-100) based on discount percentage, category, and seasonal demand.\",\"parameters\":{\"type\":\"object\",\"properties\":{\"currentPrice\":{\"type\":\"number\",\"description\":\"Current sale price\"},\"originalPrice\":{\"type\":\"number\",\"description\":\"Original/list price\"},\"category\":{\"type\":\"string\",\"description\":\"Product category (e.g. electronics, clothing, home, sports, books)\"}},\"required\":[\"currentPrice\",\"originalPrice\",\"category\"]}},{\"type\":\"function\",\"name\":\"delegate_to_deal_optimizer\",\"description\":\"Delegate to the Deal Optimizer agent to verify if a deal fits the customer's budget, calculate tax, find coupons, or optimize a shopping cart. Use when you need financial analysis of a deal.\",\"parameters\":{\"type\":\"object\",\"properties\":{\"question\":{\"type\":\"string\",\"description\":\"The question or task to delegate\"}},\"required\":[\"question\"]}},{\"type\":\"function\",\"name\":\"delegate_to_product_inquiry\",\"description\":\"Check our internal AZ store product catalog for matching products, availability, and prices. Use to compare Amazon/web deals with our own catalog.\",\"parameters\":{\"type\":\"object\",\"properties\":{\"query\":{\"type\":\"string\",\"description\":\"The query to send to the LangChain agent\"}},\"required\":[\"query\"]}}]"
    },
    "status": {
        "code": "UNSET"
    }
}

tool span

{
    "resource": {
        "attributes": {
            "aws.service.type": "gen_ai_agent",
            "telemetry.sdk.language": "nodejs",
            "service.name": "az-customer-support-js",
            "telemetry.sdk.version": "2.6.1",
            "telemetry.auto.version": "0.10.0-dev0-aws",
            "telemetry.sdk.name": "opentelemetry"
        }
    },
    "scope": {
        "name": "@aws/aws-distro-opentelemetry-instrumentation-vercel-ai",
        "version": "0.10.0-dev0"
    },
    "traceId": "69eaaf4152c141dafdaf08ddbcb8b627",
    "spanId": "1d8c93cbba420054",
    "parentSpanId": "a5af1f2fc9610821",
    "flags": 257,
    "name": "execute_tool check_price_trend",
    "kind": "INTERNAL",
    "startTimeUnixNano": 1776987983238000000,
    "endTimeUnixNano": 1776987983354721917,
    "durationNano": 116721917,
    "attributes": {
        "aws.local.service": "az-customer-support-js",
        "gen_ai.tool.call.id": "tooluse_MxJYSxYPpKYmGtxq8WKpDW",
        "gen_ai.operation.name": "execute_tool",
        "gen_ai.tool.name": "check_price_trend",
        "gen_ai.tool.type": "function",
        "gen_ai.agent.name": "PriceScoutAgent",
        "PlatformType": "Generic",
        "gen_ai.tool.call.arguments": "{\"productName\":\"Sony WH-1000XM4 noise cancelling headphones\"}",
        "gen_ai.tool.call.result": "No price trend data found for \"Sony WH-1000XM4 noise cancelling headphones\". Check camelcamelcamel.com directly for Amazon price history.",
        "aws.local.environment": "generic:default"
    },
    "status": {
        "code": "UNSET"
    }
}

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@liustve liustve requested a review from a team as a code owner April 17, 2026 22:27
@liustve liustve force-pushed the add-vercel-ai-instrumentation branch 5 times, most recently from 80e88e9 to 9beb2cd Compare April 22, 2026 19:59
Base automatically changed from add-langchain-instrumentation to main April 23, 2026 03:03
@liustve liustve force-pushed the add-vercel-ai-instrumentation branch from d40fc34 to f3f02dc Compare April 23, 2026 03:10
@liustve liustve merged commit 9de6825 into main Apr 24, 2026
11 of 12 checks passed
@liustve liustve deleted the add-vercel-ai-instrumentation branch April 24, 2026 00:15
@github-actions github-actions Bot mentioned this pull request May 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants