How AWS SageMaker, MCP servers, and Arize AI enable production-ready, observable, and scalable agentic AI workflows
🚀 Introduction
The hype around AI agents and the Model Context Protocol (MCP) is everywhere. But how do these concepts translate into real, enterprise-grade solutions?
In this post, we distill a financial services loan underwriting use case built on AWS SageMaker with Arize AI for observability. The goal: show how to design a scalable, compliant, and production-ready agentic AI architecture.
🧩 AWS Services for Generative AI
AWS offers two primary paths for deploying generative AI models:
• Amazon Bedrock → Provides API-based access to managed models (e.g., Claude).
– Best for developers who want to build apps quickly, without worrying about infrastructure.
• Amazon SageMaker → Full-control platform to train and deploy open-source or custom models.
– Ideal for ML teams who need GPU control, fine-tuned training, and flexible deployment.
👉 For our loan underwriting demo, we chose SageMaker to deploy an open-source Qwen model on GPU infrastructure.
🤖 Agents, Tools, and MCP Servers
The Challenge with Agents
• Agents excel at specific tasks.
• But enterprise workflows (like loan underwriting or insurance claims) often require multiple steps, context handoffs, and orchestration.
• Writing custom multi-agent DAGs for every use case is not scalable.
The Solution: Tools + MCP Servers
• Instead of rigid orchestration, create tools that agents can call for specialized tasks:
– Fetch credit data
– Process PDFs
– Clean applicant profiles
• Wrap these tools as MCP servers:
– Agnostic to models and frameworks
– Reusable across agent frameworks (LangGraph, LangChain, AWS Strands)
– Scalable via containers (Kubernetes/ECS/EKS)
– Discoverable at runtime (agents can query available MCP servers dynamically)
– Flexible (implemented in Python, JavaScript, or any language)
🌐 Advantages of MCP Servers
• Agnosticism → Standard protocol works with any framework.
• Scalability → Containerized MCP clients auto-scale with usage.
• Dynamic Discovery → No pre-wired DAGs; agents discover servers on the fly.
• Flexibility → Build in any language, deploy anywhere.
🔎 The Importance of Observability
For industries like finance and healthcare, observability is not optional.
• Tracing → Identify where errors or latencies occur in complex, multi-layer workflows.
• Explainability → Capture why a decision was made (key for compliance and auditing).
• Compliance → Maintain audit trails that regulators can inspect.
This is where Arize AI comes in.
🏦 Loan Underwriter Demo Architecture
We implemented a simplified loan underwriting pipeline with three MCP servers arranged in a DAG:
1. Loan Officer MCP Server
– Cleans and summarizes applicant profile
– Input: {age, income, loan_amount, credit_score, liabilities, purpose}
2. Credit Analyzer MCP Server
– Builds credit profile
– Assesses creditworthiness (low / medium / high)
3. Risk Assessor MCP Server
– Consumes credit assessment
– Issues final decision (approve / deny)
Backend: — SageMaker hosts the Qwen model on an ml.g5 GPU instance. — Input was provided as JSON, but natural language input could also be parsed by the LLM. — This demo did not use RAG, though it could be extended with retrieval pipelines.
📊 Observability with Arize AI
Arize AI is integrated to provide end-to-end visibility:
• Agent-Level Tracing → See which agents and MCP servers were invoked.
• Granularity → Inspect inputs/outputs for every step in the decision chain.
• Evaluation Metrics → Track latency, execution time, and performance.
• OpenTelemetry → Native integration with LangChain, LangSmith, and other open-source observability stacks.
• Simple Integration → Just a few lines of tracer initialization code instrument the entire workflow.
🏗️ Architecture Diagram
✅ Key Takeaways
• Agents alone are not enough → Enterprise workflows demand scalable, composable tools.
• MCP servers provide a universal protocol for agents to call specialized services.
• SageMaker powers flexible model deployment, while Bedrock fits lightweight API-based use cases.
• Observability with Arize AI ensures compliance, explainability, and production readiness.
🎯 Conclusion
Agentic AI is moving fast from POCs to production in regulated industries. By combining SageMaker, MCP servers, and Arize AI, enterprises can:
• Build modular, reusable workflows
• Scale reliably across business units
• Meet the stringent compliance and observability requirements of finance and healthcare
This architecture isn’t just experimental — it’s the blueprint for real-world, production-grade agentic AI.