← Blog

Field guide

Financial GenAI Platform Engineering: Building Operational Agentic AI with Cloud-Native Architecture

From field IT reality — an engineering path for deployment, scaling, monitoring, and finance-grade trustworthy answers

Financial GenAI Platform Engineering: Building Operational Agentic AI with Cloud-Native Architecture

Over the past year, creating GenAI demos has become relatively easy. But the real challenge for the financial industry lies in: how AI enters the actual operational environment—can it be deployed, scaled, and monitored; can it refuse to answer when there is insufficient evidence; can it stably support users from Web, Teams, and voice; can it leave an auditable trail?

This article is written for enterprise AI / platform engineers, architects, and technical decision-makers. The core problem it solves is: how to use cloud-native architecture to engineer generative AI from a PoC demo into a governable, observable, and verifiable financial-grade Agentic AI Runtime and retrieval workflow.

This article explicitly does not discuss open-domain casual chitchat, does not cover high-risk autonomous financial trading or loan underwriting decisions, and does not expand into multi-tenant enterprise control planes or legal liability assignments here (the latter is addressed in the next installment, post 39).

This article focuses on how to keep the platform running stably (Runtime, deployment, monitoring, RAG workflow). For the governance perspective of enterprise-level Control Plane, responsibility decomposition, and Agentic Operating System, please refer to the next article in the series: Financial-Grade Enterprise Agentic AI Architecture Design.

Slide PDF

Starting from a Field Operation Scene

Imagine this: a field colleague is providing on-site support at a client’s location when they suddenly encounter an IT issue—permission request blocked, device unable to connect, or an error message pops up on the screen and they don’t know who to contact.

At this moment, it is not suitable for them to stop and type to search for documents, nor can they wait for lengthy replies. In front of the client, they can only ask a question via voice: “Who should handle this error message?”

The user’s need is very clear: immediate response is required. But the financial industry’s requirements go beyond this. The AI’s answer cannot stop at just seeming reasonable; the system must verify internal knowledge, assess whether the evidence is sufficient; clearly refuse to answer when insufficient, and leave a tracing record throughout the entire process.

This tests not whether a chatbot can answer questions, but whether AI can truly enter the operational environment.

Three Lifelines for Financial AI Deployment

For financial AI to be deployed, it must simultaneously meet three operational conditions:

ConditionChallengeWhat the platform needs to answer
IntegrationKnowledge bases, permissions, ITSM, M365, and process documents are siloedCan the Agent call enterprise systems and knowledge sources in a consistent manner?
Real-time PerformanceVoice and field operations cannot tolerate a wait of over ten secondsHow can retrieval, validation, and rewriting in high-quality RAG be completed within acceptable latency?
ComplianceAuditing asks “why did it answer this way?”What data was checked, what tools were called, and is it trackable and replayable?

The challenge of financial AI does not lie in the inability to build AI, but in whether it can simultaneously pass these three conditions. This depends on platform capabilities, rather than simply upgrading model scale.

Why Do AI Projects Often Get Stuck at PoC?

Most AI projects are not fruitless, but they remain at the PoC stage. There are three common breakpoints:

1. System Silos Each scenario requires custom integration, making it difficult for Agents to use enterprise tools at scale; with every additional scenario, integration costs increase by another layer.

2. Linear RAG Generating directly after Retrieving may seem reasonable in process, but the system cannot judge whether the retrieved data is sufficient, lacking self-correction and evidence checking.

3. Black Box AI Unable to explain data and tool sources, auditing and compliance will directly block the launch. The financial industry cannot just accept AI answering “I think so”.

These three breakpoints cannot be solved by changing models, but require platform engineering—standardizing tools, making processes self-correcting, and leaving a trail for every answer.

Cloud Native AI Runtime: Three-Tier Architecture

To truly launch, the primary issue is not the model, but the runtime—this Agentic AI must be able to be deployed, scaled, monitored, and governed.

I consolidate the architecture into a three-tier understanding:

Tier 1: Controlled Entry

Users may enter through Web, Teams, or Mobile Voice. This is a logical control boundary, not a claim that every channel traverses the same physical gateway. Different protocols may have separate adapters, but each entry path needs authentication, authorization, quota enforcement, and rate limits.

Tier 2: Runtime Orchestration

The Agent Orchestrator runs on Cloud Run (or a similar containerized runtime) and is responsible for intent routing, Agent coordination, context validation, and response generation. Underneath it connects to the Retrieval Service of Hybrid Search and the MCP Tool Hub, enabling the Agent to call enterprise tools in a consistent manner.

Tier 3: Observability and Governance

From day one of launch, it must be able to log, measure, trace, and leave an audit trail. Externally, it manages four things with SLO:

  • Latency — Can it sustain voice and on-site interactions?
  • Error Rate — Is the service stable?
  • Refusal Rate (classified) — Separate justified refusal, missing evidence, denied access, and incorrect refusal; lower is not always better.
  • Trace Completeness — Can key decisions and authorization events be reconstructed without exposing sensitive payloads?

The value of Cloud Native is not in “putting AI on the cloud”, but in allowing AI services to be managed with SLOs, audited with Traces, and scaled through Runtime.

MCP: Turning Enterprise Tools into Governable Capabilities

If every AI project re-integrates APIs, it’s just renaming the system integration problem—forming API Spaghetti, where every additional scenario adds a layer of custom costs.

The value of MCP (Model Context Protocol) lies in encapsulating internal systems, M365, databases, and IT processes into standardized tool interfaces. Agents can invoke tools through a consistent contract, but MCP alone does not produce an audit-ready Tool Trace. The platform must instrument trace IDs, caller identity, tool name, parameter summaries, authorization outcomes, and errors without logging unnecessary sensitive data.

For the financial industry, tool usage is not free exploration, but tracked, governable usage limited within authorized scopes. MCP makes tools a platform capability, rather than custom code for a specific project.

Data Engineering and Hybrid Search: The Ceiling of RAG

Data quality determines the ceiling of RAG. If the data is not clean, even the most powerful model will struggle to produce credible answers. Financial industry documents cover PDFs, scanned copies, tables, process manuals, and error codes; a single parser cannot cover all formats.

In practice, we adopt hybrid parsing: text-heavy documents are processed with fast parsing, scanned copies through Vision APIs, paired with Semantic Chunking to preserve context, avoiding cutting the same segment of business logic too finely.

For retrieval:

  • Embedding excels at semantic similarity and synonymous rewriting—especially effective when user queries differ from document phrasing.
  • BM25 excels at precise matching of system names, process codes, and proper nouns.
  • Finally, use RRF (Reciprocal Rank Fusion) for rank fusion, making the two complement each other.

The first step of financial-grade RAG is not generation, but enabling the Agent to obtain verifiable evidence.

From Linear RAG to Agentic RAG

The traditional RAG process is straightforward: Retrieve, then Generate.

But the financial industry cannot just rely on a unidirectional process. The model finding data does not mean the data is sufficient to answer; data seeming relevant does not mean it constitutes correct evidence.

Agentic RAG changes to a dynamic workflow:

  1. Route to the correct data source.
  2. Hybrid search.
  3. Validate if evidence is sufficient—if not, rewrite the query and retrieve another round.
  4. Refuse to answer or guide to supplement when necessary.
  5. Leave an Agent Trace throughout the entire process.

Its core difference lies in: this is not a one-time retrieval, but a self-correcting workflow.

For more detailed context on Agentic RAG, you can refer to my previous summary: Agentic RAG: Vector Search Meets Agentic Reasoning.

Financial-Grade Accuracy: A Safe Trust Boundary

In financial scenarios, AI answering incorrectly can constitute compliance risks. Therefore, financial-grade accuracy does not mean answering every question, but every answer must be supported by evidence.

The decision boundary can be simplified as:

  • Sufficient, traceable evidence within the caller’s permissions → Answer
  • Insufficient or conflicting evidence → Retrieve again; refuse or ask for more context if uncertainty remains
  • High-risk task → Hand off to a human; autonomy is not authorization

Every judgment must leave an Agent Trace: the question, retrieval sources, tool calls, and decision path should all be replayable. Accuracy is therefore a workflow property formed jointly by data, retrieval, validation, refusal, and auditability—not a feature of one model.

The value of Agentic AI is not in complete autonomy, but in operating autonomously within controllable boundaries.

Evaluation: Define “Correctness” First, Then Talk About Accuracy Rate

In the financial industry, one cannot simply claim “high accuracy rate”; the scoring method must first be defined. We base it on a 100-question RAG Benchmark, adopting a four-level scoring system:

LevelDefinition
CorrectComplete hit, no incorrect information
Partially CorrectDirection is correct but details are insufficient (counted in weighted accuracy rate)
Correctly RefusedClearly refused when evidence is insufficient or it should not answer—this is safe behavior, not a failure
Incorrect or UnsafeInconsistent with correct answers, misquoted, or answered when it shouldn’t have—zero tolerance

The accuracy rate is not just the correct answer rate, but must measure whether it can avoid incorrect and unsafe answers.

Real-Environment Evaluation Data

The 100-question Benchmark covers high-frequency FAQs, synonymous rewriting, questions that should be refused, trap questions, and boundary questions.

The scope of application must be stated first: this is not claiming AI can handle all high-risk financial decisions, but rather verifying the credible answering capability of Agentic Runtime in low-risk, high-frequency, clearly processed IT tasks.

MetricResult
Weighted Accuracy Rate98%
Strict Correctness Rate96% (96 completely correct, 4 partially correct)
Incorrect or Unsafe Answers0 questions
Complete Agentic Workflow Avg Latency3.56 seconds
P95 Latency6.19 seconds (including retrieval, validation, rewriting, refusal judgment, and Trace)

Ablation is worth noting:

SetupAccuracy Rate
Naive RAG87%
Hybrid Search Only83.5%
Complete Agentic RAG98%

Retrieving more documents does not guarantee better answers. Hybrid-only scored below Naive RAG in this ablation, which does not establish that hybrid retrieval is universally worse. Candidate recall, context validation, and refusal must be evaluated with frozen questions, scoring rules, indexes, and versions; the latter controls matter in this particular case.

This evaluation protocol is grounded in the implementation documented in the site’s Agentic RAG Engineering Case. In v2.2, missing state validation once allowed Swagger filter placeholder parameters to reach generation; post-retrieval Context Validation and rule-first routing addressed that implementation failure. See the project page for the architecture and evaluation evidence.

High-frequency FAQs can take the fast path; boundary and permission questions go through full validation. This split depends on correct routing, and both paths should retain replayable traces.

Concrete Trade-offs and Engineering Costs

Adopting a full Agentic workflow is not without cost; landing this architecture requires three explicit trade-offs:

  1. Latency and inference cost: Routing, evidence validation, and retrieval retries add steps and model calls. The end-to-end latency in the table captures this governance trade-off; accuracy alone omits user wait time and additional token expenditure.
  2. Maintenance Overhead: Hybrid Search requires maintaining both a vector database (e.g., pgvector) and an inverted keyword index (BM25), alongside tuning RRF (Reciprocal Rank Fusion) fusion weights for domain terminology. Multi-step validation also adds prompt version management complexity.
  3. Cognitive and Architectural Burden: Engineering teams must maintain state machines, branching logic, and graceful fallback boundaries rather than calling a single LLM completion endpoint. Debugging issues requires correlated analysis across retrieval logs, tool traces, and model inference records.

What this case does not establish

  • Channel interoperability: Web, Teams, and voice are architectural entry patterns, not proof of one deployment’s end-to-end testing across all three.
  • Universal safety: Zero unsafe outcomes on 100 frozen cases do not replace prompt injection, ACL, tenant-isolation, or revocation tests.
  • Latency transferability: Mean 3.56 seconds and P95 6.19 seconds describe the measured workflow, not a different model, corpus, or concurrent load.
  • Knowledge publication: Source citations and traces do not approve draft knowledge. Versioning, effective dates, review, publish, and rollback remain separate governance controls.

These are proposed acceptance checks derived from the case, not completed production or penetration tests.

Revalidate Before Extending the Case to New Domains

This case supports an IT knowledge workflow within a defined scope; it does not show that customer service, compliance, or internal control can inherit the same results. Architecture components may be reusable, but metrics are not transferable: each new domain needs representative questions and refusal cases, checks for source freshness and permission boundaries, and measurements of errors, refusals, and latency under the same scoring rules before traffic is expanded.

This is an engineering recommendation synthesized from the case, not a cross-domain test reported in the article.

Scope Boundaries and Production Decision

This architecture has clear boundaries; engineering teams should exercise discipline during architectural selection:

  • Evidence boundary: This is a first-party case based on an internal IT knowledge base; no independent rerun is reported here. It does not establish that high-risk financial transactions, loan underwriting, or regulatory compliance can be fully automated. Poorly scanned documents and unknown policies outside the knowledge base still require human review.
  • When NOT to Adopt (Anti-Patterns): If a business scenario only requires ultra-low latency (<500ms) static FAQ queries or deterministic lookups with high fault tolerance, forcing a multi-step Agentic state machine (routing → hybrid search → validation → rewriting) is textbook over-engineering. A deterministic rule engine or simple key-value cache is vastly more cost-effective.

Therefore, use a multi-step Agentic workflow only when cross-system integration, refusal under weak evidence, and auditability outweigh its added latency and maintenance. The goal is not a more autonomous model, but a measurable, replayable path that stops at evidence or permission boundaries.

Frequently Asked Questions

Can these evaluation results be applied directly to another business domain?

No. It describes only the internal IT question set reported here. Customer service, compliance, or another knowledge domain needs its own representative sample, answer/refusal boundaries, and evaluation under a consistent scoring protocol.

Does MCP automatically apply fine-grained permissions to every tool?

No. The current MCP specification makes HTTP authorization an optional transport-level capability. Even when tokens and scopes are enabled, tool services still need to define application permissions and check whether a user may read or write the target data on each operation. A server token is not permission to perform every business action.

Three focused paths connecting architecture, contract, and engineering implementation:

  1. Next in Architecture: Financial-Grade Enterprise Agentic AI Architecture Design: From Demo to Agentic Operating System — Step from runtime into the control plane, exploring 15+ agent responsibility decomposition and E·P·J·T governance.
  2. Production Review Contract: Agentic AI Platform Contract: The Control Plane You Must Wire Before Production — Turn the control plane into a checkable production gate with seven non-bypass rules.
  3. Featured Implementation: Agentic RAG Engineering Case — Inspect first-party architecture, Swagger failure remediation, and benchmark records cited here.

Method Sources and Evidence Boundary

The Cloud Native AI Runtime, three lifelines, and evaluation design in this article are the author’s engineering framework presented at Cloud Summit, not an external standard. The evaluation data come from this site’s published low-risk IT/process case and are not a general accuracy claim for financial decisions.