Resume Contact
← Back to cases

Alfa Bank · 10.2022 - 06.2024

AI/LLM support automation in regulated banking

Product Owner for an AI/LLM support automation product integrated into support tooling, CRM and knowledge base systems.

S

Situation

A regulated banking support environment needed to reduce operator load while preserving reliability, compliance and control in customer-facing workflows.

T

Task

Own scope, stakeholder alignment, rollout, KPI tracking and API integration for an AI-assisted routing product that could classify intent, route cases and escalate safely.

A

Action

Launched an LLM intent classifier with API-driven routing, escalation logic, fallback scenarios and monitoring signals. Improved knowledge automation through LLM/RAG and coordinated Product, Engineering, Support and Compliance stakeholders.

R

Result

Reduced operator-handled contacts by about 30%, improved FCR by +12 pp, increased containment from 45% to 62% and reduced repeat contacts by 15%.

Step-by-step

How I approached the automation rollout

The product work balanced customer experience, support efficiency, API integration and control requirements expected in a regulated banking environment.

  1. Mapped support intents and escalation paths. Grouped incoming contacts by intent, ambiguity, risk level and required human involvement.
  2. Defined automation boundaries. Separated safe self-service intents from cases requiring operator review, compliance handling or fallback logic.
  3. Designed API-driven routing. Connected classifier outputs to routing, escalation, CRM context and knowledge base retrieval.
  4. Improved answer quality with LLM/RAG. Used knowledge automation to make guidance more accurate and reduce repeat contacts caused by incomplete answers.
  5. Built monitoring and controls. Defined confidence thresholds, fallback scenarios, escalation rules and signals for quality drift.
  6. Rolled out through KPI-based iterations. Tracked containment, FCR, repeat contacts, operator load and misroutes to decide where automation should expand or contract.

Artifacts

Portfolio artifacts used in the case

Flow

AI-assisted support routing map

Contact

Customer submits support request through existing channels.

Input: Message, profile, CRM context.

Classify

LLM intent classifier detects intent, confidence and possible risk.

Signal: Confidence score.

Retrieve

RAG layer pulls approved knowledge base content for response generation.

Control: Approved sources only.

Route

API logic sends the case to self-service, automation or operator queue.

Rule: Escalate low confidence.

Monitor

Quality, misroutes, repeat contacts and fallback events are tracked.

Outcome: Reliable automation.
Controls

Automation risk matrix

RiskControlSignal
Wrong intentEscalation thresholdMisroute rate
Bad answerApproved KB retrievalRepeat contact
Over-automationFallback scenariosOperator review
KPI

Support automation KPI tree

Customer support efficiency
Containment FCR Repeat contacts Operator load Automation risk
SWOT

AI support automation SWOT

Strengths

High-volume support data, clear routing opportunities, measurable KPIs.

Weaknesses

Regulatory constraints, knowledge freshness, classifier ambiguity.

Opportunities

Higher containment, improved FCR, better self-service guidance.

Threats

Automation errors, customer trust loss, compliance exceptions.

Rollout

Release plan

Pilot

Low-risk intents with human monitoring.

Scale

Expand to high-volume intents after KPI validation.

Control

Monitor drift, fallback events and escalation quality.