Head of CS says AI can’t run independently in CX workflows — it needs constant oversight

Currently, I'm the Head of Customer Success at Enginy, an AI-native end-to-end intelligent GTM platform for B2B sales teams. We are a vertical SaaS in the sales automation space, serving all segments from SMB to Enterprise. And we're seed-stage, growing this year from $3M to $10M ARR.

We currently have a high-touch customer experience model, transitioning to a segmented one. Every customer has access to Product Support via in-app chat and email. We also pair them with an onboarding manager and an account manager. We serve our clients via meetings, phone calls, and email. Earlier this year, we introduced weekly onboarding webinars to scale onboarding and training.

I oversee Account Management, Onboarding, and Product Support with 12 people. We're planning to end the year with 20.

How two simple changes can affect a CX team's efficacy

I've made two concrete changes in the last year:

The first was customer-facing. We use FIN (Intercom's AI agent) for support deflection. Over four months, the resolution rate climbed 10 percentage points. This meant fewer tier-one tickets reached the team, and the tickets that reached a human were genuinely complex. That allowed us to reallocate the team's time.

However, the CX Score dropped 9 points in February. We quickly understood why: FIN gave wrong answers because outdated content fed it. It confidently responded to questions it shouldn't have answered, and the escalation logic failed to catch it. Customers became stuck.

We fixed this by conducting a content audit and a full review of escalation rules: determining what FIN should handle, what it should immediately route, and what happens when it doesn't know. We rewrote the escalation logic to hand off earlier on signals like repeated rephrasing, frustration markers, and specific topic categories like billing and bugs. By April, the CX Score recovered and sat 5 points above our starting point.

Angela Shares

…AI in support is not a one-time deployment deflection tool. It’s a system with a decaying content layer, and if you don’t maintain it, you run a confident bot that erodes customer trust slowly enough that you don’t notice until the numbers move.

The learning here was that AI in support is not a one-time deployment deflection tool. It's a system with a decaying content layer, and if you don't maintain it, you run a confident bot that erodes customer trust slowly enough that you don't notice until the numbers move. The February drop showed the numbers moving. The upside is the clean signal. When something goes wrong, you can trace it. This is more than you get with a human team at scale.

The second change was internal. We improved our call-prep workflow by connecting Claude to Granola, Intercom, HubSpot, and our own product. Before, AMs spent 20–30 minutes gathering context prior to every call: last meeting notes, open support tickets, deal stage, and product usage.

Now, Claude pulls all of it into one pre-call brief automatically. AMs walk in with precise context instead of generic check-ins. This allowed us to cut prep time, and the hours we save go into proactive outreach and value conversations.

How AI-powered churn analysis enhances decision making

Let's dive into a churn-analysis workflow I built in Claude. The obvious starting point for churn analysis is the "churn reason" field in the CRM. It's also where most of the distortion lies. The Account Manager fills in that field after losing the deal, under pressure to close the month. It reflects how the AM framed the loss, not necessarily why the customer left.

I started with a manual AI prompt I'd run each month. The output was inconsistent; I spent more time fixing it than I saved, so I built an automated skill that pulls directly from HubSpot with clear rules of which properties and activities to look for in the CRM.

It pulls every churned and at-risk deal for a given time window, normalizes CRM labels, and clusters by multiple dimensions, including an inferred reason based on the full deal history: notes, call transcripts, activities, and onboarding completion. Crucially, it reassigns churn ownership by causal logic, not by what the AM recorded.

For example, a recent deal was logged as "No ROI," which most people read as a CS failure. But the deal notes showed the sales process set expectations that the product never could have met. That's a sales qualification problem, not a CS one. We would have needed someone reading every note on every churned deal (which doesn't happen) to catch that without AI.

The output is a CEO-ready doc plus separate extracts for CS, Product, and Sales, each with the cases and root-cause analyses that belong to them.

Why the insight-to-outcome flow must be proven before adding AI

I deploy AI where a human has already proven the insight-to-outcome flow. If that flow is still being tested, AI doesn't touch it — that validation work can't be skipped. If you automate before you've confirmed the pattern, you scale the wrong behavior.

Here's where AI currently does the work:

• Analysis: renewal forecasting and churn pattern detection. Here's the clearest example. I stopped trusting AM-stated churn reasons (that cluster around "budget" and "no fit," as they are easy to log) and started inferring real reasons from notes, transcripts, and product usage. AI surfaced stalled onboarding behind half the "budget" labels. That moved our intervention upstream.
• Summarization: meeting prep, follow-ups, weekly synthesis of decisions. The only way one CSM can carry the relationship density we need at our efficiency target.
• Support chat: first-line answers grounded in documentation that a human reviewed first. Order matters.

We try to keep a human in the loop. When AI suggests an action (risk flags, expansion signals, next steps), a CSM or I review and validate before it reaches the customer. AI catches patterns faster than I do, but the person who carries the relationship retains judgment.

As I said, I need to prove the insight-to-outcome flow myself before integrating AI, so AM coaching and performance reviews are the clearest examples of tasks that have remained fully human. I'll delegate it eventually, but first, I need to test across my AMs whether pairing specific actions with specific signals drives better portfolio outcomes. Until that pattern holds in 1:1s, automating it merely scales a guess.

Why CX leaders must provide AI with context

This is particularly important with chatbots — work on your customer-context layer. An agent without full customer context doesn’t just give a bad answer; it gives a confident, wrong one…

Share This Quote on:

• Share on Twitter
• Share on LinkedIn
• Share on Facebook

Angela GuedesOpens new window

Head of Customer Success at Enginy

Never delegate thinking and never skip context.

Remember, AI output is only as good as the context you feed it. And nothing can replace human thinking. Yes, use AI to challenge your assumptions, to find holes in your thinking, to benchmark your ideas against industry standards. But for relevant output, you need to give it the context from your head and internal documents.

If you just ask it to build something without this, the output will be bland at best and irrelevant in most cases.

This is particularly important with chatbots — work on your customer-context layer. An agent without full customer context doesn't just give a bad answer; it gives a confident, wrong one. And remember that chatbots can't just run by themselves. Someone must own the content feeding the AI, define escalation rules, and actively QA its output.

How AI can reveal gaps in your data

My advice? Start with your data, not your tools.

The most useful thing AI can do right now is surface what's broken in your existing systems — not generate a report, but find gaps. My first churn analysis with Claude didn't tell me why customers were leaving; it showed me that AMs tagged churn reasons inconsistently, CRM data was outdated, and critical fields were missing. That's a data problem that I didn't fully see until AI tried to work with it.

That's the starting point. Understand what you have, fix inconsistencies, and enrich what's missing. AI will only be as good as the inputs you give it. Leaders who skip this step will automate noise and wonder why the insights don't land.

Clean data isn't glamorous advice, but it's the prerequisite.

Angela Shares

My advice? Start with your data, not your tools…That’s the starting point. Understand what you have, fix inconsistencies, and enrich what’s missing…Clean data isn’t glamorous advice, but it’s the prerequisite.

Follow along

Follow along with Angela Guedes on her LinkedIn and Substack.

More expert interviews to come on the CX Lead!