Seasoned Tech CRO Says AI Can’t Replace Domain Expertise
David Karp explains how AI can scale customer success and predict churn while showing why domain expertise, human empathy, and strong customer data remain essential.
David Karp
SVP of Customer Success at Billtrust
Health Scoring: A 12-signal AI model predicts churn weekly, helping teams intervene within days instead of months.
Scaled Support: Agentic support and automated workflows could expand coverage from 500 accounts to nearly 2,000 without proportional hiring.
Human Judgment: AI handles signals and routine service, but humans remain essential for empathy, recovery, and complex relationships.
Data Foundation: Customer experience AI works best when integrated product, sentiment, interaction, and outcome data provide complete client context.
New Roles: Product Value Architects pair domain expertise with AI signals, designing targeted interventions that close customer value gaps.
David Karp has 30+ years of experience in post-sales leadership. Currently, he's the SVP of Customer Success at Billtrust, a $200M+ SaaS company.
We talked to David to understand what aspects of CX are changing. He told us about the new workflows — and new roles — he's creating.
A modest-but-mighty CS team
As a seasoned leader with over 30 years of experience in post-sales leadership, I have pursued one mission: to transform customer relationships into powerful growth engines.
My passion lies in harmonizing culture, technology, and data. As a certified mentor and lifelong learner, I blend my expertise with a genuine commitment to uplifting my colleagues, clients, and community. Each day, I begin with faith, exercise, and the conviction that today's challenges will become tomorrow's success stories.
Sign up to access the full article
Unlock more content to help you turn AI into leverage, design experiences that build trust, and drive business impact.
Have an account? Log In
I am currently the SVP of Customer Success at Billtrust, where I'm responsible for CSMs, Customer Value Delivery, and Renewals. I am responsible for all clients — Enterprise, Corporate, and SMB — globally, which is primarily the US and Europe.
We are a $200M+ SaaS company with around 800 employees. My team is modest but mighty, with about 25 team members.
How AI affects churn, support, and knowledge
We are accelerating the knowledge that we deliver to clients, including the relevance and accuracy of our knowledge articles. This is done via workflows and agents that automatically update these articles.
Thanks to AI, I've made big changes to how customer experience is delivered. Here are three examples:
1. Health scoring and churn prediction
We've aggressively embedded AI into our health scoring and churn prediction model, allowing us to identify and intervene with issues much faster than before.
We use a combination of tools. Claude is our front end, but we built a 12-signal model that taps into source data, sentiment data (Gong, CSAT, NPS), and key risk signals specific to our business. We started with three signals and calibrated the model as we retained, grew, or churned accounts. In Q1, it was 84% accurate.
We update the scoring every week. This allows us to intervene within days — rather than weeks or months — instead of only seeing signals after a conversation or an updated health score from a CSM or AE.
2. Customer support
We're just on the verge, but our agentic approach will extend our ability to serve customers who previously lacked CSM support and coverage. In fact, we've successfully moved from email-case submissions to an initial "chatbot" handling all non-phone interactions.
We use a combination of Claude and an internal tool we built that adds skills for collecting data and insights, and delivering visuals. The skills we create let us harmonize source data, usage data, and sentiment (from Gong, NPS, CSAT), incorporating all that data into a consistent but tailored set of engagement messages, presentations, etc.
We are also replacing our CSP. Together, these create scale for us.
It will take another 4-6 months, but we will increase our coverage from 500 accounts to nearly 2,000.
3. Knowledge bases
We are accelerating the knowledge that we deliver to clients, including the relevance and accuracy of our knowledge articles. This is done via workflows and agents that automatically update these articles.
What humans must still handle in CX
AI signals heavily drive escalation, support, and content, but we still heavily rely on humans to cover the "last mile" of issue resolution/service recovery.
And for high-leverage relationships, it's clear where AI stops being helpful. Only a human can relay the empathy needed to correct a spiraling relationship.
Why AI can't replace domain expertise
Domain expertise is still essential to understanding and addressing true customer pain.
Domain expertise allows a CSM, for example, to understand the day-to-day challenges of a persona, user, or executive. AI excels at creating solutions, but I've yet to see it replace the domain knowledge of someone who has walked in the client's shoes and understands how to create a connection when a customer is in a tough situation.
AI can replace many things, but it cannot replace the domain knowledge that creates a different level of problem-solving.
How AI increases both engagement and slop
I've seen two big benefits to AI: reduced human dependence and increased engagement with our accounts. Because AI drives much of our issue identification and intervention decisioning, we can work with 50% more accounts with up to 20% fewer people.
Of course, there are negative impacts as well. We have some customers who don't want to start with a chatbot. Long-standing customers are used to reaching out to the same person and don't want to change behavior. So, there's a balance there.
And in a few cases, people worked too quickly with AI and created some slop that reached clients.
How AI errors can impact relationships
AI is only as good as your ability to challenge the results. You have to be intellectually curious and iterate with AI to get to the best results over time. The first answer is almost never good, let alone good enough.
It's tempting when time is tight to take a few shortcuts. Maybe that client deliverable needs a little less review because you're sure AI got it.
But I've made that mistake enough times — with clients and internally — to know that AI often fills in blanks with information it doesn't possess. Incorrectly filling those blanks can undermine the credibility of everything else.
I had a former partner tee up AI to answer his lower-priority emails without intervention. But then the system routed one of my important emails to the wrong queue, and I received a clearly AI-driven response. I had no idea he was using AI for responses, and in that moment, he was embarrassed because it seemed he didn't value our long relationship.
It was a powerful learning moment for each of us.
Why integrating data sources is critical to CX
Data, data, data…Thoughtfully making (this) data available to AI creates the context it needs to deliver a better, more tailored, and client-specific experience.
Data, data, data.
If you don't have a strong data foundation — integrating data sources through a tool like Snowflake is fine — you won't get the benefits of AI.
The reason is simple: AI cannot deliver real benefits unless it is rooted in data that provides broad client context, combined with your product knowledge and how they create value.
It's almost inexcusable not to know your client's journey, given all the data we can collect now — recorded calls, emails, product signals, etc. You need a full view of every customer interaction, KPI, goal, challenge, etc.
Thoughtfully making this data available to AI creates the context it needs to deliver a better, more tailored, and client-specific experience.
Why CX leaders can benefit from creating new roles
We are creating a team of forward-deployed engineers (I call them Product Value Architects) who understand how to create interventions for each signal to close the value gap and get the client back on track. And CSMs work with those PVAs in context because they understand client goals and KPIs.
Without the signals and scalable interventions, we couldn't do it. AI is key to the signals and is embedded in many (not all) interventions.
Why CX leads must experiment and go first
Here's my advice: Experiment, go first, make decisions.
Don't rely on point solutions or AI embedded in point software. The power of AI will emerge across solutions and datasets.
Follow along
You can follow David Karp's work on LinkedIn.
More expert interviews to come on The CX Lead!
