AI Designed Our Churn System. Here's Why I Didn't Use It.

Every membership business loses money two ways: failed payments, and members who cancel. We wanted real people reaching out to both, so we asked AI to build us a system. The same AI designed two very different versions. Here’s the video, and why I built the second one.

Two kinds of churn

Churn is a big focus in any membership business, and it comes in two kinds. Involuntary churn is a failed payment: if the card never gets updated, that member is gone. Voluntary churn is someone raising their hand and saying “I want to cancel.”

There are plenty of apps that plug into Stripe and promise to fix both. We found them pricey and not that effective.

One of the biggest parts of our membership is human connection. In an age of AI, we wanted to lean into that and have actual people reach out to members who are leaving. That can recover revenue, and it also tells us why people leave and what they value, so we can keep improving the business.

So we needed a way to work through both kinds of churn systematically, and we asked AI to build it.

The typical design, and why it’s brittle

I’m not ripping on anyone here. This is a typical design of what comes out when people bring a problem like this to AI.

The first design we got back had three Zapier Zaps, a Google Sheet, and a couple of Claude prompts to interpret the information. Our support person would work through the sheet line by line. For each one she’d double check Stripe, make sure the Zaps had fired and the information was current, then email the member.

It would technically work. As an operator, I’d call it brittle, for two reasons.

• Zaps fail. They fail all the time, and someone has to fix them or look the information up again in Stripe.
• There’s more than one source of truth. Slack holds the notifications, the sheet holds the status as of the last time the Zaps ran, and Stripe holds what’s actually true. Three places triangulating the same information is friction, and the information drifts.

The rebuild: Stripe as the database

I reimagined it the way I like to use AI, drawing on my experience as a software developer and a DevOps engineer. In my version, everything lives in Stripe. It’s the same rule I use for where operational information should live: in one place, and only one.

Under the hood, Stripe has customers. Customers have subscriptions, and subscriptions create invoices, which are what actually get charged. Something not many people use is that each of those objects can carry metadata. You can write information back to Stripe and it stays attached to that object.

So instead of three Zaps, a sheet and a set of prompts, I wrote one script that uses Stripe’s metadata as its storage.

Failed payments. When a payment fails for the third time, Stripe sends a webhook to the script. The script reads who the member is: how long they’ve been a member, their plan, what they pay each month, and why the card declined. It posts all of that to Slack for the support team, with the member’s email and a suggested message to send. The team member reaches out and marks the Slack message once they’ve worked it, so Slack doubles as their to-do list.

The script also writes back to Stripe that the failed payment is being worked, along with the Slack message ID and a status on the customer. Stripe becomes the database for payment activity. If the payment later goes through, the script finds the original Slack message and updates it to show the member was recovered. At the end of the month, I can run a report on how many we actually recovered.

Cancellations. These work a little differently, because there are far more of them and I didn’t want to bury the Slack channel. When someone cancels, Stripe sends a webhook and the script writes to the metadata again. Every day at 8 am, one Slack message goes to the support team with everyone who canceled the day before: name, email, plan, and how long they were a member. That gives the team a separate queue for voluntary churn.

Same AI, different operator

The system has been running for about a week, so there are no recovery numbers yet. I’ll share them once the first month is in.

The goal is the same one we started with: real people reaching out to members who are leaving, and a clear record of what we recovered and why people left.

The same AI designed both versions. The difference was knowing where the truth should live. AI amplifies what’s already in order, which is why the systems come first. Same AI, different operator.