What Happens When You Feed AI 21,000 Emails and Ask What to Do Next

At a Glance

What platform does Scrappy ABM use for email marketing?

ActiveCampaign, after moving through a couple of earlier platforms (starting with SendFox) that couldn’t support behavior-based segmentation at scale.

What results did the new email system produce?

A 52.66% open rate on the welcome sequence (vs. a 36% industry benchmark), a 42% average open rate across all campaigns, 3x email list growth in one year, and an unsubscribe rate under 1%, all across roughly 21,000 emails sent, without adding headcount.

How is AI used in this workflow?

Through ActiveCampaign’s Active Intelligence, which is used to analyze campaign performance and surface patterns. The team asks it specific, context-rich questions (naming the campaign, stating the goal, asking for patterns) rather than vague ones, which is what makes the output usable.

Does AI replace the need for good segmentation?

No, segmentation comes first. Scrappy ABM’s 650+ workshop registrants are sorted into five tiers based on behavior (sign-up, show-up, engagement afterward), and each tier gets different follow-up. AI helps interpret that data, but it doesn’t organize it.

Who runs the day-to-day mechanics of this system?

Evan Cox Consulting, as Scrappy ABM’s fractional marketing director since Fall 2024. Scrappy ABM sets the goals and owns the results, while ECC executes the tagging, tiering, and prompting behind the scenes.

Recently, ActiveCampaign approached our client, Scrappy ABM, and asked us to walk through how we use their platform to turn nearly 21,000 emails into a welcome sequence performing well above the industry benchmark.

That’s a big thing, as ActiveCampaign is a vendor Scrappy ABM uses rather than a partner we pitched this to. They noticed what Scrappy ABM had built and asked to feature it in front of their own list.

Mason Cosby (the founder of Scrappy ABM) and our founder, Evan Cox, said yes, and this case study is our walkthrough in writing: what wasn’t working with the old setup, how we changed their workflow, and what moved the numbers.

The Program Behind the Results

Before getting into the email side, it’s worth pointing out what this case study is about: Scrappy ABM’s marketing program.

Scrappy ABM specializes in running events and workshops for B2B marketers as a function of their go-to-market strategy. The email system described below is the layer that gets people to the right event, at the right time, based on what they’ve done. It supports the program rather than replacing it.

More on what Scrappy ABM does day-to-day here.

Scrappy ABM is founder-led and bootstrapped, a perfect fit for a fractional marketing director role, which we’ve been filling for them since Fall 2024. Our team covers strategy, website, and the webinar and email program this case study is built on.

That matters here because everything below is work we built and run on Scrappy ABM’s behalf. Scrappy ABM sets the goals and owns the results, and ECC is the team executing the mechanics day-to-day. 

(See our full client spotlight on Scrappy ABM for extra details on our partnership.)

Outgrowing the Platform

Scrappy ABM didn’t start with ActiveCampaign. The team started with SendFox, then moved through a couple of platforms before landing where they are now, and each move happened for the same reason: the tool they had stopped matching what they needed to do.

The lesson we took from that, and the one we’d pass on, is this: don’t pick the tool you need today. Instead, pick the one you’re going to need next year. Tool migration costs more than it looks like it will up front, both in time and lost momentum.

By the time Scrappy ABM outgrew its earlier setup, the gaps were clear:

  • The same email was going to every contact, regardless of what they’d done.
  • Follow-up was manual, which worked at a few hundred contacts and fell apart past that.
  • There was no visibility into who was engaged, who wasn’t, or why.

What the program needed was a platform that could segment contacts automatically based on behavior, send different follow-up content to different groups automatically, and support a welcome sequence personalized enough to feel like it was written for one person.

The Fix: Workflow + Segmentation

Once Scrappy ABM had a platform that could segment on behavior, the first thing the team did wasn’t automation but cleaning up how they grouped people in the first place. (This is where ECC comes in day-to-day. We run the tagging and tiering logic on Scrappy ABM’s behalf, but the calls on what qualifies as engaged are theirs.)

For a recent workshop, we sorted 650+ registrants into five tiers based on what they did: whether they signed up, whether they showed up, and how long they stayed engaged after. That ranged from highly qualified (people who showed up and stayed engaged) down to awareness-level contacts who’d registered but hadn’t done much else. Each tier got a different follow-up: a nurture sequence, a re-engagement push, or just a light touch, depending on where they landed.

This is the part that’s easy to skip past, but it matters most. Segmentation has to come before AI, since AI doesn’t fix messy data. If contacts aren’t grouped in a way that reflects behavior, no amount of analysis on top of that data is going to tell you anything useful.

Once the segmentation was in place, that’s when AI became useful, as it helped us understand what the data was telling us. That’s the next part.

The AI Layer: Active Intelligence

With segmentation in place, the next shift was in how we analyzed campaigns. We used to pull reports after the fact (look at what happened, write it up, move on). Now we use ActiveCampaign’s Active Intelligence to ask it directly: given this data, what should we do next?

The biggest factor in getting something useful back is all about how you prompt it. There’s a massive difference between a vague prompt and a specific one:

Vague prompt: “Analyze my email campaign.”

Result: generic output. Nothing you can act on the same day.

Context-rich prompt: “Here’s the tag for our November workshop. Goal: increase show-up rate for January. What patterns do you see in open rate by subject line type?”

Result: specific, actionable patterns tied to a real decision.

The difference comes down to three things:

  1. Name the campaign or tag you’re asking about, not just “my emails.”
  2. State the goal, aka what you’re trying to improve.
  3. Ask for a pattern rather than a summary. Summaries describe what happened, patterns tell you what to change.

We also use it to compare groups against each other, like one segment’s subject lines against another’s or one send time against another, which is harder to do by hand once you’re working across thousands of contacts.

None of this replaces judgment. It just means less time spent building the report and more time spent deciding what to do with it.

The Results

Here’s what the shift in workflow produced, across roughly 21,000 emails sent over the past year:

  • 52.66% open rate on our welcome sequence, compared to a 36% industry benchmark
  • 42% average open rate across all campaigns
  • 3x email list growth over the past year
  • Under 1% unsubscribe rate
  • All of this without adding headcount to manage it

We’re including the industry benchmark alongside our welcome sequence number because we think the comparison matters more than the raw figure. A 52% open rate on its own doesn’t mean much without knowing what normal looks like for this kind of email.

The unsubscribe rate is the number we’d point to if someone’s skeptical that segmentation and personalization can feel like a good experience rather than a controlled one. People aren’t leaving, which suggests the emails they’re getting match what they signed up for.

The Real Shift

The bigger shift for us was smaller and less obvious: we stopped writing reports about what happened and started asking questions about what to do next.

That sounds like a small distinction, but it changes how you spend your time. A report tells you what occurred. A question tells you what to try next. We used to spend hours pulling numbers together just to describe a campaign after the fact. Now that part takes minutes, which means the time we used to spend reporting, we now spend deciding.

That’s what Active Intelligence changed for us: not the analysis itself, but what the analysis is for.

From Mason

“This has been a partnership since Day 1. We set the strategy and goals, and Evan’s team helps us execute the day-to-day, like the segmentation and prompting. Neither side could get here alone. So when ActiveCampaign reached out wanting to highlight the results, it felt like recognition for what we built together.”

Our Best Advice

  1. Start with segmentation

Before you ask any tool to analyze your emails, make sure your contacts are grouped by behavior. Think who showed up, who engaged, and who didn’t.

  1. Pick the tool you’ll need next year

We moved platforms more than once because we outgrew what we had. Migrating costs more time and momentum than it looks like it will upfront, so it’s worth choosing with some room to grow.

  1. Give AI context before you ask it a question

Name the campaign or tag, state the goal, and ask for a pattern instead of a summary. That one habit is the difference between a generic response and something you can act on the same day.

Fortunately, you don’t need a bigger team or budget for any of this. What you do need is a better starting point, with clean segmentation, the right tool, and a habit of asking better questions.

Links Worth Your Time

Want to go deeper on any of this? Here’s where to find more:

Evan Cox

For more than fifteen years, I have served businesses, nonprofits, churches, and the leaders who guide them. Whether serving on staff at internationally-renowned nonprofits or running a coffee company in Southeast Asia, my work has taken me around the globe. Today, I work alongside organizations as a StoryBrand Certified Guide, so that they communicate the right message and then infuse it throughout all their marketing–from strategy and copy to websites and fundraising.

Create a strategic fundraising plan with ease.

Establish a plan to effectively raise more money for your nonprofit organization