I recently spoke with Jen Towns, director of retention marketing at AOL, who has the intriguing job of retaining AOL’s two million paid subscribers (notably a very small portion of its 220 million users). Unsurprisingly, those customers who engage with products are least like to drop-off.

To put some metrics around retention, Towns uses Adobe’s marketing cloud to do A/B and multivariate tests to know which outreach yields the best outcomes. Since implementing testing last year, Towns had some curious instances in which data defied instinct.

For one, her division had a program that waived the outstanding balance for delinquent subscribers if they updated their method of payment. Going on pure instinct, this program seemed to be "a no brainer."

“And yet when we tested, we found a 10% decrease in those updating payments,” Towns said, adding the approach may have felt suspicious to consumers accustomed to a more direct payment method from AOL.

Another tested the call-to-action(CTA) button for AOL’s legal product Hyatt Legal Plans.

“Everyone went with their gut on this and just created a button that says ‘Call now,’” but the engagement was less than 1%. “So we tested a CTA button that said ‘Get Started’ and found more than 1000% increase in engagement.”

For the PR industry, the challenge is often client or executive buy-in, especially for organizations that haven’t traditionally incorporated testing into their services. For Towns, the key was capturing a measurable gain first.

“You don’t need to tell everyone what you’re doing - do it on top of your day job,” she says. “When I got my first win, I went for executive buy-in and was able to show them I could do this without a lot of extra resources.”

In an interesting defiance of popular convention, Towns eschews real-time optimization.

“Real-time is dangerous,” she says. “Often I’ll see a 200% lift right away and then within two weeks, it’s a 36% loss. Time will normalize data.”

So how do you get an entire organization into a testing mindset? Towns recalled an example of an organization (not AOL) that uses poker chips to bet on outcomes.

“Testing lets you take the highest paid opinions in the room and test them,” she adds.