AI Copywriting Tools: What Actually Works and What Fails

We've burned through $4,200 testing AI copywriting tools across 12 client accounts since January. Some tools genuinely moved conversions up 18-34%. Others produced garbage that wasted 3 hours of editing per piece. Here's what we learned: AI copywriting isn't a yes-or-no question. It's *which tool, for which job, in which workflow*.

What Works: AI as Your Second Draft, Not Your First

The tools that deliver ROI aren't the ones that "write your ad copy from scratch." They're the ones that accelerate the refinement loop. We've had success with Claude (via API) and ChatGPT-4o when we feed them three inputs: a winning email subject line from the past 90 days, the target persona details, and the specific conversion metric we're optimizing for.

For example, a client selling landscaping services used ChatGPT-4o to generate 40 subject line variations around their existing winner ("[City] lawn care without the guesswork"). We tested 8 of them. Two outperformed the original by 22% and 19%. The AI didn't invent the concept—it extrapolated from the pattern. That's where AI excels. Copysmith and Jasper failed at this same task because they tried to be too "creative" and abandoned the working formula entirely.

• Claude API: Best for subject lines and email body copy when you give it context (wins 65% of the time)
• ChatGPT-4o: Strong for ad copy variations; weaker on long-form sales pages (works 58% of the time)
• Conversion.ai (formerly Copy.ai): Decent for landing page headers; tends to oversell (wins 42% of the time)
• Jasper: Flashy UI, mediocre output; marketed heavily to non-marketers (wins 31% of the time)

Where AI Copywriting Fails (Don't Waste Your Money Here)

Long-form sales pages. We tested AI-written 1,500+ word sales pages for three courses and a B2B software product. All three underperformed control pages written by a human copywriter by 24-41%. The AI rambled, lacked specific proof points, and couldn't maintain narrative tension across 1,500 words. This isn't a 2025 problem—it's structural. LLMs are pattern-matching machines, not storytellers.

Brand voice. If your brand voice is distinctive, AI will dilute it. We had a client with a blunt, irreverent tone (think Dollar Shave Club). Every AI tool we tested genericized it into corporate blandness within two sentences. Needed a human to inject the personality back. If your brand voice is indistinct already, AI won't hurt—but it won't help either.

AI copywriting is best as a production accelerator for proven formats—not as a creative replacement. Use it to generate 40 email subject line tests. Don't use it to write your brand manifesto.

The Setup That Actually Works

Here's the workflow we use now with clients who see real lift: Human writes the first draft or identifies the winning angle. AI generates 20-40 variations on that angle. Human picks the 3-5 strongest and A/B tests them. AI is the *production layer*, not the creative layer. When we tried to flip that (AI writes first, human edits), output quality dropped 40% and time investment skyrocketed.

Timing matters too. AI tools are worse at copywriting in isolation—better when they're responding to a live conversion data point. Example: Your email open rate just dropped 12%. Feed that insight + your top 20 historical subject lines into Claude, ask it why they worked, and request 30 new variations anchored to those patterns. You'll get usable copy. Ask it to "write a subject line that will perform well" with no context, and you'll get mediocrity.

← Back to all articles