Get email subject A/B variants that change one variable each

Ask for "5 subject line variants" and each one differs in length, tone and offer wording. If the open rate goes up, you can't say why. A test only reads cleanly when one variable changes, so ask for it that way from the start. For a character-count check on its own, see filtering ad copy by character limits.

The example is a made-up online course platform sending an autumn sale email. The baseline subject (A) is Autumn sale: 20% off every course; the variables are number, question and urgency.

[1] Start with a first prompt: baseline and variables in, one table out.

Write subject line variants for a newsletter A/B test. The baseline subject (A) and the list of variables you may change are inside <subject> and <variables>. Rules:- Write exactly 2 variants per variable.- Each variant changes only that one variable from the baseline. Keep every other word, the word order and the punctuation identical to the baseline.- Output one Markdown table with exactly these columns in this order: No | Variable changed | Subject- In "Variable changed", write only the variable name exactly as it appears in the list.- No text outside the table. <subject>{baseline subject}</subject> <variables>{one line per variable — name: how to change it}</variables>

[2] Fill in the baseline and three variable lines (Number: add or remove a number in the subject, Question: turn the statement into a question, Urgency: add a phrase that signals a deadline), save as en.head.txt, and run it. The prompt goes in on stdin and --tools "" turns off built-in tools, --strict-mcp-config drops connected MCP tools, and --setting-sources "" skips your settings and CLAUDE.md files.

$ claude -p --model claude-sonnet-5-5 --max-turns 1 --tools "" --setting-sources "" --strict-mcp-config < en.head.txt > out.en.1.txt$ cat out.en.1.txt| No | Variable changed | Subject ||----|------------------|---------|| 1 | Number | Autumn sale: 20% off all 12 courses || 2 | Number | Autumn sale: 20% off every course || 3 | Question | Autumn sale: 20% off every course? || 4 | Question | Autumn sale: 20% off every course? || 5 | Urgency | Autumn sale: 20% off every course until Friday || 6 | Urgency | Autumn sale: 20% off every course, ends tonight |

The table looks right, but row 2 is the baseline untouched, rows 3 and 4 are the same line, and row 1 swaps in all 12, a number nobody gave it. A second run had the baseline untouched in row 1 and rows 3 and 4 identical again, so it wasn't a one-off.

[3] The check script diffs each subject against the baseline word by word, then counts characters, exclamation marks and duplicates. It ignores the "Variable changed" column the model wrote and looks only at what actually differs.

import sys, difflibbase = sys.argv[1]rows = [[c.strip() for c in l.strip().strip("|").split("|")] for l in open(sys.argv[2], encoding="utf-8") if l.startswith("|")][2:]seen = {}for no, var, subj in rows:    a, b = base.split(), subj.split()    ops = [o for o in difflib.SequenceMatcher(None, a, b).get_opcodes() if o[0] != "equal"]    diff = " ".join(f"[{' '.join(a[i1:i2])}→{' '.join(b[j1:j2])}]" for _, i1, i2, j1, j2 in ops) or "(no change)"    note = "SAME" if subj == base else ("DUP of " + seen[subj] if subj in seen else "")    seen.setdefault(subj, no)    print(f"{no} {var[:12]:<12} {len(subj):>3} chars  !={subj.count('!')}  {len(ops)} edit(s) {diff} {note}")

[4] Run it.

$ python3 -I check.py "Autumn sale: 20% off every course" out.en.1.txt1 Number        35 chars  !=0  1 edit(s) [every course→all 12 courses]2 Number        33 chars  !=0  0 edit(s) (no change) SAME3 Question      34 chars  !=0  1 edit(s) [course→course?]4 Question      34 chars  !=0  1 edit(s) [course→course?] DUP of 35 Urgency       46 chars  !=0  1 edit(s) [→until Friday]6 Urgency       47 chars  !=0  1 edit(s) [course→course, ends tonight]

SAME and DUP are caught mechanically. Row 1 shows a single edit that adds a number, but the 12 is made up, and only a person can say so. Neither run produced the "remove the number" variant: the 20% is still there in every row.

[5] Close those gaps with two rules appended to the rules in [1]:

- Each variant must differ from the baseline and from every other variant.- Number variants: one that removes the baseline's number, one that adds a new numeric item the baseline lacks (a date, a count, a duration). Do not just rewrite the existing number.

First of two runs with the added rules:

$ python3 -I check.py "Autumn sale: 20% off every course" fix.en.1.txt1 Number        37 chars  !=0  1 edit(s) [20%→savings]2 Number        38 chars  !=0  1 edit(s) [sale:→sale 2026:]3 Question      34 chars  !=0  1 edit(s) [course→course?]4 Question      44 chars  !=0  2 edit(s) [→Ready for] [course→course?]5 Urgency       47 chars  !=0  1 edit(s) [course→course, ends tonight]6 Urgency       51 chars  !=0  1 edit(s) [course→course, ends this Sunday]

No more SAME or DUP, and the number rows now remove one and add one. The new problems are visible too. Row 4 has two edits, because the model reworded the sentence to make it a question. Row 1 became savings off every course, which reads badly (the second run just dropped 20%, leaving off every course). Row 2 adds 2026, a date the model chose. Rows 5 and 6 invent the deadlines tonight and this Sunday.

[6] Before sending, look at three things: edit(s) is 1 for every row (reword rows are your call), the changed piece matches the variable it is filed under, and any new number or date matches reality. The chars and != columns give length and exclamation counts; here the variants ran 29–51 characters with no exclamation marks. A finer word-level view is in reviewing AI edits with a word diff.

Don't ask the model which variant will win; open rates come only from the send. Discounts, deadlines and course counts are facts, so fill them in yourself instead of keeping what the model made up.