Abstract
Language models can be conditioned to answer as members of described groups, and research has found that such synthetic samples can reproduce some response patterns of real subpopulations. This suggests pre-launch testing of marketing messages before budget is spent. We review the evidence, contrast it with controlled online experiments, and describe GreenSignal AI, an iBOY product that scores campaign variants against 10,000 simulated users built from a persona. We argue that simulation is useful for ranking and filtering variants, and that real experiments remain the basis for decisions.
Keywordssynthetic audience; campaign testing; language model simulation; A/B testing; marketing experiments
1Introduction
Most campaigns learn after the budget is committed. Controlled experiments improve decisions, but only once traffic and money are already flowing. Language models raise a question: can some of that learning happen earlier?
2Evidence on synthetic respondents
Argyle and colleagues conditioned a language model on detailed demographic backstories and found that its responses could reproduce patterns in real survey data for those subgroups, a property they called algorithmic fidelity, while also noting limits and risks [1]. Controlled online experiments remain the established method for causal decisions about what works [2].
Chart 1
Uncertainty shrinks slowly with sample size
| users in the sample n | half-width |
|---|---|
| 100 | 3.34 |
| 133 | 2.90 |
| 178 | 2.51 |
| 237 | 2.17 |
| 316 | 1.88 |
| 422 | 1.63 |
| 562 | 1.41 |
| 750 | 1.22 |
| 1000 | 1.06 |
| 1334 | 0.915 |
| 1778 | 0.793 |
| 2371 | 0.687 |
| 3162 | 0.595 |
| 4217 | 0.515 |
| 5623 | 0.446 |
| 7499 | 0.386 |
| 10000 | 0.334 |
Source: Computed exactly: 95% half-width 1.96 · √(p(1 − p) / n) for a true rate p = 3%, in percentage points.
3System: GreenSignal AI
GreenSignal AI is an iBOY product. A marketer enters campaign copy, an audience persona, a platform and an objective. Each run is scored against 10,000 simulated users built from that persona, and the workspace returns predicted click-through, engagement and conversion for each variant, ranked side by side. The roadmap includes syncing real campaign results back to recalibrate predictions.
4Discussion and limits
Simulation is best used to compare and filter: catching an unclear message or ranking three headlines before any goes live. It cannot promise how a real market will respond in a given week, and its predictions inherit the biases of the underlying model. This paper does not report a validation of GreenSignal's predictions against live campaigns.
Chart 2
Clicks from 100 users when the true rate is 3%
| clicks k | P(k) |
|---|---|
| 0 | 0.048 |
| 1 | 0.147 |
| 2 | 0.225 |
| 3 | 0.227 |
| 4 | 0.171 |
| 5 | 0.101 |
| 6 | 0.050 |
| 7 | 0.021 |
| 8 | 0.007 |
Source: Computed exactly from the binomial distribution, P(k) = C(100, k) · 0.03ᵏ · 0.97¹⁰⁰⁻ᵏ.
import numpy as np
def simulate(true_rate=0.03, users=1000, runs=10000, seed=0):
rng = np.random.default_rng(seed)
clicks = rng.binomial(users, true_rate, size=runs)
p_hat = clicks / users
se = np.sqrt(p_hat * (1 - p_hat) / users)
covered = (p_hat - 1.96 * se <= true_rate) & (true_rate <= p_hat + 1.96 * se)
return p_hat.std(), covered.mean()
spread, coverage = simulate()
print(f"spread of estimates {spread:.4f}, interval coverage {coverage:.1%}")5Conclusion
Language model simulation can move part of campaign learning before launch. It complements, and does not replace, controlled experiments with real audiences.
Key findings
- 01Use simulation to rank and filter variants before spending.
- 02Use controlled experiments with real audiences to decide.
- 03Feed real results back to check and recalibrate predictions.
- 04Treat synthetic audiences as models of people, not people.
Questions and answers
- What is a synthetic audience?
- Simulated respondents generated from a described persona, used to estimate how a real audience with those traits might respond.
- Can AI replace A/B testing?
- No. Simulation can filter options early, but controlled experiments with real audiences remain the basis for decisions.
- Is GreenSignal AI available?
- It is an iBOY product currently open for early access through its waitlist.
References
- [1]Argyle, L. P., Busby, E. C., Fulda, N., and others (2023). Out of one, many: Using language models to simulate human samples. Political Analysis, 31(3), 337 to 351. doi.org/10.1017/pan.2023.2
- [2]Kohavi, R., Tang, D., and Xu, Y. (2020). Trustworthy Online Controlled Experiments: A Practical Guide to A/B Testing. Cambridge University Press. doi.org/10.1017/9781108653985
Cite this paper
Jaiganesh D (2026). Simulating audiences with language models before launch: promise and limits. iBOY Research Notes, Paper 09. https://www.iboyhub.com/blog/simulating-audiences-with-language-models-before-launch
Projects examined
Jaiganesh D
Founder of iBOY Innovation Hub, a creative technology studio from Tamil Nadu, India, established in 2025. Writes about what the studio learns while designing and building real websites, products and AI systems.