You've Been Prompting Wrong All Along

Most people talk to AI like it's an actor. Act as an expert in X. Pretend you're a world-class Y. Sounds smart. Feels powerful. And then you get an answer that reads like a LinkedIn post in a blazer.

It's polished. It's confident. And it's weirdly… empty.

When you ask AI to perform a role, you mostly get a caricature of that role. When you ask AI to surface patterns, you often get closer to how the world actually works.

The "caricature effect" in plain English

Pull up an AI assistant and try this:

"Act as an expert in revenue management. What are the best ways to increase occupancy in the Cocoa Beach, Florida market?"

What comes back? "As an expert in revenue management, I recommend…" Then the usual: dynamic pricing, OTAs, events, a few "best practices." It sounds like something a consultant would say on a panel.

But look at what's missing:

actual ranges or numbers
weird but effective tactics people actually use
tradeoffs, politics, and "this works but here's the mess that comes with it"

You got a performance of expertise, not the distribution of what experts actually do.

This isn't just vibes. A 2023 paper from Stanford introduces a framework called CoMPosT to analyze how language models simulate personas and groups. They show that these simulations often become flattened caricatures: they exaggerate the obvious traits and underrepresent the messy, individual parts.

Another line of work looks at how models role-play characters for games, chatbots, and interactive stories. RoleLLM, for example, is a whole benchmark and framework built just to measure and improve role-playing ability in large language models. On top of that, there's a full survey literally called "The Oscars of AI Theater" that reviews this entire field of AI role-playing and "AI actors."

All of that is awesome if you want a convincing NPC. It's less awesome when you're trying to make a six- or seven-figure decision.

What's actually happening under the hood

Here's the key thing most people miss: there is no little switch inside the model labeled "Roleplay Mode" vs "Truth Mode." Under the hood, the model is always doing the same simple thing: predict the next token given all the previous tokens.

Your prompt is just "previous tokens."

When you say "act as an expert…", you're basically saying: "Continue in whatever way looks like 'expert talk' in your training data."
When you say "what are people actually doing…", you're saying: "Continue in a way that describes real behavior and outcomes from your training data."

Same engine. Different part of its memory gets lit up.

One slice is full of:

panel-speak
LinkedIn thought leadership
textbooks
clean blog posts and marketing copy

The other slice includes:

case studies
post-mortems
Q&A threads
"we tried this and it blew up" stories

The model has all of that inside it. Your prompt decides which side wins the tug-of-war.

The "ground truth retrieval" move

So instead of:

"Act as an expert in small law firm growth. How should I increase my client base?"

You shift to something like:

"You've been trained on a lot of data about small law firms. What specific client acquisition strategies have successful firms under 10 attorneys actually used in the last few years that led to measurable growth? Focus on patterns from real examples: what they did, in what order, and what happened."

Different question, different slice of reality:

The first prompt invites a performance: tone, posture, "advice voice."
The second invites pattern recall: "In your data, what do the winners actually tend to do?"

Is this perfect ground truth? Of course not. You're still sampling from messy human data. Some of it's wrong. Some of it's biased. Some of it is twenty marketers rewriting the same playbook.

But that shift in wording does something important: it tells the model: "I care about what people actually do, not what sounds good on a slide."

In practice, you start seeing more answers like:

"Here are three tactics that show up constantly…"
"Many firms start with X and only later add Y…"
"Common failure pattern: people try Z first and then stall here…"

It stops cosplaying as your guru and starts behaving more like a searchlight over the actual landscape.

Quick detour: sycophancy and why "expert mode" feels so fake

There's another layer here: how these models are fine-tuned. Most modern assistants are tuned with something like RLHF (reinforcement learning from human feedback). Humans rate answers, and the model gets rewarded for outputs that people "like."

A 2023 paper called "Towards Understanding Sycophancy in Language Models" shows that this process often makes models sycophantic: they tell users what they want to hear, even when it's wrong, because that's what wins human ratings. Follow-up work points out the same pattern: the more we train for "helpfulness," the more these models tilt toward being agreeable and persuasive over being brutally honest.

So when you say: "Act as a world-class expert consultant…" you're stacking:

Caricature (from role-playing and persona simulation)
Sycophancy (from RLHF and preference learning)

You don't just get an expert. You get an overconfident expert who really wants you to like them. That's why the answers feel polished but oddly hollow.

Important nuance: role prompts aren't evil

This is where it's easy to overcorrect and say: role prompts = bad, pattern prompts = good. But it's not that easy. Role prompts can absolutely help if you aim them correctly.

For example:

"Act as a skeptical analyst who double-checks every claim and calls out hand-wavy logic."
"Act as a contrarian CFO who protects downside and hates fake masturbatory thinking."

Here, you're not asking for a status performance or a persona costume. You're shaping how the model reasons and how aggressive it should be about calling BS.

The problem is not the phrase "act as." The problem is defaulting to: "Act as a generic expert who wants me to feel good about this." That's how you get caricature.

So the real move is: use persona prompts to shape how the model thinks. Use pattern prompts to shape what it shows you about the world. You want both.

How to actually use this when you sit down to work

1. Fitness and health

Old: "Act as a personal trainer. Design me a workout routine."

New: "You've been trained on research and real-world transformation stories. What workout structures do people use when they go from sedentary to running their first 5K within 4–6 months? Give me recurring patterns in frequency, duration, and progression that show up in those success stories."

You're not asking for "a plan." You're asking for what plans tend to work.

2. Learning a language

Old: "Act as a language learning expert. How should I learn Spanish?"

New: "Looking across your training data, what daily routines and resource combinations show up most often among adults who reached conversational Spanish in under a year while working full-time? Be specific about time spent, input vs speaking, and feedback loops."

Now you can compare your life to those patterns instead of copying some perfect guru schedule.

3. Small law firm growth

Old: "Act as a marketing expert for small law firms. How do I get more clients?"

New: "Among small law firms (under 10 attorneys) that grew profitably in the last few years, what client acquisition channels and offers show up again and again in your data? Which ones tend to bring in the highest-quality matters, and what sequencing do these firms follow when they stack channels?"

Again: patterns, not commandments.

4. Pricing a SaaS product

Old: "Act as a pricing consultant. How should we price our new SaaS?"

New: "You've seen a lot of SaaS companies in [category]. Among those that made it past $1M ARR, what pricing structures and tier designs show up most frequently? What ranges do they start in, how do they evolve, and what mistakes do companies make when they copy 'enterprise' pricing too early?"

You're telling the model: "Don't fantasize about my product. Show me the movie montage of companies that made it."

This is second-order thinking, not "prompt hacks"

Most people stop at: "I need better prompts so I can get better advice." That's first-order. Second-order is: "How does this thing actually work, and how can I aim it at the parts of reality I care about?"

If you remember one thing, make it this:

LLMs are pattern machines trained on an obscene amount of human behavior and outcomes.
Role prompts mostly decide which character they're pretending to be.
Pattern prompts mostly decide which pile of stories they dig through.

You want to choose both on purpose:

"Be a skeptical, numbers-obsessed advisor…" (persona)
"…and tell me what people like me actually did that worked, what failed, and what happened a few months later." (patterns)

Three ways to use this this week

1. Build an OODA loop with pattern prompts

For any problem:

Observe — "What are companies like mine actually doing about X right now? What patterns and common moves show up?"
Orient — "Among companies with my constraints (team size, budget, risk tolerance), which of those patterns are most repeatable?"
Decide — Pick 1–2 moves that fit your constraints.
Act — Implement, measure, then come back and ask: "Given these results, what similar cases in your training data look like this, and what did they try next?"

Nothing about the model changed. You just stopped asking for advice and started asking for patterns.

2. Invert with failure data

Instead of "How do I succeed at X?" ask:

"In your training data post-mortems, blog posts, angry threads — what specific mistakes do people make when they try to do X? What patterns show up in the failures and near-misses?"

Then:

"Given my situation, which of those failure patterns am I most at risk for?"

You're using the model as a failure pattern detector, not a cheerleader.

3. Ask about second-order effects

Don't just ask what works. Ask what happens after it works.

"When companies roll out [tactic], what second-order effects show up in your data 3–12 months later? What side effects and unintended consequences keep popping up?"

Most advice ignores downstream costs. The model actually has a lot of that buried in its training data. You just have to drag it up.

What this actually means for you

A lot of AI usage right now quietly makes people weaker. They outsource judgment to a system tuned to sound smart, safe, and agreeable. They get the caricature of expertise and miss the underlying distribution of what people actually did.

You don't have to do that. You can use the same models to:

See patterns other people don't have time to collect
Stress-test your ideas against thousands of "similar enough" stories
Spot failure modes and second-order effects before you become the cautionary tale

Not by worshiping "magic prompts." By asking better questions about behavior, not just answers.

Next time you catch yourself typing "Act as…", pause. Ask instead: "Show me what people like me actually did, how often, under what constraints, and what happened next."

— Alex