Prompting Folklore, Checked Against What Papers Measured

Barakaeli Lawuo, Jan 21, 2026

In September 2023 a Google DeepMind team published a table of instructions for grade school math problems. The best one was not written by a person. It was "Take a deep breath and work on this problem step-by-step." Placed at the start of the answer, it got the pre-trained PaLM 2-L model to 80.2% accuracy on the GSM8K test set, with no worked examples in the prompt. The best human-written instruction the paper compared, "Let's think step by step.", scored 71.8%. With no instruction at all, the same model scored 34.0%.¹ Another language model, an instruction-tuned PaLM 2-L, had proposed the winning sentence during an iterative search in which every candidate was scored on a small slice of training problems.¹

GSM8K is a dataset of 8.5K grade school math word problems that take several steps to solve, built by OpenAI to measure multi-step reasoning.² The paper that ran the search calls its method OPRO, short for Optimization by PROmpting, and reports that its best prompts beat human-designed ones by up to 8% on GSM8K and by up to 50% on Big-Bench Hard tasks.¹

The deep-breath sentence sounds like prompting folklore: a motivational line that someone found once and everyone repeats. Here it is a measured number, with a model name, a benchmark and a baseline attached. That contrast is the point of this post. Most prompting advice arrives as a claim. A handful of papers took those claims and measured them. Below are four claims people repeat, each followed by what a paper actually found, including the places where the effect was small, flipped sign, or held only for one model.

Instruction: the plain-language part of a prompt that says what to do, as opposed to the question itself or any worked examples.
Zero-shot, one-shot, few-shot: how many solved examples (demonstrations) sit in the prompt before the real question: none, one, or several. No weights change in any of them.
Scorer model: in OPRO, the model whose accuracy is being improved. Each candidate instruction is run on it and graded.
Optimizer model: in OPRO, the model that reads past instructions and their scores and writes new candidates.
Execution accuracy: the share of test questions a model gets right when it follows a given instruction. APE and OPRO both rank instructions by it.

Redrawn from Table 4 of Yang et al., 2023.¹ The first four are human-written or baseline instructions; the last four were found by OPRO with different optimizer models. All use pre-trained PaLM 2-L as the scorer, with the instruction placed at the start of the answer.

Claim one: putting examples in the prompt helps

This is the oldest claim and the best supported one. The GPT-3 paper built its whole evaluation around it. For each task the authors ran the 175 billion parameter model three ways: few-shot, with as many demonstrations as fit in the context window (typically 10 to 100); one-shot, with a single demonstration; and zero-shot, with only a plain-language instruction.³ Across 42 benchmarks scored by accuracy, zero-shot performance rose steadily with model size, and few-shot performance rose faster, which the authors read as larger models being better at learning from context.³

The per-task tables are more interesting than the aggregate, because the gain from examples varies a lot. On WebQuestions, closed-book question answering went from 14.4% zero-shot to 41.5% few-shot. On three-digit addition it went from 34.2% to 80.4%. On TriviaQA, a task the model already did well, the rise was smaller: 64.3% to 71.2%.³ Then there are tasks where examples barely registered. On ARC Challenge, multiple-choice questions from grade school science exams filtered to defeat simple retrieval methods, the model scored 51.4% zero-shot and 51.5% few-shot. On the high school split of RACE, a set of English reading exams, 45.5% and 46.8%.³

Redrawn from Tables 3.2, 3.3, 3.6, 3.7 and 3.9 of Brown et al., 2020.³ Same model and same tasks, only the number of demonstrations changes. The LAMBADA zero-shot run used a different prompt format from the other two settings.

LAMBADA shows that one example can be worse than none. The task asks for the last word of a passage. A plain language model does not know the answer must be exactly one word, so the authors framed it as a fill-in-the-blank test and let the examples teach that format. Few-shot reached 86.4%, but one-shot fell to 72.5%, below the 76.2% zero-shot score. The paper notes that this format "is not effective one-shot, where it always performs worse than the zero-shot setting," and suggests the models need several examples to recognize the pattern.³ The same framing lowered the smallest model's score by almost 20% while raising GPT-3's by 10%.³ So the effect of examples depends on the model's size as well as the task.

Some tasks resisted examples entirely. On WiC, which asks whether a word means the same thing in two sentences, GPT-3 scored 49.4% few-shot, which the paper calls random chance. The authors tried a number of different phrasings and formulations, and none reached strong performance.³ The paper itself lists natural language inference and some reading comprehension sets as tasks where few-shot performance struggles even at GPT-3's scale.³

The authors are also careful about what the gain means. They write that it is unclear whether few-shot learning "actually learns new tasks 'from scratch' at inference time, or if it simply recognizes and identifies tasks that it has learned during training," and that the answer may vary from task to task.³ For a practitioner the reading is modest: examples reliably help when the output has a format the model cannot infer from the instruction, and help little when the model either already knows the task or cannot do it at all.

Claim two: small changes in wording matter

This claim is true, and in a less comfortable way than the folklore suggests. OPRO's authors describe it as a core difficulty of their search. With PaLM 2-L on the GSM8K test set, "Let's think step by step." scored 71.8% and "Let's solve the problem together." scored 60.5%. "Let's work together to solve this problem step by step.", which the paper calls the semantic combination of the two, scored only 49.4%.¹ Merging two decent instructions produced one worse than either. Swings like that made single instructions noisy and the search unstable, which is why OPRO generates eight candidates at every step rather than one.¹

Paraphrases can move scores upward too. On the Big-Bench Hard task ruin_names, which asks for the funniest pun on an artist or movie name, the search went from an empty instruction at 64.0% training accuracy to 72.0% at step 1, 80.0% at step 18 and 82.0% at step 38. The paper notes the later instructions were semantically similar and that a paraphrase still gave a notable accuracy improvement.¹ Big-Bench Hard is a set of 23 BIG-Bench tasks on which earlier language model evaluations had not beaten the average human rater.⁵

The part folklore skips is that the best wording depends on the model. OPRO ran the same human-written baselines on two scorers. On pre-trained PaLM 2-L, "Let's think step by step." beat APE's longer step-by-step phrase by 71.8% to 58.8%. On text-bison, an instruction-tuned PaLM 2 model, the order reversed: 65.6% for APE's phrase and 64.4% for the short one.¹ The gap from having any instruction also shrank. On text-bison the empty instruction already scored 56.8%, against 34.0% on the pre-trained model.¹ My reading of that table: wording matters a lot for a model that was never trained to follow instructions and less for one that was, and the ranking of two phrasings on one model says little about their ranking on another.

Two line plots of training accuracy against optimization step. Left, GSM8K: accuracy starts near 60, dips to about 48, then climbs with large step-to-step swings to around 75 by step 50 and stays there through step 150. Right, BBH movie_recommendation: accuracy starts near 70 and rises unevenly to above 90 by step 200. — Figure 1 from Yang et al., 2023,¹ reproduced under CC0 1.0. Each dot is the average training accuracy of up to eight instructions generated in one step; the shading is one standard deviation. The GSM8K run uses PaLM 2-L as scorer and PaLM 2-L-IT as optimizer.

Claim three: you can search for a better instruction instead of writing one

If wording matters and nobody can predict which wording wins, the natural move is to try many and measure. APE, the Automatic Prompt Engineer, did this in 2022. Given a handful of input and output pairs for a task, a language model proposes candidate instructions that could have produced those outputs. Each candidate is then run on the target model, scored by execution accuracy, and the best one is kept.⁴ The authors also tried an iterative version that asks the model for variations of the top candidates, and found that it gave only a marginal improvement, so they used the simple version by default.⁴

On 24 instruction induction tasks, with InstructGPT (text-davinci-002) following the instructions, APE's picks matched or beat the human-written instructions on all 24. Summarized as an interquartile mean, a mean that drops the top and bottom quarters of the scores, APE scored 0.810 against 0.749 for the human instructions.⁴ On a harder curated set of 21 BIG-Bench tasks, APE was comparable or better on 17, which means the human instruction won on the other 4.⁴ Putting APE's instruction in front of ordinary few-shot examples was comparable or better than the examples alone on 21 of 24 tasks; on Rhymes, Large Animal and Second Letters the combination did worse. The authors suspect instructions chosen for the zero-shot setting overfit to it.⁴

OPRO changes the search. Instead of rewording one candidate, the optimizer model sees a list of the 20 best instructions so far with their scores, in ascending order, plus three example problems, and is asked for a new instruction that scores higher.¹ For GSM8K the scores came from a fixed 3.5% sample of the 7,473 training problems, about 260 of them by my arithmetic, and the finished instructions were then graded on the full test set.¹ On Big-Bench Hard, the found instructions beat "Let's think step by step." by more than 5% on 19 of 23 tasks with the PaLM 2-L scorer and on 15 of 23 with text-bison.¹

Two details in those results are easy to miss. First, the deep-breath sentence is one of several near the top. Pre-trained PaLM 2-L, acting as its own optimizer, found "Break this down.", which scored 79.9%, only 0.3 points behind.¹ My reading is that the phrase itself is not special: the search found a family of instructions that work on this model, and many different sentences belong to it. Second, the search overfits. The authors report that training accuracies were often 5% to 20% higher than test accuracies, though test scores still mostly beat the human-written instructions.¹ The GSM8K instructions did carry over to two other math sets with the same scorer: the deep-breath instruction scored 95.3% on MultiArith and 54.3% on AQuA, against 85.7% and 44.9% for "Let's think step by step."¹

Claim four: being polite, or offering a tip, changes the answer

The paper most often cited for this is Bsharat, Myrzakhan and Shen's 2023 "Principled Instructions Are All You Need," which lists 26 prompting principles. Principle 1 says that if you prefer more concise answers there is no need to be polite, so drop phrases like "please" and "thank you." Principle 6 says to add "I'm going to tip $xxx for a better solution!"⁶ The abstract-level result is large: on GPT-4, the principled prompts improved response quality by an average of 57.7% and accuracy by 36.4%.⁶

How those numbers were produced matters. All evaluation ran on ATLAS, a benchmark the same authors built by hand. Each principle had 20 human-selected questions, asked with and without the principle, and each question got a single response from each model. Human evaluators judged whether the response improved and whether it was correct.⁶ The paper does not report repeated samples, confidence intervals, or how many evaluators there were.

Read per principle, the tipping and politeness effects look small and uneven. In the paper's heatmaps, the tip line has a quality boosting score of 45% on GPT-4 and 25% on LLaMA-2-13B, and a correctness improvement of 35% on GPT-4 and 5% on LLaMA-2-13B. Dropping politeness scores 5% for quality on GPT-4 and 0% for correctness.⁶ Every value in those heatmaps is a multiple of 5, which fits 20 questions per principle. If that is how the percentages were computed, the politeness result for GPT-4 and the tip's correctness gain on the 13B model each come down to a single question. That is my reading of the paper's setup; the paper does not state it that way.

The authors state the weaknesses themselves. They write that the assessment "was based on a limited selection of questions," that models with different architectures might respond differently, and that "the criteria and results may vary across various personnel assessments on the model responses."⁶ Compare that with OPRO, where every instruction was decoded at temperature 0 and scored automatically against known answers on public benchmarks.¹ The Bsharat paper is a useful list of things to try. As evidence that any single item works, it is thin, and I found no test in it that separates the effect of politeness or tipping from sampling noise.

A tuned prompt belongs to the model it was tuned on

OPRO's own tests of transfer changed the dataset and kept the scorer fixed, so they say nothing about moving a prompt to a different model.¹ Its conclusion lists other limits: the optimizer did not make good use of error cases, and the method needs a training set of at least tens of examples so the found prompt does not badly overfit.¹

APE tested the cross-model question directly, in an appendix. The authors took instructions that InstructGPT had generated and selected for itself, used them to steer the base GPT-3 model, and did the reverse. Both directions showed "a significant performance drop," and a human-written instruction reduced the loss.⁴ Their conclusion is that the model that scores instructions and the model that runs them need to match, and that "the instructions generated by InstructGPT work best for the InstructGPT itself but do not transfer well to a different model like GPT-3."⁴ GPT-3, which followed human instructions poorly, still generated prompts that steered it well, even though the authors describe those prompts as unintuitive.⁴

That is the limit on everything above. The measured gains were real, and each was measured on one model, one benchmark and one scoring rule. When the model changes, the found instruction has to be found again, on data from the task it will actually serve.