Why do large language models hallucinate?

They are trained to produce plausible text, not verified facts, so when the right information is missing they can generate fluent but unfounded statements.

Does retrieval-augmented generation stop hallucinations?

It reduces them by giving the model relevant documents at answer time, but the model can still misread or go beyond the sources, so checks and review remain necessary.

What are scaling laws?

Empirical findings that model loss improves predictably as parameters, training data and compute increase, with later work showing data and model size should grow together.

Large language models: architecture, scaling and failure modes in production

1Introduction

Large language models moved from research to everyday tools in a few years. They now draft, summarise, classify and converse well enough to sit inside real products. They also fail in ways that are easy to miss, because their mistakes are written in the same confident prose as their correct answers.

This paper reviews how these models are built, what scaling research tells us, how they fail, and which system design patterns contain those failures in production.

2Architecture and pre-training

Modern language models are built on the transformer, which uses self-attention so that each token can draw on every other token in its context [1]. Pre-training teaches the model to predict text over very large corpora. Bidirectional pre-training proved the value of learning general language representations before task-specific tuning [2], and autoregressive models trained at much larger scale showed that one model could perform many tasks from a few examples placed in its prompt [3].

Attention(Q, K, V) = softmax(Q Kᵀ / √dₖ) V(1)

Scaled dot-product attention, the core operation of the transformer.

attention.py

python

import numpy as np

def softmax(x, axis=-1):
    x = x - x.max(axis=axis, keepdims=True)   # numerical stability
    e = np.exp(x)
    return e / e.sum(axis=axis, keepdims=True)

def attention(Q, K, V, causal=False):
    d_k = Q.shape[-1]
    scores = Q @ K.T / np.sqrt(d_k)
    if causal:  # a token may only attend to itself and earlier tokens
        mask = np.tril(np.ones(scores.shape, dtype=bool))
        scores = np.where(mask, scores, -1e9)
    weights = softmax(scores)
    return weights @ V, weights

rng = np.random.default_rng(0)
Q, K, V = (rng.normal(size=(5, 16)) for _ in range(3))
out, w = attention(Q, K, V, causal=True)
print(out.shape, np.round(w[2], 2))  # (5, 16), row 2 attends to tokens 0 to 2 only

Listing 1. Scaled dot-product attention with an optional causal mask, in NumPy.

3What scaling research shows

Kaplan and colleagues reported that language model loss falls smoothly as a power law in the number of parameters, the size of the dataset and the compute used for training [4]. Hoffmann and colleagues later found that, for a fixed compute budget, many large models had been trained on too little data, and that parameters and training tokens should be scaled in roughly equal proportion [5].

For product teams the lesson is indirect but useful: capability is bought with data and compute at a scale few organisations can match, so value is created in how a model is applied, grounded and evaluated.

L(N) = (N꜀ / N)^αN, C ≈ 6 · N · D(2)

Loss falls as a power law in parameters N; training compute is approximately six times parameters times training tokens D.

4Following instructions and reasoning

A pre-trained model predicts text; it does not necessarily do what a user asks. Fine-tuning with human feedback, in which people rank model outputs and the model is optimised towards preferred answers, substantially improved how well models follow instructions [6]. Prompting a model to write out intermediate steps before its answer improved performance on multi-step reasoning tasks [7].

5Decoding: from probabilities to text

A language model outputs a score, a logit, for every token in its vocabulary. Turning those scores into text is a separate choice. Greedy decoding always takes the most likely token; sampling draws from the distribution, and a temperature parameter controls how spread out that distribution is.

Lower temperatures make output more predictable and higher temperatures more varied. For tasks that must stay close to sources, such as summarising records, iBOY uses low temperatures and still relies on the checks described below.

p(tokenᵢ) = exp(zᵢ / T) / Σⱼ exp(zⱼ / T)(3)

Softmax with temperature T over the logits z.

sampling.py

python

import numpy as np

def softmax(z):
    z = z - z.max()
    e = np.exp(z)
    return e / e.sum()

def next_token(logits, temperature=0.0, rng=np.random.default_rng()):
    if temperature == 0:
        return int(np.argmax(logits))          # greedy
    probs = softmax(np.asarray(logits) / temperature)
    return int(rng.choice(len(probs), p=probs))

logits = np.array([2.0, 1.0, 0.5, -1.0])
print(next_token(logits))                          # 0, always
print([next_token(logits, 1.2) for _ in range(8)]) # varied

Listing 2. Greedy decoding and temperature sampling over a vector of logits.

6Failure modes: hallucination

A survey of natural language generation describes hallucination as generated content that is nonsensical or unfaithful to the provided source [8]. Because the model optimises for plausible text, it can state invented facts, figures or citations fluently. Retrieval-augmented generation reduces this by supplying relevant documents at answer time so the model can ground its output in them [9], but it does not remove the risk.

7Production patterns at iBOY

In our own systems we combine four patterns. First, grounding: agents receive verified records, such as a project's case study, and are instructed to use only facts present in them. Second, structured output: models return JSON with named fields, which is validated before use. Third, rule-based checks after generation: an editor that costs nothing to run blocks drafts containing figures absent from the source, identity misstatements or forbidden punctuation. Fourth, approval: anything that leaves the studio, such as an email or a published post, waits for a person or for a rule the Founder has explicitly switched on.

The research journal you are reading applies the same idea to citations. Articles may cite only works held in a fixed reference library with stable links, so a model drafting a paper cannot invent a source.

groundingCheck.ts

typescript

/** Figures such as 95%, 3x, 10k or 500+ that appear in a draft but not in its source. */
export function unsupportedFigures(draft: string, source: string): string[] {
  const figures = draft.match(/\b\d+(?:[.,]\d+)?\s?(?:%|x\b|k\b|\+)/gi) ?? [];
  const known = source.toLowerCase();
  return [...new Set(figures)].filter((figure) => !known.includes(figure.toLowerCase()));
}

const issues = unsupportedFigures('Accuracy improved by 95%.', 'The model was evaluated on test cases.');
console.log(issues); // ['95%'] so the draft waits for review

Listing 3. A rule-based check after generation: any figure missing from the source blocks the draft.

Algorithm 1

Grounded generation with approval

Input:: A question and a store of verified records
Output:: An answer that is either released or held for a person

1Retrieve the records most relevant to the question.
2Prompt the model with those records and an instruction to use only their facts.
3Require structured output and validate it against a schema.
4Run rule-based checks for unknown figures, names and citations.
5If any check fails, hold the answer for review; otherwise apply the release rule.
6Log the answer, the sources and the decision.

8Limitations

The patterns above reduce, but do not eliminate, errors. Grounding fails if the source itself is wrong, rule-based checks catch only what they are written to catch, and human review is only as good as the reviewer's attention. This paper describes design practice and does not report a measured error rate.

9Conclusion

Large language models are transformers pre-trained at a scale that makes capability largely a function of data and compute. Their characteristic failure is fluent, ungrounded output. Dependable products treat the model as one component inside a system of grounding, validation, checks and approval.

Key findings

01LLMs are transformers pre-trained to predict text, then tuned to follow instructions.
02Capability scales with parameters, data and compute together.
03Hallucination is a structural risk, not an occasional bug.
04Ground, structure, check and approve: the model is one part of the system.

Questions and answers

Why do large language models hallucinate?: They are trained to produce plausible text, not verified facts, so when the right information is missing they can generate fluent but unfounded statements.
Does retrieval-augmented generation stop hallucinations?: It reduces them by giving the model relevant documents at answer time, but the model can still misread or go beyond the sources, so checks and review remain necessary.
What are scaling laws?: Empirical findings that model loss improves predictably as parameters, training data and compute increase, with later work showing data and model size should grow together.

References

[1]Vaswani, A., Shazeer, N., Parmar, N., and others (2017). Attention is all you need. Advances in Neural Information Processing Systems 30. arxiv.org/abs/1706.03762
[2]Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. NAACL-HLT 2019. arxiv.org/abs/1810.04805
[3]Brown, T. B., Mann, B., Ryder, N., and others (2020). Language models are few-shot learners. Advances in Neural Information Processing Systems 33. arxiv.org/abs/2005.14165
[4]Kaplan, J., McCandlish, S., Henighan, T., and others (2020). Scaling laws for neural language models. arXiv:2001.08361. arxiv.org/abs/2001.08361
[5]Hoffmann, J., Borgeaud, S., Mensch, A., and others (2022). Training compute-optimal large language models. arXiv:2203.15556. arxiv.org/abs/2203.15556
[6]Ouyang, L., Wu, J., Jiang, X., and others (2022). Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems 35. arxiv.org/abs/2203.02155
[7]Wei, J., Wang, X., Schuurmans, D., and others (2022). Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems 35. arxiv.org/abs/2201.11903
[8]Ji, Z., Lee, N., Frieske, R., and others (2023). Survey of hallucination in natural language generation. ACM Computing Surveys, 55(12). doi.org/10.1145/3571730
[9]Lewis, P., Perez, E., Piktus, A., and others (2020). Retrieval-augmented generation for knowledge-intensive NLP tasks. Advances in Neural Information Processing Systems 33. arxiv.org/abs/2005.11401

Cite this paper

Jaiganesh D (2026). Large language models: architecture, scaling and failure modes in production. iBOY Research Notes, Paper 03. https://www.iboyhub.com/blog/large-language-models-architecture-scaling-and-failure-modes

Projects examined

Jaiganesh D

Founder of iBOY Innovation Hub, a creative technology studio from Tamil Nadu, India, established in 2025. Writes about what the studio learns while designing and building real websites, products and AI systems.