The technological frontier has unquestionably been going through a period of accelerated transformation since large language models (LLMs) became widely available. Artificial intelligence, as an aspiration, is nothing new in human history. From Warren McCulloch and Walter Pitts’s first model of synthetic neurons to depictions in popular culture such as "The Terminator", the idea that synthetic creations might reproduce human reasoning has stirred the popular imagination, at times fascinated by creativity, at times paralyzed by fear (after all, who would not fear the self-destructive capacity of boundless human inventiveness? Remember the Manhattan Project). It would therefore be almost a formality to challenge contemporary skepticism about the unregulated use of a technology with such vast potential. Before any apocalyptic reflection, however, a more immediate step is needed: understanding what this technology actually is. That is precisely the subject this essay sets out to explore.
The rise of artificial intelligence as both an object of study and a working tool has created growing demand for literacy in its methods. In response, courses and tutorials have multiplied that teach people to operate LLMs and train neural networks, often without requiring students to understand the underlying mechanics, the famous "under the hood". Although accessible, this utilitarian approach does not meet the need for structural understanding, in which the central interest lies in how models work and learn, and not only in their "outputs". From this more rigorous perspective, the real tools of analysis are found in mathematics and statistics textbooks, and not only in programming and writing interfaces.
What happens under the hood
The way a large language model works illustrates this relationship clearly. When it receives text as input, the model converts it into numerical vector representations and processes them through successive layers that combine linear and nonlinear transformations (Vaswani et al., 2017). Nonlinearity is precisely what gives the model its expressive power: a sequence of purely linear layers would be mathematically reducible to a single linear transformation, leaving the architecture trivially incapable of approximating complex functions. The main mechanisms behind this property are multi-head self-attention, which operates through inner products among query, key, and value matrices (Q, K, and V) followed by a softmax function (making processing dynamic and sensitive to the content of the input); nonlinear activation functions (such as ReLU or GELU) in the feedforward sublayers; and layer normalization. Structurally, this set of operations resembles complex regression models whose coefficients (weights) are estimated by minimizing a loss function that, from a probabilistic standpoint, corresponds to the negative log-likelihood of the training data (Goodfellow; Bengio; Courville, 2016). The goal of this training is identical to that of maximum-likelihood estimation: finding the parameters that make the observed data most probable. At the end of the computation, the model produces a probability distribution over its entire known vocabulary, from which the next token (a word or fragment) is sampled iteratively. Understanding this process (distinguishing estimation from inference, mastering the bias-variance trade-off, interpreting the loss function, and questioning the calibration of probabilities) requires the conceptual framework of applied statistics (Hastie; Tibshirani; Friedman, 2009). Without that foundation, a user may be able to command artificial intelligence superficially, but will be unable to understand what it optimizes, diagnose spurious correlations, or assess its real limitations.
In this sense, mastering statistics is the necessary condition (although I, José, still have my doubts about whether it is sufficient) for AI to stop being treated as an abstract entity and to be understood instead as a scientific instrument. And if statistics is the foundation of artificial intelligence, we should ask what the foundation of statistics is. In other words, what, after all, is the "probability" on which this whole computational edifice rests? This is precisely the ontological dilemma that David Spiegelhalter (2024), one of the most influential statisticians working today, takes on in his article "Why Probability Probably Doesn't Exist (but It's Useful to Act Like It Does)", published in Nature in December 2024.
References
GOODFELLOW, I.; BENGIO, Y.; COURVILLE, A. Deep learning. Cambridge: MIT Press, 2016.
HASTIE, T.; TIBSHIRANI, R.; FRIEDMAN, J. The elements of statistical learning: data mining, inference, and prediction. 2nd ed. New York: Springer, 2009.
McCULLOCH, W. S.; PITTS, W. A logical calculus of the ideas immanent in nervous activity. The Bulletin of Mathematical Biophysics, [S. l.], v. 5, n. 4, p. 115-133, 1943.
SPIEGELHALTER, D. Why probability probably doesn't exist (but it is useful to act like it does). Nature, London, v. 636, n. 8043, p. 560-563, Dec. 2024.
VASWANI, A. et al. Attention is all you need. In: ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS, 30., 2017, Long Beach. Proceedings [...]. [S. l.]: Curran Associates, 2017. p. 5998-6008.