Literature Review: LLMs and Inflation Forecasting

Given the recent hype, we review a paper at the frontier of LLMs and macroeconomic (inflation) forecasting. We also provide a brief summary of this literature and our assessment of it. The TL;DR message of this note is as follows: LLMs can be very useful (mainly by expanding the information set), but they generally fail to outperform standard benchmarks. A strong macro model is still needed.

ChatMacro: Evaluating Inflation Forecasts of Generative AI

Alam, Boyle, Li, Sekhposyan (January 27, 2026)

What the paper does

The paper (link) provides a systematic real-time evaluation of whether a general-purpose large language model (ChatGPT) can produce reliable forecasts of U.S. CPI inflation. The authors’ starting point is that much of the recent literature documents surprisingly strong macro-forecasting performance by LLMs, but typically under pseudo out-of-sample settings. In such exercises, the model is asked to “pretend” it is at a past date and to avoid using information that would not have been available at that time. The authors argue that for LLMs this approach is fundamentally fragile: it is difficult to verify whether the model is implicitly recalling future realizations embedded in its training data, and the internal representation of the data-generating process may already incorporate forward-looking information.

The core contribution of the paper is the construction of a fully real-time forecasting dataset. The authors query ChatGPT every hour between May 13, 2024 and June 30, 2025, asking for forecasts of year-over-year CPI inflation for the current month and the following twelve months. The prompt is standardized and explicitly instructs the model not to use information unavailable at the current date, and to report its information sources. In parallel, the authors build a pseudo out-of-sample dataset from January 2019 through April 2024 using the same prompt structure but executed retrospectively. Forecast performance is benchmarked against realized CPI (August 2025 vintage) and, crucially, against the Cleveland Fed’s daily inflation nowcast, which serves as a professional frontier benchmark for the current-month forecast.

Main results

The central finding is that the apparent forecasting ability of LLMs is substantially overstated when evaluated in pseudo out-of-sample exercises. When forecasts are collected in genuine real time, ChatGPT appears “stale,” insufficiently responsive to incoming data, and clearly inferior to professional benchmarks even in relatively simple tasks such as nowcasting CPI inflation.

In the pseudo out-of-sample period before April 2023, ChatGPT tracks inflation movements closely, including the large swings during 2021–22. After the presumed knowledge cutoff, however, forecast quality deteriorates markedly. This deterioration persists into the true out-of-sample period (May 2024 onward), even though inflation during that time is more stable than during the pandemic surge. Figure 2 reinforces this result using directional accuracy: the frequency of directional misses increases sharply for ChatGPT after April 2023, whereas Cleveland Fed forecasts do not display a similar break. This suggests that the structural break in ChatGPT’s performance is not driven by a change in the inflation process itself, but rather by the model’s information environment.

Figure 1. Main result of Alam et al. (2026) – Multi-horizon forecasts evaluation, pseudo out-of-sample vs. out-of-sample.

The quantitative results are stark. Cleveland Fed forecasts dominate ChatGPT forecasts across all subperiods. The relative mean squared error (ChatGPT divided by Cleveland Fed) rises from roughly 2.2 in the pre-cutoff pseudo out-of-sample period to about 12.8 in the post-cutoff pseudo out-of-sample period, and remains elevated at approximately 7.3 in the real-time out-of-sample window. A similar pattern holds for mean directional error, where relative losses jump from around 1.33 pre-cutoff to above 8 post-cutoff, and remain significantly elevated thereafter. While absolute error levels vary across periods due to changes in inflation volatility, the relative comparison against Cleveland Fed makes clear that ChatGPT’s earlier apparent competitiveness does not replicate in genuine real-time forecasting.

Our critical comment

The paper is carefully designed and addresses a central methodological issue in evaluating LLMs for macro forecasting. The collection of hourly real-time forecasts over more than a year is a major strength, as it eliminates ambiguity about look-ahead bias. The comparison between pre- and post-knowledge-cutoff periods convincingly demonstrates how sensitive apparent forecast performance can be to training data windows.

That said, some interpretative caution is warranted. The exercise evaluates a general-purpose LLM used out of the box and without live web access. This is appropriate for the authors’ scientific question but should not be interpreted as evidence that LLM-based systems cannot forecast inflation. Rather, it shows that a generic conversational model without structured time-series design or controlled retrieval is not a reliable quantitative forecaster in real time.

Also, the primary benchmark is a highly specialized nowcasting model. This is entirely appropriate for assessing quantitative forecast accuracy, but it narrows the scope of comparison. LLMs may have comparative advantages in narrative synthesis, scenario construction, or qualitative signal extraction from news, areas not directly assessed here (see our summary of the literature below). The authors do note that the cited “information sources” are loosely related to forecast values, raising concerns about interpretability.

A brief summary of the LLM/macro literature

LLM-based inflation forecasting is a very recent but fast-growing research frontier. The emerging literature broadly agrees on one key point: LLMs can add forecasting value, but mainly when used as text-based information extraction tools rather than as direct “forecasting engines.” In particular, LLMs appear most useful for incorporating unstructured information (news, central bank communication, narratives, surveys, and qualitative reports) into inflation prediction frameworks, especially at short horizons.

A recurring finding is that LLM-derived indicators (sentiment, topics, narratives, or embeddings) can improve inflation nowcasting and short-term forecasting performance relative to traditional models that rely only on structured macroeconomic data. This is especially relevant in environments where inflation dynamics shift quickly, because text sources often contain early signals of supply shocks, energy pressures, or policy changes. Studies show that these gains are often strongest during periods of instability, when standard Phillips-curve style relationships break down and expectations become more important (e.g., news-driven inflation narratives).

However, the literature is cautious: LLMs alone are not consistently superior to established econometric or machine learning benchmarks. Instead, the best-performing approaches tend to be hybrid: combining LLM-extracted features with time-series models (VARs, Bayesian models, random forests, boosting, or neural forecasting architectures). In this sense, the consensus is that LLMs are best viewed as a feature engineering technology that enhances the information set, rather than replacing forecasting models altogether.

Another important theme is interpretability. Many papers emphasize that LLMs can help identify which narratives or which shocks are being priced into inflation expectations (e.g., food vs. energy vs. wage pressures). This provides a meaningful advantage for policy analysis, even when pure forecast accuracy gains are moderate. At the same time, researchers highlight risks such as instability across model versions, sensitivity to prompting, and difficulties in replicability.

Overall, the frontier literature suggests that LLMs improve inflation forecasting mostly by converting qualitative text into quantitative macro signals, particularly for nowcasting and short horizons, while longer-term inflation forecasting remains dominated by structural forces and expectations formation that LLMs alone cannot reliably capture.

References

Alam, M. Jahangir, Shane Boyle, Huiyu Li, and Tatevik Sekhposyan. 2026. “ChatMacro: Evaluating Inflation Forecasts of Generative AI.” Federal Reserve Bank of San Francisco Working Paper 2026-04. https://doi.org/10.24148/wp2026-04

Carriero, Andrea, Davide Pettenuzzo, and Shubhranshu Shekhar. 2024. “Macroeconomic Forecasting with Large Language Models.” Preprint. https://doi.org/10.48550/arxiv.2407.00890

Fan, Yingying, Jinchi Lv, Ao Sun, and Yurou Wang. 2025. “LLM-Powered CPI Prediction Inference with Online Text Time Series.” arXiv:2506.09516. https://arxiv.org/abs/2506.09516

Zarifhonarvar, Ali. 2026. “Generating Inflation Expectations with Large Language Models.” Journal of Monetary Economics 157 (C). https://doi.org/10.1016/j.jmoneco.2025.103859

Anesti, Nikoleta, Edward Hill, and Andreas Joseph. 2025. “Inflation Attitudes of Large Language Models.” arXiv:2512.14306. https://arxiv.org/abs/2512.14306

Faria-e-Castro, Miguel, and Fernando Leibovici. 2023/2024. “Artificial Intelligence and Inflation Forecasts.” Federal Reserve Bank of St. Louis Working Paper 2023-015 (rev. 2024). https://doi.org/10.20955/wp.2023.015

Kwon, Byeungchun, Taejin Park, Phurichai Rungcharoenkitkul, and Frank Smets. 2025. “Parsing the Pulse: Decomposing Macroeconomic Sentiment with LLMs.” BIS Working Paper No. 1294. https://www.bis.org/publ/work1294.htm

Allard, Marc-Antoine, Paul Teiletche, and Adam Zinebi. 2024. “Enhancing Inflation Nowcasting with LLM: Sentiment Analysis on News.” (Preprint / Papers With Code). https://paperswithcode.com/paper/enhancing-inflation-nowcasting-with-llm

“Measuring Inflation Expectations Using Artificial Intelligence.” 2025. Computational Economics 66 (4). https://link.springer.com/article/10.1007/s10614-025-11231-5

Contact

Want something more tailored?

We provide tailored consulting on ad-hoc projects.