LLMs Are Not (Consistently) Bayesian: Quantifying Internal (In)consistencies of LLMs’ Probabilistic Beliefs

New research quantifies how LLMs fail to update beliefs like Bayesian reasoners. The study introduces an information processing gap metric showing inconsistencies in how models revise probabilities from evidence. It matters for AI developers deploying LLMs in medicine, science, and law, where calibrated uncertainty is critical.

Opening Kapyn…