This article explores how the research of John J. Hopfield and Geoffrey E. Hinton, which combined principles of statistical physics with neural networks, led to groundbreaking developments in artificial intelligence and earned them the 2024 Nobel Prize in Physics. The work of these pioneers in machine learning and neural networks has had a significant impact on fields like computational materials science and generative AI.
Connecting Physics and CS streams.
John J. Hopfield and Geoffrey E. Hinton were awarded the 2024 Nobel Prize in Physics — They proposed a sophisticated method of mixing concepts from statistical mechanics and neural networks at a time when hybrid solutions equated to magic sorcery. There is a clear case of the irony here, it feels like cursing oneself when something unexpected happens and in this particular case, it reflects what physics could bring to AI.
I was especially enthralled about the fact that a Nobel Prize was given for this theme that, as computational materials scientist, falls well into my field of interest. Hopfield and Hinton’s contribution has helped me in studying new horizons in materials chemistry, specifically on generative learning for materials. Through what they have taught us, we have been able to exploit the many-body nature of particles (as described by statistical mechanics) in solving complex computational problems.
Neural Networks 088 Neural networks and statistical mechanics.
Hopfield and Hinton’s work is built on principles of statistical mechanics (the branch of physics that uses statistical methods to study the collective behavior of systems with a large number of particles). In this area of study, scientists are often more interested in the larger scale properties of a system such as temperature, pressure and magnetization, rather than particle level effects.
For example, in statistical mechanics one learns about the Boltzmann distribution telling you the probability of a system being in a solid/liquid/gaseous state as function of its energy and temperature. In studies of phase transitions, such as the melting of ice or in how proteins fold into structures that require folding intermediates before they can perform their function as antibodies.
In realizing this similarity, Hopfield and Hinton drew analogies between the concepts of Statistical Mechanics and the operation of neural networks. Just the way Boltzmann distribution suggests about the most probable state of a physical system, neural networks can incorporate similar principles to find out about more likely solutions of even complex computational problems.
Statistical Physics to Generative AI
Hopfield and Hinton´s work has had a significant effect in the area of artificial intelligence, particularly Craig (2001) pecially in generative learning. Generative learning involves a neural network that learns, from training data only, to generate new data samples that resemble the training data (for example: generating new images of handwritten numbers (MNIST) or human-like text).
At the core of that approach is the Boltzmann machine, a neural network architecture invented by Hinton which involves visible neurons (the input data) as well as hidden neurons (interacted patterns discovered by the network). By summing the probabilities of all possible states for the hidden neurons, a Boltzmann machine can find out how probable one configuration is for, say visible neurons.
Over the past three years, my research group has been investigating how to run Boltzmann machines on quantum computers to extend generative learning even more. The work has countless applications ranging from art and entertainment to scientific discoveries and technological breakthroughs.
