|
Résumé |
Associative memories in physics, such as Hopfield-like networks, are models with interactions between spins built so that a set of chosen configurations, the examples, are local minima of the free energy in a certain region of the phase diagram. These minima are called memories, and any minimum that forms in the process that is not an example is called spurious. The traditional view on spurious minima is that they are undesirable and should be eliminated.
We showed [1, 2] that a type of spurious states, the mixtures of memories, allows even the simplest Hopfield network to "memorize" previously unseen examples if they share features with examples that were memorized in the first place.
I will review our recent works on random-features Hopfield networks and present a new strategy [3] to extend the same framework to modern architectures of deep learning. Preliminary results show that mixed spurious states play an important role even in generative diffusion [4, 5] and attention-based
architectures [6] |
