notes-cog-ai-fristonFreeEneregy

" Friston s free energy principle says that all life...is driven by the same universal imperative.... To be alive, he says, is to act in ways that reduce the gulf between your expectations and your sensory inputs. Or, in Fristonian terms, it is to minimize free energy ... Over time, Hinton convinced Friston that the best way to think of the brain was as a Bayesian probability machine. The idea, which goes back to the 19th century and the work of Hermann von Helmholtz, is that brains compute and perceive in a probabilistic manner, constantly making predictions and adjusting beliefs based on what the senses contribute. According to the most popular modern Bayesian account, the brain is an inference engine that seeks to minimize prediction error. ... So far, as you might have noticed, this sounds a lot like the Bayesian idea of the brain as an inference engine that Hinton told Friston about in the 1990s. And indeed, Friston regards the Bayesian model as a foundation of the free energy principle ( free energy is even a rough synonym for prediction error ). But the limitation of the Bayesian model, for Friston, is that it only accounts for the interaction between beliefs and perceptions; it has nothing to say about the body or action. It can t get you out of your chair.

This isn t enough for Friston, who uses the term active inference to describe the way organisms minimize surprise while moving about the world. When the brain makes a prediction that isn t immediately borne out by what the senses relay back, Friston believes, it can minimize free energy in one of two ways: It can revise its prediction absorb the surprise, concede the error, update its model of the world or it can act to make the prediction true. If I infer that I am touching my nose with my left index finger, but my proprioceptors tell me my arm is hanging at my side, I can minimize my brain s raging prediction-error signals by raising that arm up and pressing a digit to the middle of my face. ... For Friston, folding action and movement into the equation is immensely important. Even perception itself, he says, is enslaved by action : To gather information, the eye darts, the diaphragm draws air into the nose, the fingers generate friction against a surface. And all of this fine motor movement exists on a continuum with bigger plans, explorations,10 and actions. ... Julie Pitt, head of machine-learning infrastructure at Netflix, discovered Friston and the free energy principle in 2014, and it transformed her thinking. (Pitt s Twitter bio reads, I infer my own actions by way of Active Inference. ) Outside of her work at Netflix, she s been exploring applications of the principle in a side project called Order of Magnitude Labs. Pitt says that the beauty of the free energy model is that it allows an artificial agent to act in any environment, even one that s new and unknown. Under the old reinforcement-learning model, you d have to keep stipulating new rules and sub-rewards to get your agent to cope with a complex world. But a free energy agent always generates its own intrinsic reward: the minimization of surprise. And that reward, Pitt says, includes an imperative to go out and explore.

In late 2017, a group led by Rosalyn Moran, a neuroscientist and engineer at King s College London, pitted two AI players against one another in a version of the 3D shooter game Doom. The goal was to compare an agent driven by active inference to one driven by reward-maximization.

The reward-based agent s goal was to kill a monster inside the game, but the free-energy-driven agent only had to minimize surprise. The Fristonian agent started off slowly. But eventually it started to behave as if it had a model of the game, seeming to realize, for instance, that when the agent moved left the monster tended to move to the right.

After a while it became clear that, even in the toy environment of the game, the reward-maximizing agent was demonstrably less robust ; the free energy agent had learned its environment better. It outperformed the reinforcement-learning agent because it was exploring, Moran says. In another simulation that pitted the free-energy-minimizing agent against real human players, the story was similar. The Fristonian agent started slowly, actively exploring options epistemically foraging, Friston would say before quickly attaining humanlike performance.

Moran told me that active inference is starting to spread into more mainstream deep-learning research, albeit slowly. Some of Friston s students have gone on to work at DeepMind? and Google Brain, and one of them founded Huawei s Artificial Intelligence Theory lab. It s moving out of Queen Square, Moran says. But it s still not nearly as common as reinforcement learning, which even undergraduates learn. You don t teach undergraduates the free energy principle yet. ... Maxwell Ramstead...met Friston, who told him that the same math that applies to cellular differentiation the process by which generic cells become more specialized can also be applied to cultural dynamics...In 2017, Ramstead and Friston coauthored a paper, with Paul Badcock of the University of Melbourne, in which they described all life in terms of Markov blankets. Just as a cell is a Markov-blanketed system that minimizes free energy in order to exist, so are tribes and religions and species. " -- [www.wired.com/story/karl-friston-free-energy-principle-artificial-intelligence/]