Who is Andrew Barto?
Andrew G. Barto studies learning to act from consequences, connecting computer science, control, psychology and neuroscience.[4]
Education in mathematics and computing
He received his mathematics degree at Michigan in 1970 and his computing doctorate at the same university in 1975. [1]
His thesis was titled Cellular Automata as Models of Natural Systems. This early work places the modelling of natural systems among his interests before reinforcement learning. [17]
Research with Richard Sutton
He arrived at Massachusetts Amherst in 1977 as a postdoctoral researcher in a neural-network laboratory. There he began working with Richard Sutton, then a doctoral student, exploring learning models inspired by Harry Klopf’s ideas. Their collaboration sought to translate questions about adaptation and behaviour into artificial systems that improved through experience. [2]
He became an associate professor in 1982 and full professor in 1991. He chaired the department from 2007 to 2011 and retired in 2012. [1]
The 1983 control experiment
With Sutton and Charles Anderson, he published a system in 1983 that learned to balance a pole on a moving cart without being given its equations of motion. It combined an element selecting actions with another learning to evaluate them. The available signal indicated failure; the critic constructed a more informative evaluation to guide learning. [7]
Learning to act and plan
With Steven Bradtke and Satinder Singh, he introduced real-time dynamic programming, RTDP, in 1995. The method combines decisions and updates during interaction, focusing computation on situations the agent encounters. The paper connected learning and planning under uncertainty and established mathematical results for specified problem classes, developing a link with optimal control. [8]
A research case: coordinating elevators
In 1998, Robert Crites and Barto studied elevator-group control using multiple agents, each responsible for one car. All received a shared reward and had to learn with incomplete information and changing demand. The comparative results came from simulation; the case explored coordinating local decisions to improve the behaviour of the system as a whole. [9]
Hierarchies, skills and intermediate goals
With Sridhar Mahadevan, he examined in 2003 how to organise reinforcement learning through activities extending over several steps. Temporal abstraction reduces the need to make every decision through elementary actions. Their review compared hierarchical frameworks and analysed outstanding problems in coordination, concurrent activities and partial observation, helping organise a research direction broader than any single algorithm. [18]
With Amy McGovern, he developed a method in 2001 for discovering intermediate goals from common points in successful trajectories. Experiments used gridworld environments. The idea was that learning to reach these points could accelerate one task and provide reusable skills for related tasks, avoiding the need to specify every intermediate step manually. [10]
In 2005, with Özgür Şimşek and Alicia Wolfe, he investigated another way to discover subgoals: partitioning local graphs of observed transitions. The method sought access states between densely connected regions of the environment. The structure of recent experience could thereby suggest useful skills without first constructing a complete map of every possible situation. [11]
Intrinsic motivation and exploration
With Satinder Singh and Nuttapong Chentanez, he investigated in 2004 how an agent can acquire skills during activities not directed at an immediate external task. Their work presented initial computational results on intrinsic motivation and hierarchies of reusable skills. This question extends reinforcement learning towards exploration that prepares a system for future problems. [12]
In 2010, with Singh, Richard Lewis and Jonathan Sorg, he approached reward design from an evolutionary perspective. Their experiments distinguished the signal guiding learning from the criterion used to assess an agent’s success. The study examined how intrinsically and extrinsically motivated behaviours can emerge, rather than assuming a rigid computational boundary between them. [13]
With Marco Mirolli and Gianluca Baldassarre, he published Novelty or Surprise? in 2013. The article distinguishes encountering something not previously experienced from observing something that contradicts an expectation. This matters when interpreting exploration, attention and learning: a familiar stimulus can also be unexpected. It contributes a conceptual distinction for specifying which signals activate or modify behaviour. [14]
Robots learning from demonstrations
With George Konidaris, Scott Kuindersma and Roderic Grupen, he published a method in 2012 that divides demonstrations into skills and organises them into trees. Each component has a goal and can be improved through learning. The work included experiments with the uBot-5 mobile manipulator, connecting human demonstration, task representation and behaviour reuse on a robotic platform. [15]
Connections with the neuroscience of learning
He coauthored a 2011 Neuron study of hierarchical learning and brain activity. In three neuroimaging experiments, the team observed responses consistent with prediction errors relating to subgoals. The results supported investigating behavioural sequences through hierarchical models; the contribution was a specific experimental correspondence, rather than a complete explanation of how the brain works. [16]
Books and organising the discipline
He published Reinforcement Learning: An Introduction with Sutton in 1998. [17]
The second edition appeared in 2018.[3]
He also coedited the 2004 Handbook of Learning and Approximate Dynamic Programming with Jennie Si, Warren Powell and Donald Wunsch. [17]
Laboratory and researcher training
His teaching career is reflected in researchers including Richard Sutton, Satinder Singh, Amy McGovern, George Konidaris and Philip Thomas, listed among his doctoral students. In his personal introduction, Barto identifies learning in machines and animals as a shared interest. This combination explains the coexistence of engineering applications and questions about behaviour in his research. [6]
Turing Award and other recognition
He shared the 2024 Turing Award, announced in March 2025, with Richard Sutton. [4]
He received the IEEE Neural Networks Society Pioneer Award in 2004. [1]
In 2017 he received the IJCAI Award for Research Excellence. [19]
Rewards and ergonomics in collaborative robots
Barto and Richard Sutton systematized reinforcement learning: an agent learns to act through rewards and their future consequences. In prevention, this distinction requires asking which objective is being rewarded before judging a system’s ability to optimize it. [3]
An ergonomics experiment by Xie and colleagues cites their textbook and uses reinforcement learning to adjust a robot and improve working postures. [5]
Productivity is not a measure of protection
The connection is not a claim that Barto designed an OSH solution. His methods form part of the scientific foundation that other teams use to study a specific intervention: changing the robot’s position according to human posture.
The preventive lesson is to check what the reward leaves out. A movement that reduces an awkward posture may require reviewing reach, repetition and interactions with other people. A favorable ergonomic indicator therefore does not, on its own, establish the safety of the entire workstation or demonstrate fewer injuries.
