Who is Richard Sutton?
Richard S. Sutton is a University of Alberta computing science professor and Amii chief scientific adviser. He studies how agents learn by acting and observing consequences.[7]
From psychology to computing
He graduated in psychology from Stanford in 1978 and earned his computing master’s and doctorate at Massachusetts in 1980 and 1984. He subsequently worked at GTE Laboratories and returned to Massachusetts as a researcher. In 1998 he joined AT&T Shannon Labs. His career combines the study of learning with the construction of computational systems. [1]
The research community in Alberta
He joined the University of Alberta and Amii in 2003. There he helped develop a reinforcement-learning community connecting research and education. In 2021 he was elected a Fellow of the Royal Society for his contributions to the field, recognising a career that began long before generative AI reached a mass audience. [8]
In July 2017, DeepMind announced its Edmonton laboratory, the company’s first outside the United Kingdom. Sutton was named among its scientific leaders alongside Michael Bowling and Patrick Pilarski. All three would retain their university positions, so the collaboration brought together corporate research, teaching and academic work in Alberta. [9]
Temporal-difference learning
His 1988 paper on temporal differences explained how to update a prediction using a later prediction, without always waiting for the final outcome. The method enables learning as new experience arrives. The paper established mathematical properties for particular cases; its importance does not amount to a convergence guarantee for every network, task or configuration. [10]
Dyna: learning, planning and acting
Introduced in 1991, Dyna combines learning, planning and reactive execution. The agent learns a model of the effects of its actions and uses it for planning, alongside learning directly from interaction. The architecture combines actual experience with experience calculated using that model, whose approximate nature forms part of the problem the system must address. [11]
Actions extending over multiple steps
With Doina Precup and Satinder Singh, he developed the options framework in 1999. An option represents behaviour that can extend across several steps and respond to events during execution. This abstraction allows reinforcement-learning systems to reason about complete tasks, rather than working only with elementary actions of a fixed duration. [12]
Policy gradients and stability
In another 1999 paper, with David McAllester, Satinder Singh and Yishay Mansour, he investigated directly adjusting a policy: the rule that selects actions. The result connected improvement with gradients estimated from experience and value approximations. Its guarantees concerned mathematical conditions and local optima, rather than a universal solution for every decision problem. [13]
With A. Rupam Mahmood and Martha White, he subsequently investigated emphatic temporal-difference methods. Published in JMLR in 2016, the paper changes the weighting of updates to stabilise learning when data come from a different policy than the one being evaluated. Its central result concerns the stability of the expected update with linear approximation. [15]
Simultaneous predictions in robotics
Work with Joseph Modayil and Adam White on nexting brought these ideas to a robot learning many simultaneous predictions of its sensory signals. The 2012 version examined horizons from one tenth of a second to eight seconds. The experiment demonstrated real-time learning on that platform, connecting environmental representation with expectations that could be tested against subsequent observations. [14]
The textbook with Andrew Barto
Sutton and Andrew Barto published Reinforcement Learning: An Introduction in 1998. Their collaboration helped bring together the ideas and methods of a field connecting learning, control and decision-making. The textbook forms part of a wider contribution to training researchers and building a scientific community dedicated to learning from experience. [3]
MIT Press published the second edition in 2018.[22]
The Bitter Lesson and computation
In his 2019 essay The Bitter Lesson, he argued that general methods able to benefit from more computation have outperformed many manually engineered, domain-specific approaches over the long term. He presented this as a historical interpretation of AI and a guide to research, rather than a theorem determining the best method for every application. [17]
The reward hypothesis and its debate
In Reward is enough, published in 2021 with David Silver, Satinder Singh and Doina Precup, he proposed that maximising reward could explain the emergence of many intelligent abilities. The article advances a hypothesis about intelligence and learning. It does not demonstrate that any reward function produces general reasoning or that every relevant objective is easy to design. [18]
The proposal received a scientific response from Peter Vamplew and other researchers. Their objection concerns representing objectives through a single scalar reward, and they argue for explicitly considering multiple objectives. This exchange shows that the conceptual foundations of reinforcement learning involve debates about what is optimised, as well as questions about how to optimise it. [24]
The Alberta Plan for AI Research
With Michael Bowling and Patrick Pilarski, he presented The Alberta Plan for AI Research, released from 2022 and revised in 2023. The agenda proposes agents that continually learn, perceive, act and plan with limited resources. It organises representation, prediction and control problems into a research roadmap, describing outstanding work rather than general intelligence already achieved. [4]
Retaining the ability to keep learning
He coauthored a 2024 Nature study of plasticity loss: networks becoming worse at learning from new examples after prolonged training. The team investigated a backpropagation variant that reinitialises little-used units. Its experiments provided evidence about maintaining adaptation in the settings examined and distinguished this problem from forgetting previously acquired knowledge. [16]
In 2025, David Silver and Sutton published Welcome to the Era of Experience. They proposed agents generating experience through their interactions rather than relying mainly on collected human data. This is a research perspective on future capabilities and sources of learning.[19]
Keen Technologies and Oak Lab
In 2023 he began collaborating with John Carmack at Keen Technologies on artificial general intelligence research. [7]
In July 2026 he announced that he and Khurram Javed had left Keen to establish Oak Lab. He presented the change as a new route to investigating intelligence based on experience and revisiting learning methods. The announcement distinguishes this stage from his previous corporate positions, without equating a new organisation with a scientific result. [20]
Oak Lab defines its mission around agents learning from streams of experience and planning in real time with greater efficiency. Its goals describe a development direction. The connection with Sutton’s career lies in continual learning and the selection of useful information, questions already present in his earlier research. [21]
Turing Award and recognition
He shared the 2024 Turing Award, announced in 2025, with Andrew Barto. [3]
He also received the 2021 IJCAI Award for Research Excellence. [23]
Reinforcement learning applied to working posture
Sutton coauthored research with Thomas Degris and Patrick Pilarski on reinforcement learning with continuous actions. This approach investigates how a system can gradually adjust its behavior using the feedback it receives, rather than choosing only between discrete actions. [5]
A later study by Xie and colleagues uses reinforcement learning to adjust a collaborative robot’s position and orientation and improve participants’ awkward-posture scores. It cites Sutton’s work. [6]
What the result means for ergonomics
The preventive connection is specific: changing where a robot presents a part can change the posture a person adopts. The learning objective must represent the postural exposure being reduced; optimizing cycle time alone does not address that question.
Sutton is not an author of the ergonomics experiment. Moreover, improvement during a controlled task does not establish fewer long-term injuries. Transferring the method to a workplace would require checking variations in body size, reach, fatigue and tasks, alongside conditions for safe robot interaction.
