In 2018, Alphabet’s subsidiary Google DeepMind developed AlphaZero, a program that employed machine learning and a proprietary algorithm to master three video games: chess, shogi, and Go, ultimately determining optimal moves within predefined grids to secure victories. A team of researchers from California Institute of Technology (Caltech) has created a novel algorithm for autonomous robots, a planning and decision-making framework that enables free-moving robots to choose optimal actions while navigating real-world environments.
“According to Quickly-Jo Chung, Caltech’s Bren Professor of Management and Dynamical Systems, and senior research scientist at JPL, managed by Caltech on behalf of NASA, their algorithm employs strategic thinking, simulates various scenarios and chooses the optimal solution through dynamic simulation – a process akin to playing multiple simulated video games featuring robotic movements.” “The game-changer lies in developing an eco-friendly method to identify the optimal safe trajectory, which conventional optimization techniques often overlook.”
In their December article in the journal, the team outlines SETS, a methodology for approaching problems that they term Spectral Enlargement Tree Search.
Robots are able to move freely along various routes without restriction. A cutting-edge companion, engineered to alleviate the daily burdens of a mature individual within their home environment, this humanoid robot is equipped with advanced sensors and AI-driven algorithms to intuitively understand the unique needs of its elderly charge. A robot designed for complex tasks should be capable of adapting to diverse scenarios and navigating through any path within the designated area as it encounters obstacles or unexpected events during task completion. While that robotic’s set of actions, obstacles, and challenges may diverge significantly from those of a self-driving car, such as navigating through dense forests or avoiding pedestrians on busy city streets.
How can a solitary algorithm integrate disparate robotic techniques to make informed decisions about navigating their surroundings?
John Lathrop, a graduate student in management and dynamical systems at Caltech, notes that “you don’t need a designer to dictate a discrete set of strikes a robot should be capable of performing, allowing for adjustments to be made without human intervention.” “To overcome this challenge, we came up with a solution called SETS.”
Utilizing principles from management and linear algebra, the SETS framework identifies optimal motion paths for robotic platforms to maximize their capabilities within a physical environment.
The fundamental concept hinges on a Monte Carlo Tree Search, a decision-making algorithm employed by Google’s renowned AlphaZero. Monte Carlo methods are often associated with randomness, while tree search typically describes traversing a hierarchical structure representing interconnected data within a complex system. A root branch diverges into infant nodes, connected by arcs. By applying Monte Carlo Tree Search to the game of Go, potential moves are visualized as novel nodes, with the tree expanding as more randomized iterations of possible paths are explored. The algorithm executes a series of feasible strike scenarios to visualize the ultimate consequences of various nodes, subsequently choosing the optimal outcome based on a predetermined scoring system.
The challenge, as Lathrop describes it, lies in the fact that the exponential growth of trajectories in the branching tree construction hinders the effective implementation of steady dynamical techniques reminiscent of robotic navigation within the physical environment. “For complex problems, the sheer scale of individual risks means that attempting to prioritize them one by one could take decades, if not centuries.”
By leveraging an optimal balance between exploration and exploitation, SETS effectively navigates the intricate decision-making process. “For us to truly understand the uncharted territories of trajectory simulation, we must venture beyond what has been explored before – that’s the essence of exploration,” Lathrop explains. “And so, we must continue exploring avenues that have previously produced substantial rewards – namely, exploitation.” By harmoniously striking a balance between exploration and exploitation, the algorithm swiftly converges on the optimal solution among all feasible pathways.
If a robot initially calculates a few feasible actions that it deduces would cause it to collide with a wall, there is no need for it to explore any other nodes on that branch of the decision tree.
Benjamin Rivière, a postdoctoral scholar in mechanical and civil engineering at Caltech and co-author of the study, notes that their approach enables robotic systems to balance exploration and exploitation by iteratively refining their motions, allowing for real-time adaptation to new data and motion planning.
SETS can execute a full-fledged tree search in mere tenths of a second. By processing that moment in real-time, the system could potentially generate hundreds or even thousands of feasible pathways, select the optimal one, and then execute accordingly. The relentless loop iterates ceaselessly, empowering the robotic system to generate a multitude of decisions in mere seconds.
A primary advantage of the SETS algorithm lies in its ability to be seamlessly integrated with a wide range of robotic platforms. Options and capabilities do not need to be programmed separately. The researchers’ novel approach is showcased in a series of three distinct experimental scenarios, a rarity in robotic literature that underscores the algorithm’s versatility.
In a simulated environment, a quadrotor drone successfully observed four hovering white objects while simultaneously evading four orange obstacles, all while navigating an airfield characterized by unpredictable and hazardous wind currents, or thermals. The CAST research centre at Caltech played host to a pioneering drone experiment. Within just two seconds, an autonomous system successfully augmented the driving skills of a human operator in a tracked floor vehicle, enabling it to safely navigate a narrow and winding track without colliding with the side rails? Within the remaining setup, sets enabled a pair of tethered spacecraft to capture and reorient a third agent – potentially characterising another spacecraft, an asteroid, or another object.
Researchers from California Institute of Technology, in collaboration with a team of college students, are currently applying a modified version of the Self-Evading Target System (SETS) algorithm to an Indy car that will compete in the Indy Autonomous Challenge at the Consumer Electronics Show (CES) in Las Vegas on January 9.
The research was funded by the Protection Superior Analysis Tasks Company’s Learning Integrated for Novel Cognitive systems (LINC) program, the Aerospace Corporation, and Supernal, with partial support from the National Science Foundation Graduate Research Fellowship Program.
