As I settle into my seat at the dimly lit cinema, I find myself consumed by an unexpected concern: just how much soda remains in my oversized cup? Instead of removing the lid and visually inspecting the contents, you opt to gently rattle the cup to gauge the amount of ice within, thereby gaining insight into whether a complimentary refill is warranted.
As you place the drink back down, your gaze drifts away to wonder whether the armrest is crafted from genuine wood or not. Despite giving the object only a few taps and listening to the hollow echo, you conclude that it must be made of plastic.
We take for granted this ability to perceive and interpret the world through sound waves emitted by objects. Researchers are poised to bring this capability to robots, amplifying their rapidly expanding repertoire of sensing abilities.
The CoRL 2024 Convention on Robotic Studying, scheduled for November, will provide a premier forum for researchers and scientists to share their latest findings and advancements in the field of robotics. Researchers at Duke University have unveiled a innovative system, dubbed “6-9,” that enables robots to interact with their surroundings in ways previously thought to be exclusive to humans.
“Today’s robots primarily rely on visual perception to interpret their surroundings,” said Jiaxun Liu, lead author of the study and a first-year Ph.D. student. A scholar working within the laboratory of Professor Boyuan Chen, a renowned expert in mechanical engineering and materials science at Duke University. “To develop an answer that could effectively accommodate the ever-growing number of complex objects being discovered daily, we aimed to endow robots with a significantly enhanced capacity to experience and comprehend the world.”
The device incorporates a robotic hand featuring four articulated fingers, each equipped with a contact microphone discreetly integrated into the distal tip. The sensors accurately capture and record the vibrational patterns produced as the robotic faucets interact with objects through grasping or shaking actions. Since the microphones are in close proximity to the item, this enables the robot to filter out ambient noise effectively.
Utilizing advanced algorithms and integrating previous knowledge with cutting-edge AI advancements, this system identifies material compositions and three-dimensional structures by analyzing interactions and indicators. Innovative objects that defy prior experience require significantly more data processing; a minimum of 20 distinct interactions are necessary for the system to form a definitive understanding. When it’s an object that already exists within their database, they might efficiently establish a connection to it in as few as four steps.
According to Chen, the innovation provides robots with a novel means of perceiving and interacting with objects in a manner akin to humans, potentially revolutionizing current robotic capabilities. “While imagination is essential, sonic layers can unveil subtleties the human eye might overlook, providing a nuanced understanding.”
Within the paper and demonstrations, Chen and his laboratory showcase a range of capabilities empowered by their innovative technologies. When manipulating a container filled with cubes, it is possible to reveal both the quantity held inside and its shape. Can you tell exactly how much liquid is contained within by comparing it to an identical bottle of water? By traversing the external surface of an object in a manner akin to stumbling upon an object in the dead of night, the technology could potentially build a three-dimensional reconstruction of its shape and identify the constituent materials.
While traditional approaches often fall short in achieving optimal results, this innovative method surpasses previous efforts by leveraging four fingers instead of one, incorporating advanced touch-based microphones that effectively filter out ambient noise, and harnessing the power of sophisticated artificial intelligence capabilities. The setup enables the system to create complex objects comprised of multiple materials featuring intricate geometries, transparent, or reflective surfaces, and provides the capability to handle challenging visual-based tasks.
As Liu noted, unlike many datasets gathered in controlled laboratory environments or with human assistance, the team aimed to enable their robot to collaborate seamlessly with objects without external guidance in a freely accessible laboratory setting. It’s troublesome to replicate such complexity in simulations. The gap between managed and real-world information is crucial, bridging it by allowing robots to seamlessly collaborate with the complex, unstructured realities of the physical world.
These talents form a robust foundation for training robots to comprehend objects in dynamic and unstructured settings. By leveraging the same contact microphones used by musicians to capture sound from guitars, the innovative solution utilizes commercial components and 3D printing techniques, keeping production costs remarkably low at just over $200.
As they look towards the future, the team is focused on enhancing their collaborative abilities by successfully integrating with various objects and entities. As object-tracking algorithms are integrated, robots will be able to handle dynamic and cluttered environments with greater ease, ultimately achieving a level of adaptability similar to that of humans in real-world scenarios.
Significant advancements have been achieved through the innovative design of the robotic hand itself. “That is solely the start. As technology advances, we foresee the development of more sophisticated robotic hands with refined manipulation capabilities, allowing them to perform tasks that demand a delicate sense of touch, as envisioned by Chen. “We’re eager to explore the potential of combining multiple sensory modalities, such as strain and temperature, to create even more sophisticated and advanced interaction experiences.”
The work was supported by the Military Analysis Laboratory’s STRONG program (W911NF2320182, W911NF2220113), DARPA’s FoundSci program (HR00112490372), and TIAMAT (HR00112490419).
Jiaxun Liu and Boyuan Chen The ArXiv model is available for download at reference number 2406.17932v2, as well as on the website of the Normal Robotics Laboratory.
