The supermarket just around the corner has placed an order with a major manufacturer of frozen pizzas for 20 tons of Tonno, 10 tons of Salami, and 25 tons of Margherita. This task sounds like a straightforward one, but it remains manual labor that is currently carried out by hand. A straightforward and uncomplicated task that often lacks appeal to private individuals. During a five-day hackathon at igus in Cologne, 17 participants from the Fraunhofer Institute IAIS and IML, Lamarr-Institut für Maschinelles Lernen und Künstliche Intelligenz, and University of Bonn collaborated to develop a solution. Und zwar mithilfe von . The objective of the hackathon was to develop an automation solution that utilized Artificial Intelligence (AI) to automatically package pizzas. The challenge lay in programming a robot to recognize and pick the correct products upon verbal instruction from employees, then package them in mixed pallets. By integrating generative artificial intelligence (GenAI) and robotics, a solution should be developed that streamlines repetitive tasks in a more efficient and cost-effective manner.
During the hackathon, three groups worked on a real-world use case, positioning a ReBeL robotic arm in front of a conveyor belt with various pizza products placed on it. With the aid of a webcam and a KI-based segmentation system (Section Something Mannequin, or SAM), the systems recognized the various products on the conveyor belt and pinpointed their locations. The language model assigned these objects to the desired products based on natural-language instructions. Subsequently, the robot placed the products into the boxes, following the instructions provided by the speech model.
Alexander Zorn from the Fraunhofer IAIS expressed enthusiasm for the hackathon’s results: “We’re delighted to be working with igus’ robotics expertise to develop real Proof-of-Concepts for industrial customers.” The synergy between artificial intelligence and robotics holds vast potential to streamline and optimize work processes. Additionally, Alexander Mühlens, Prokurist and head of the Low Value Automation department at igus GmbH, emphasized the importance of hackathons: “Collaborating with institutes allows us to demonstrate to our customers what is possible through AI and low-value robotics.” Our dream is to effortlessly control robots using a voice command, and make that happen in every application. The hackathon demonstrated the vast potential for further automation through LLMs and robotics, such as simultaneously packing multiple boxes, sorting products according to specific instructions or verifying ingredient contents for allergens with camera systems. The successful collaboration between igus, the Fraunhofer IAIS, the Lamarr Institute of the University of Bonn, and the Fraunhofer IML underscores the potential of these technologies for industrial automation?
**Über die Associate**
Focusing on Artificial Intelligence (AI), Machine Learning, and High-Performance Computing, the University of Bonn is renowned as a hub for Artificial Intelligence and Robotics. The Fraunhofer Institute for Logistics (IML) in Dortmund brought its expertise in logistics automation and material flow management to the table. While artificial intelligence and machine learning are advancing rapidly.
