Reliability and predictability
Working together with computer systems seamlessly and predictably is essential. When developing software programs, engineers first create detailed designs, then write code that instructs the computer exactly how to execute each step. With its agentive AI course, we do not provide step-by-step instructions. Quite logically, we define our desired outcome and task the agent to achieve this goal. While the software program’s agent exhibits autonomy, this freedom may also introduce unpredictable elements into its outputs.
We observed a similar issue when ChatGPT and other Large Language Model-based technologies were initially introduced. Within the past two years, significant advancements have been made in the consistency of generative AI outputs, driven by fine-tuning, human-in-the-loop feedback mechanisms, and sustained efforts to train and refine these models. To ensure the reliability of agentic AI methods, we’ll need to invest a comparable level of energy in reducing their inherent unpredictability and making them more stable.
Information privateness and safety
While some corporations are hesitant to leverage agentive AI due to privacy and security concerns mirroring those surrounding generative AI, the stakes may prove even higher in this case. When someone interacts with an AI model, every piece of information provided to it becomes deeply ingrained within its digital framework. There’s no feasible way to revise and overlook that information, effectively precluding any possibility of reconsideration or reevaluation. Several types of malicious attacks, such as direct injection, take advantage of vulnerabilities in safety systems by attempting to extract sensitive information from models. As a consequence of software program brokers’ unfettered access to multiple systems with high levels of autonomy, the risk is heightened that they may inadvertently expose sensitive personal information from diverse sources.
