Scientists have developed rationalisation strategies to mitigate the risks associated with machine learning fashions that can produce unreliable results, enabling users to better comprehend when and how to trust a model’s predictions.
Despite their complexity, these explanations may occasionally include information on numerous model variants. While they’re often presented in a visually rich format, these explanations can prove challenging for those without prior machine learning knowledge to fully grasp and internalize.
Researchers at MIT employed large language models (LLMs) to convert complex, narrative-driven AI explanations into clear and concise language, facilitating a deeper understanding for the general public.
Developed was a sophisticated two-part system that seamlessly translates machine-learning rationalizations into coherent paragraphs of human-readable text, followed by objective evaluations of their narrative quality, empowering end-users with the confidence to determine credibility.
By providing limited instances, researchers can tailor narrative descriptions to meet customer preferences or specific application requirements.
In the long term, the researchers aim to build upon this methodology by allowing users to pose follow-up queries to the model regarding its thought process and predictive reasoning in real-world contexts.
“Our goal was to pioneer a new frontier in human-machine interaction by enabling customers to engage in nuanced conversations with machine-learning models about their decision-making processes, empowering them to make more informed choices.”
She collaborates with Sara Pido, an MIT postdoctoral fellow; Sarah Alnegheimish, a graduate student in Electrical Engineering and Computer Sciences; Laure Berti-Équille, research director at the French National Research Institute for Sustainable Development; and senior author Kalyan Veeramachaneni, principal research scientist in the Laboratory for Data and Decision Programs. The analysis will be provided on the IEEE Large-Scale Information Conference.
Researchers focused on a specific type of machine learning explanation known as SHAP. In a SHAP (SHapley Additive exPlanations) rationalization, a unique value or “value” is assigned to each feature used by the model for making a prediction, illustrating the contribution of each characteristic in driving the outcome. If a mannequin is tasked with predicting home costs, a crucial factor could be the location or situational context of the property in question. The location’s impact on the model’s prediction can be quantified by assigning an optimistic or adverse weight, indicative of the magnitude of change caused by that feature.
Typically, SHAP (SHapley Additive exPlanations) explanations are presented as bar plots showcasing the most and least important features. Despite being effective for a mannequin with fewer options, the bar plot quickly becomes cumbersome when there are more than 100 choices.
“As researchers, we must now navigate a plethora of decisions regarding the visual presentation of our findings.” When focusing exclusively on the top 10, people might wonder what happened to other aspects outside of the narrative. Without linguistic constraints, we are freed from the need to make such choices.
Instead of relying on a large language model to produce a text in plain language, researchers employ the LLM to convert an existing SHAP explanation into a coherent narrative.
By isolating the LLM’s focus solely on pure language processing within this methodology, Zytek notes that it minimizes the potential for inaccuracies in the reasoning process.
The EXPLINGO system comprises two interconnected components.
The core component, dubbed the Narrator, leverages Large Language Models (LLMs) to generate bespoke narrative summaries of Simplified Hypertextual Algorithmic Propositions (SHAP) interpretations tailored to users’ individual tastes and preferences. By seeding the NARRATOR with 3-5 exemplar narrative explanations, the Large Language Model will learn to emulate this style in generating textual content.
According to Zytek, instead of having consumers try to articulate their requirements, it’s more effective to simply ask them to write down what they’re looking for.
By displaying a tailored set of user-generated examples, the NARRATOR can be effortlessly customized for novel usage scenarios.
The NARRATOR generates a straightforward explanation, while the GRADER leverages Large Language Models (LLMs) to evaluate the narrative against four key criteria: brevity, precision, comprehensiveness, and coherence. The grading system automatically presents the Large Language Model (LLM) with the narrative text provided by the narrator, accompanied by the SHAP explanation outlining the reasoning behind the prompt.
“When evaluating the performance of language models, we find that even when they err in executing a task, they typically refrain from making mistakes during the subsequent validation process.”
Customers have the flexibility to tailor GRADER to assign distinct weights to each metric, allowing for a high degree of customization.
“In high-stakes situations, you may need to prioritize accuracy and completeness over fluency, she suggests.”
One of the primary hurdles faced by Zytek and her team was calibrating the Large Language Model to produce narrative text that mimicked real-world conversations with a high degree of authenticity. Will the large language model (LLM) accurately process these additional pointers, or will it inadvertently introduce errors into the reasoning?
“With meticulous attention to detail, she undertook the laborious task of identifying and rectifying each individual error.”
The researchers assessed their system’s efficacy by soliciting independent narrative explanations from a diverse group of customers for each of the nine machine-learning datasets provided. This enabled them to assess the versatility of the NARRATOR in mimicking unique styles. They employed the GRADER tool to extract every narrative justification across all four key performance indicators.
Over time, the research team found that their innovative system was capable of producing top-notch narrative explanations and effectively replicating various writing styles.
The research reveals that providing merely a handful of handcrafted example explanations significantly enhances the quality of the storytelling approach. Despite their importance, examples must be crafted meticulously – even seemingly innocuous phrases like “larger” can lead the grading algorithm to flag otherwise correct responses as inaccurate.
To leverage the findings, the researchers must develop tactics that enhance their system’s ability to effectively process comparative expressions. Additionally they need to increase EXPLINGO by providing logical explanations for the reasons.
Ultimately, their goal is to develop an interactive framework that enables users to engage in conversation with AI-powered models, requesting additional information and clarification on specific data points.
That will help facilitate decision-making in numerous ways. When people dispute a mannequin’s forecast, they should be able to quickly determine whether their intuition aligns with the model’s or not, and pinpoint where the disparity originates.
