A confidently wrong chatbot is a nuisance. A confidently wrong rescue drone could cost a life. Stevens’ own K.P. “Suba” Subbalakshmi, a professor in the Department of Electrical and Computer Engineering, is working to make AI more reliable. Subbalakshmi recently received a three-year grant for her part in project SEA-NET (Scalable Edge Analytics and Networking for Autonomous Maritime Emergency Response). The collaborative project is supported by the National Science Foundation’s Verticals-enabling Intelligent Network Systems (VINES) program, a more than $110 million initiative to advance next-generation wireless networking. SEA-NET is led by the company Spectronn and includes researchers from Stevens, Purdue University, and Georgia Tech.
The project aims to build a reliable communications and computing system that helps drones and other equipment coordinate emergency response over areas where a central network may become interrupted. Subbalakshmi’s part is an “explainable network telemetry prediction,” which aims to forecast how well a network might perform in the near future.
However, this feat requires the AI to be able to distinguish between a human in immediate danger among debris and wreckage. The AI could incorrectly rely on misleading context clues and make the wrong assumptions about the given surroundings. Professor Raju Datla and Research Associate Professor Jon Miller of the Department of Civil, Environmental and Ocean Engineering at Stevens will help run experiments in the university’s tow tank at Davidson Laboratory, which has the ability to simulate oil spills and waves. The team eventually plans to test its work on the Hudson River.
Subbalakshmi identifies the problem with Large Language Models (LLMs) to be that they rely on context clues to come to a conclusion instead of focusing on genuinely identifiable data. She references an experiment in which a model was trained to tell dogs from wolves, which seemed to work until researchers examined the reasoning that brought the AI to its conclusion: It was relying on snow in the background, not the animal itself. The model got the right answer for the wrong reason, which could fail under different conditions, a risk that matters when lives are at stake.
Subbalakshmi’s attention is on regulating the LLM to self-correct and integrate training into it to allow for rewarding a truly correct answer rather than a response, more or less correcting hallucinations before they occur in the output.
Subbalakshmi, who has been at Stevens for 26 years, cautions against using AI as if it were lying or scheming as a person does. Its behavior comes from the training it is given, she says. As AI becomes increasingly independent, she says, anyone who relies on it for important decisions should be able to explain why.
