IN036
Opportunities, Challenges and Lessons Learned on Safe and Effective Use of Large Language Models and Agentic Workflows for Sciences
Session ID#: 252723
Session Description:
Large language models (LLMs) and agentic workflows are revolutionizing scientific research, enhancing literature/data discovery, analysis, coding, and insight generation. However, deploying these powerful AI tools demands careful attention to safety, reliability, and trustworthiness specific to science.This session examines both the transformative applications and the critical challenges. We will explore practical uses while emphasizing the need for robust safety measures. Key challenges include creating specialized guardrails against hallucinations and bias, developing rigorous evaluation methods (including uncertainty quantification), ensuring scientific integrity, data governance, and reproducibility, and addressing unique scientific risks.We seek submissions on novel science applications, LLM techniques for research, methods ensuring integrity, safety mechanisms (guardrails, risk mitigation, alignment), responsible AI frameworks (usability, human-in-the-loop, bias, ethics, fairness), and lessons learned from applications.
Index Terms:
1916 Data and information discovery [INFORMATICS]
1938 Knowledge representation and knowledge bases [INFORMATICS]
1954 Natural language processing [INFORMATICS]
1968 Scientific reasoning/inference [INFORMATICS]
Primary Convener: Muthukumaran Ramasubramanian, University of Alabama in Huntsville, Huntsville, AL, United States
Conveners: Armin Mehrabian, NASA Goddard Space Flight Center, ADNET Systems Inc., GES DISC, Greenbelt, United States, Bishwaranjan Bhattacharjee, IBM Research USA, Yorktown Heights, United States and Geeth De Mel, IBM Research UK, Hampshire, United Kingdom
Use of a Generative AI Chatbot for Developing Documentation of GFDL’s FRE Workflow (1880540)
Lauren Chilutti1, Ian Laflotte1,2, Thomas E Robinson Jr1, Chris Blanton1, Mikyung Lee1,2, Ciheim Brown1,2, Dana Singh1,2, Avery Kiihne1,2, Carolyn Whitlock1,2 and Jesse Lentz1,2, (1)Geophysical Fluid Dynamics Laboratory, Princeton, United States, (2)Science Applications International Corporation, Princeton, United States
A Hybrid LLM-Bayesian Network Agent Framework for Flood Resilience Analysis (1881926)
Erfan Amiri, GaiaMinds, Chief Technology Officer, Waterloo, ON, Canada, Shaghayegh Akbarpour, GaiaMinds, Chief Executive Officer, Waterloo, ON, Canada, Mohammad Sina Jahangir, Cornell University, Biological and Environmental Engineering, Ithaca, United States; GaiaMinds, Chief Scientific Officer, Waterloo, ON, Canada, Hamidreza Shirkhani, National Research Council, Research Officer, Ottawa, Canada, Scott Steinschneider, Cornell University, Ithaca, United States and Jan F Adamowski, McGill University, Department of Bioresource Engineering, Montreal, QC, Canada
Making Earth Science data accessible: An AI Co-Pilot for NASA’s North American Land Data Assimilation System (1948192)
Mahya Hashemi1, Sujay V Kumar2, Jocelynn Hartwig3, Juan Carlos Lopez3, Christopher Hain4 and John D Bolten5, (1)NASA Goddard Space Flight Center, SCIENCE APPLICATION INTL CORP, Greenbelt, United States, (2)NASA Goddard Space Flight Center, Greenbelt, United States, (3)Microsoft Corporation, Redmond, United States, (4)NASA Marshall Space Flight Cetner, Huntsville, United States, (5)Hydrological Sciences Laboratory, NASA Goddard Space Flight Center, Greenbelt MD, United States
Agentic Workflows for Gap-Aware Literature Reviews (1949929)
Movina Moses, IBM Research USA, Yorktown Heights, NY, United States, Nishan Pantha, University of Alabama in Huntsville, Huntsville, United States, Mohab Elkaref, IBM Research Europe, Warrington, United Kingdom, James Barry, IBM Research Europe, Dublin, Ireland, Vishnudev Kuruvanthodi, IBM Research Europe, Hursley, United Kingdom, Muthukumaran Ramasubramanian, University of Alabama in Huntsville, Huntsville, AL, United States, Campbell Watson, Organization Not Listed, New York, United States and Geeth De Mel, IBM Research UK, Hampshire, United Kingdom
INNOVATIVE LLM-DRIVEN AI AGENTS FOR CRISIS INTELLIGENCE: TOWARDS ALIGNING HUMAN SENSOR NETWORKS AND EO IN REAL TIME (1955491)
Sirisha Fnu, University of Maryland Baltimore County, Computer Science and Electrical Engineering, Baltimore, MD, United States, Milton Halem, University of Maryland Baltimore County, Computer Science, Baltimore, MD, United States and Oleg Aulov, University of Maryland Baltimore County, CSEE, Baltimore, MD, United States
Construction of Plant Disease Knowledge Graphs Using Large Language Models (1956301)
Alakananda Mitra1, Bryan Duong2, David Fleisher3, Vangimalla Reddy3 and Chittaranjan Ray4, (1)University of Nebraska Lincoln, Nebraska Water Center, Lincoln, NE, United States, (2)University of Nebraska Lincoln, School of Computing, Lincoln, United States, (3)USDA-ARS, Adaptive Cropping Systems Laboratory, Beltsville, United States, (4)University of Nebraska-Lincoln, Nebraska Water Center, Lincoln, NE, United States
Building Trustworthy AI Collaborators: Factuality and Source Attribution in Agentic Workflows (1963562)
Alessandra Pascale1, James Barry2, Movina Moses3, Mohab Elkaref4, Tigran Tchrakian2, Leo Thomas5, Muthukumaran Ramasubramanian6 and Geeth De Mel7, (1)Dublin, Dublin, IRELAND, (2)IBM Research Europe, Dublin, Ireland, (3)IBM Research USA, Yorktown Heights, NY, United States, (4)IBM Research Europe, Warrington, United Kingdom, (5)Development Seed, Washington DC, United States, (6)University of Alabama in Huntsville, Huntsville, AL, United States, (7)IBM Research UK, Hampshire, United Kingdom
Can LLM Fine-Tuning Solve the Puzzle of Mixed Benthic Habitats? A Case Study from the Red Sea (1967388)
Dr. Wenzhao Li1, Joshua Udo2, Surendra Maharjan1, Rejoice Thomas3 and Hesham Mohamed El-Askary1, (1)Chapman University, Schmid College of Science and Technology, Orange, CA, United States, (2)Grambling State University, Computer Science & Digital Technologies, Grambling, United States, (3)Chapman University, Schmid College of Science and Technology, Orange, United States
From Validation to Societal Value: A User-Centric Framework for Evaluating Regional Climate Models (1988068)
Hugo Lee, NASA Jet Propulsion Laboratory, Pasadena, CA, United States, Aashish Panta, University of Utah, Salt Lake City, UT, United States, Prof. Valerio Pascucci, University of Utah, Scientific Computing and Imaging Institute, Salt Lake City, UT, United States and Aashish
Towards Building a Scientific AI Ecosystem for FAIR and Trustworthy Earth and Space Science Research (1989769)
Muthukumaran Ramasubramanian1, Nishan Pantha2, Geeth De Mel3, Ajinkya Kulkarni4, Movina Moses5, Alessandra Pascale6, Iksha Gurung1, James Barry7, Tigran Tchrakian7, Leo Thomas8, Pranath Reddy Kumbam9, Carson Davis1 and Rahul Ramachandran10, (1)University of Alabama in Huntsville, Huntsville, AL, United States, (2)University of Alabama in Huntsville, Huntsville, United States, (3)IBM Research UK, Hampshire, United Kingdom, (4)The University of Alabama in Huntsville, Earth System Science Center, Huntsville, United States, (5)IBM Research USA, Yorktown Heights, NY, United States, (6)IBM research, Dublin, Ireland, (7)IBM Research Europe, Dublin, Ireland, (8)Development Seed, Washington DC, United States, (9)University of Alabama in Huntsville, Earth System Science Center, Huntsville, United States, (10)NASA Marshall Space Flight Center, Huntsville, United States