Language Models as Reasoners for Out-of-Distribution Detection

Paper: Here

Our paper Language Models as Reasoners for Out-of-Distribution Detection has been presented at the Workshop on AI Safety Engineering (WAISE) 2024 and recieved the best paper award by popular vote.

It constitutes a basic extension of our idea of Out-of-Distribution Detection with Logical Reasoning, where we replaced the prolog-based reasoning component with a LLM.

Abstract

Deep neural networks (DNNs) are prone to making wrong predictions with high confidence for data that does not stem from their training distribution. Consequentially, out-of-distribution (OOD) detection is important in safety-critical applications, as it identifies such inputs. Using prior knowledge about the training distribution through formal constraints has shown promise in enhancing OOD detection. However, developing and maintaining formal knowledge bases can be cumbersome. Large language models (LLMs) have recently excelled in various natural language processing tasks. In this study, we investigate the use of LLMs for OOD detection, where domain constraints are expressed in natural language. Our results indicate that LLMs can outperform random guessing by leveraging general world knowledge learned during training. Moreover, LLMs can par with methods based on formal constraints when supplemented with domain-specific constraints articulated in natural language.

Presentation

The presentation slides are available here.


Last Updated: 17 Sep. 2024
Categories: Anomaly Detection · Neuro-Symbolic
Tags: WAISE · Anomaly Detection · Large Language Models · Neuro-Symbolic

Related Posts

Towards Deep Anomaly Detection with Structured Knowledge Representations, 15 Jun. 2023 (papers)
My paper Towards Deep Anomaly Detection with Structured Knowledge Representations has been accepted on the Workshop on AI Safety Engineering at SafeComp. Abstract Machine learning models tend to only make reliable predictions for inputs that are similar to the training data. Consequentially, anomaly detection, which can be used to detect unusual inputs, is critical for ensuring the safety of machine learning agents operating in open environments. In this work, we identify and discuss several limitations of current anomaly detection methods, such as their weak performance on tasks that require abstract reasoning, the inability to integrate background knowledge, and the opaqueness that undermines their trustworthiness in critical applications.
Thumbnail for Towards Deep Anomaly Detection with Structured Knowledge Representations
Out-of-Distribution Detection with Logical Reasoning, 04 Jan. 2024 (papers)
Our paper Out-of-Distribution Detction with Logical Reasoning has been accepted on the WACV 2024. Abstract Machine Learning models often only generalize reliably to samples from the training distribution. Consequentially, detecting when input data is out-of-distribution (OOD) is crucial, especially in safety-critical applications. Current OOD detection methods, however, tend to be domain agnostic and often fail to incorporate valuable prior knowledge about the structure of the training distribution. To address this limitation, we introduce a novel, hybrid OOD detection algorithm that combines a deep learning-based perception system with a first-order logic-based knowledge representation.
Thumbnail for Out-of-Distribution Detection with Logical Reasoning