8.2CVApr 20
LLM-as-Judge Framework for Evaluating Tone-Induced Hallucination in Vision-Language ModelsZhiyuan Jiang, Weihao Hong, Xinlei Guan et al.
Vision-Language Models (VLMs) are increasingly deployed in settings where reliable visual grounding carries operational consequences, yet their behavior under progressively coercive prompt phrasing remains undercharacterized. Existing hallucination benchmarks predominantly rely on neutral prompts and binary detection, leaving open how both the incidence and the intensity of fabrication respond to graded linguistic pressure across structurally distinct task types. We present Ghost-100, a procedurally constructed benchmark of 800 synthetically generated images spanning eight categories across three task families -- text-illegibility, time-reading, and object-absence -- each designed under a negative-ground-truth principle that guarantees the queried target is absent, illegible, or indeterminate by construction. Every image is paired with five prompts drawn from a structured 5-Level Prompt Intensity Framework, holding the image and task identity fixed while varying only directive force, so that tone is isolated as the sole independent variable. We adopt a dual-track evaluation protocol: a rule-based H-Rate measuring the proportion of responses in which a model crosses from grounded refusal into unsupported positive commitment, and a GPT-4o-mini-judged H-Score on a 1-5 scale characterizing the confidence and specificity of fabrication once it occurs. We additionally release a three-stage automated validation workflow, which retrospectively confirms 717 of 800 images as strictly compliant. Evaluating nine open-weight VLMs, we find that H-Rate and H-Score dissociate substantially across model families, reading-style and presence-detection subsets respond to prompt pressure in qualitatively different ways, and several models exhibit non-monotonic sensitivity peaking at intermediate tone levels -- patterns that aggregate metrics obscure.
3.3AIJan 17, 2025
An Ontology for Social Determinants of Education (SDoEd) based on Human-AI Collaborative ApproachNavya Martin Kollapally, James Geller, Patricia Morreale et al.
The use of computational ontologies is well-established in the field of Medical Informatics. The topic of Social Determinants of Health (SDoH) has also received extensive attention. Work at the intersection of ontologies and SDoH has been published. However, a standardized framework for Social Determinants of Education (SDoEd) is lacking. In this paper, we are closing the gap by introducing an SDoEd ontology for creating a precise conceptualization of the interplay between life circumstances of students and their possible educational achievements. The ontology was developed utilizing suggestions from ChatGPT-3.5-010422 and validated using peer-reviewed research articles. The first version of developed ontology was evaluated by human experts in the field of education and validated using standard ontology evaluation software. This version of the SDoEd ontology contains 231 domain concepts, 10 object properties, and 24 data properties
5.6AIJan 19, 2017
Fuzzy Ontology-Based Sentiment Analysis of Transportation and City Feature Reviews for Safe TravelingFarman Ali, D. Kwak, Pervez Khan et al.
Traffic congestion is rapidly increasing in urban areas, particularly in mega cities. To date, there exist a few sensor network based systems to address this problem. However, these techniques are not suitable enough in terms of monitoring an entire transportation system and delivering emergency services when needed. These techniques require real-time data and intelligent ways to quickly determine traffic activity from useful information. In addition, these existing systems and websites on city transportation and travel rely on rating scores for different factors (e.g., safety, low crime rate, cleanliness, etc.). These rating scores are not efficient enough to deliver precise information, whereas reviews or tweets are significant, because they help travelers and transportation administrators to know about each aspect of the city. However, it is difficult for travelers to read, and for transportation systems to process, all reviews and tweets to obtain expressive sentiments regarding the needs of the city. The optimum solution for this kind of problem is analyzing the information available on social network platforms and performing sentiment analysis. On the other hand, crisp ontology-based frameworks cannot extract blurred information from tweets and reviews; therefore, they produce inadequate results. In this regard, this paper proposes fuzzy ontology-based sentiment analysis and SWRL rule-based decision-making to monitor transportation activities and to make a city- feature polarity map for travelers. This system retrieves reviews and tweets related to city features and transportation activities. The feature opinions are extracted from these retrieved data, and then fuzzy ontology is used to determine the transportation and city-feature polarity. A fuzzy ontology and an intelligent system prototype are developed by using Protégé OWL and Java, respectively.