5.2AIMay 19
BLINKG: A Benchmark for LLM-Integrated Knowledge Graph GenerationCarla Castedo, Enrique Iglesias, Manuel Lama et al.
Generating Knowledge Graphs (KGs) remains one of the most time-consuming and labor-intensive tasks for knowledge engineers, as they need to identify semantic equivalences between input data sources and ontology terms. While declarative solutions (e.g., RML, SPARQL-Anything) have helped to generalize this process, aligning input schema elements with ontology terms still involves intricate transformations and requires considerable manual effort. With the advent of Large Language Models (LLMs), there is growing interest in leveraging their capabilities to assist KG engineers. Although some studies have explored using LLMs to automate KG construction, there is still no standardized framework for assessing how effectively they establish correspondences between data schemes and ontology concepts. Therefore, in this paper, we propose BLINKG, a benchmark designed to evaluate the mapping capabilities of LLMs in constructing KGs from heterogeneous data sources. The benchmark includes a set of scenarios with increasing complexity, based on real-world use cases. We conduct an extensive experimental evaluation of several stateof-the-art LLMs using BLINK and observe that they already offer promising solutions. However, their performance remains limited in complex scenarios. Thanks to this benchmark, we can already assess the current capabilities of LLMs for KG construction. Additionally, we define a set of requirements for achieving (semi)automated (LLM-driven) KG construction, opening new research lines in this area.
12.9CLJul 12, 2024
Scalability of Bayesian Network Structure Elicitation with Large Language Models: a Novel Methodology and Comparative AnalysisNikolay Babakov, Ehud Reiter, Alberto Bugarin
In this work, we propose a novel method for Bayesian Networks (BNs) structure elicitation that is based on the initialization of several LLMs with different experiences, independently querying them to create a structure of the BN, and further obtaining the final structure by majority voting. We compare the method with one alternative method on various widely and not widely known BNs of different sizes and study the scalability of both methods on them. We also propose an approach to check the contamination of BNs in LLM, which shows that some widely known BNs are inapplicable for testing the LLM usage for BNs structure elicitation. We also show that some BNs may be inapplicable for such experiments because their node names are indistinguishable. The experiments on the other BNs show that our method performs better than the existing method with one of the three studied LLMs; however, the performance of both methods significantly decreases with the increase in BN size.
4.2AIOct 23, 2024
Explaining Bayesian Networks in Natural Language using Factor Arguments. Evaluation in the medical domainJaime Sevilla, Nikolay Babakov, Ehud Reiter et al.
In this paper, we propose a model for building natural language explanations for Bayesian Network Reasoning in terms of factor arguments, which are argumentation graphs of flowing evidence, relating the observed evidence to a target variable we want to learn about. We introduce the notion of factor argument independence to address the outstanding question of defining when arguments should be presented jointly or separately and present an algorithm that, starting from the evidence nodes and a target node, produces a list of all independent factor arguments ordered by their strength. Finally, we implemented a scheme to build natural language explanations of Bayesian Reasoning using this approach. Our proposal has been validated in the medical domain through a human-driven evaluation study where we compare the Bayesian Network Reasoning explanations obtained using factor arguments with an alternative explanation method. Evaluation results indicate that our proposed explanation approach is deemed by users as significantly more useful for understanding Bayesian Network Reasoning than another existing explanation method it is compared to.
0.5CLMay 23, 2023
Process-To-Text: A Framework for the Quantitative Description of Processes in Natural LanguageYago Fontenla-Seco, Alberto Bugarín-Diz, Manuel Lama
In this paper we present the Process-To-Text (P2T) framework for the automatic generation of textual descriptive explanations of processes. P2T integrates three AI paradigms: process mining for extracting temporal and structural information from a process, fuzzy linguistic protoforms for modelling uncertain terms, and natural language generation for building the explanations. A real use-case in the cardiology domain is presented, showing the potential of P2T for providing natural language explanations addressed to specialists.
0.5CLMay 16, 2023
Fuzzy Temporal Protoforms for the Quantitative Description of Processes in Natural LanguageYago Fontenla-Seco, Alberto Bugarín-Diz, Manuel Lama
In this paper, we propose a series of fuzzy temporal protoforms in the framework of the automatic generation of quantitative and qualitative natural language descriptions of processes. The model includes temporal and causal information from processes and attributes, quantifies attributes in time during the process life-span and recalls causal relations and temporal distances between events, among other features. Through integrating process mining techniques and fuzzy sets within the usual Data-to-Text architecture, our framework is able to extract relevant quantitative temporal as well as structural information from a process and describe it in natural language involving uncertain terms. A real use-case in the cardiology domain is presented, showing the potential of our model for providing natural language explanations addressed to domain experts.
4.9ROMay 31, 2019
Graduated Fidelity Lattices for Motion Planning under UncertaintyAdrián González-Sieira, Manuel Mucientes, Alberto Bugarín
We present a novel approach for motion planning in mobile robotics under sensing and motion uncertainty based on state lattices with graduated fidelity. The probability of collision is reliably estimated considering the robot shape, and the fidelity adapts to the complexity of the environment, improving the planning efficiency while maintaining the performance. Safe and optimal paths are found with an informed search algorithm, for which a novel multi-resolution heuristic is presented. Results for different scenarios and robot shapes are given, showing the validity of the proposed methods.
4.5AIMay 17, 2016
Fuzzy Sets Across the Natural Language Generation PipelineA. Ramos-Soto, A. Bugarín, S. Barro
We explore the implications of using fuzzy techniques (mainly those commonly used in the linguistic description/summarization of data discipline) from a natural language generation perspective. For this, we provide an extensive discussion of some general convergence points and an exploration of the relationship between the different tasks involved in the standard NLG system pipeline architecture and the most common fuzzy approaches used in linguistic summarization/description of data, such as fuzzy quantified statements, evaluation criteria or aggregation operators. Each individual discussion is illustrated with a related use case. Recent work made in the context of cross-fertilization of both research fields is also referenced. This paper encompasses general ideas that emerged as part of the PhD thesis "Application of fuzzy sets in data-to-text systems". It does not present a specific application or a formal approach, but rather discusses current high-level issues and potential usages of fuzzy sets (focused on linguistic summarization of data) in natural language generation.
3.0LGJul 17, 2015
FRULER: Fuzzy Rule Learning through Evolution for RegressionI. Rodríguez-Fdez, M. Mucientes, A. Bugarín
In regression problems, the use of TSK fuzzy systems is widely extended due to the precision of the obtained models. Moreover, the use of simple linear TSK models is a good choice in many real problems due to the easy understanding of the relationship between the output and input variables. In this paper we present FRULER, a new genetic fuzzy system for automatically learning accurate and simple linguistic TSK fuzzy rule bases for regression problems. In order to reduce the complexity of the learned models while keeping a high accuracy, the algorithm consists of three stages: instance selection, multi-granularity fuzzy discretization of the input variables, and the evolutionary learning of the rule base that uses the Elastic Net regularization to obtain the consequents of the rules. Each stage was validated using 28 real-world datasets and FRULER was compared with three state of the art enetic fuzzy systems. Experimental results show that FRULER achieves the most accurate and simple models compared even with approximative approaches.
7.4AINov 27, 2014
On the analysis of set-based fuzzy quantified reasoning using classical syllogisticsM. Pereira-Fariña, F. Díaz-Hermida, A. Bugarín
Syllogism is a type of deductive reasoning involving quantified statements. The syllogistic reasoning scheme in the classical Aristotelian framework involves three crisp term sets and four linguistic quantifiers, for which the main support is the linguistic properties of the quantifiers. A number of fuzzy approaches for defining an approximate syllogism have been proposed for which the main support is cardinality calculus. In this paper we analyze fuzzy syllogistic models previously described by Zadeh and Dubois et al. and compare their behavior with that of the classical Aristotelian framework to check which of the 24 classical valid syllogistic reasoning patterns or moods are particular crisp cases of these fuzzy approaches. This allows us to assess to what extent these approaches can be considered as either plausible extensions of the classical crisp syllogism or a basis for a general approach to the problem of approximate syllogism.
16.6AINov 18, 2014
Linguistic Descriptions for Automatic Generation of Textual Short-Term Weather Forecasts on Real Prediction DataA. Ramos-Soto, A. Bugarín, S. Barro et al.
We present in this paper an application which automatically generates textual short-term weather forecasts for every municipality in Galicia (NW Spain), using the real data provided by the Galician Meteorology Agency (MeteoGalicia). This solution combines in an innovative way computing with perceptions techniques and strategies for linguistic description of data together with a natural language generation (NLG) system. The application, named GALiWeather, extracts relevant information from weather forecast input data and encodes it into intermediate descriptions using linguistic variables and temporal references. These descriptions are later translated into natural language texts by the natural language generation system. The obtained forecast results have been thoroughly validated by an expert meteorologist from MeteoGalicia using a quality assessment methodology which covers two key dimensions of a text: the accuracy of its content and the correctness of its form. Following this validation GALiWeather will be released as a real service offering custom forecasts for a wide public.
4.6RONov 14, 2014
Learning Fuzzy Controllers in Mobile Robotics with Embedded PreprocessingI. Rodríguez-Fdez, M. Mucientes, A. Bugarín
The automatic design of controllers for mobile robots usually requires two stages. In the first stage,sensorial data are preprocessed or transformed into high level and meaningful values of variables whichare usually defined from expert knowledge. In the second stage, a machine learning technique is applied toobtain a controller that maps these high level variables to the control commands that are actually sent tothe robot. This paper describes an algorithm that is able to embed the preprocessing stage into the learningstage in order to get controllers directly starting from sensorial raw data with no expert knowledgeinvolved. Due to the high dimensionality of the sensorial data, this approach uses Quantified Fuzzy Rules(QFRs), that are able to transform low-level input variables into high-level input variables, reducingthe dimensionality through summarization. The proposed learning algorithm, called Iterative QuantifiedFuzzy Rule Learning (IQFRL), is based on genetic programming. IQFRL is able to learn rules with differentstructures, and can manage linguistic variables with multiple granularities. The algorithm has been testedwith the implementation of the wall-following behavior both in several realistic simulated environmentswith different complexity and on a Pioneer 3-AT robot in two real environments. Results have beencompared with several well-known learning algorithms combined with different data preprocessingtechniques, showing that IQFRL exhibits a better and statistically significant performance. Moreover,three real world applications for which IQFRL plays a central role are also presented: path and objecttracking with static and moving obstacles avoidance.
9.1AIOct 27, 2014
The probatilistic Quantifier Fuzzification Mechanism FA: A theoretical analysisFelix Diaz-Hermida, Alberto Bugarin, David E. Losada
The main goal of this work is to analyze the behaviour of the FA quantifier fuzzification mechanism. As we prove in the paper, this model has a very solid theorethical behaviour, superior to most of the models defined in the literature. Moreover, we show that the underlying probabilistic interpretation has very interesting consequences.