Qian Liu

Xinyuan Lu, Liangming Pan, Qian Liu et al.

Current scientific fact-checking benchmarks exhibit several shortcomings, such as biases arising from crowd-sourced claims and an over-reliance on text-based evidence. We present SCITAB, a challenging evaluation dataset consisting of 1.2K expert-verified scientific claims that 1) originate from authentic scientific publications and 2) require compositional reasoning for verification. The claims are paired with evidence-containing scientific tables annotated with labels. Through extensive evaluations, we demonstrate that SCITAB poses a significant challenge to state-of-the-art models, including table-based pretraining models and large language models. All models except GPT-4 achieved performance barely above random guessing. Popular prompting techniques, such as Chain-of-Thought, do not achieve much performance gains on SCITAB. Our analysis uncovers several unique challenges posed by SCITAB, including table grounding, claim ambiguity, and compositional reasoning. Our codes and data are publicly available at https://github.com/XinyuanLu00/SciTab.

Xinxing Wu, Tao Wang, Qian Liu et al.

We prove that the space of intuitionistic fuzzy values (IFVs) with a linear order based on a score function and an accuracy function has the same algebraic structure as the one induced by a linear order based on a similarity function and an accuracy function. By introducing a new operator for IFVs via the linear order based on a score function and an accuracy function, we show that such an operator is a strong negation on IFVs. Moreover, we observe that the space of IFVs is a complete lattice and a Kleene algebra with the new operator. We also demonstrate that the topological space of IFVs with the order topology induced by the above two linear orders is not separable and metrizable but compact and connected. From some new perspectives,our results partially answer three open problems posed by Atanassov [Intuitionistic Fuzzy Sets: Theory and Applications, Springer, 1999] and [On Intuitionistic Fuzzy Sets Theory, Springer, 2012]. Furthermore, we construct an isomorphism between the spaces of IFVs and q-rung orthopedic fuzzy values (q-ROFVs) under the corresponding linear orders. To this end, we introduce the concept of admissible similarity measures with particular orders for IFSs, extending the existing definition of the similarity measure for IFSs, and construct an admissible similarity measure with a linear order based on a score function and an accuracy function, which is effectively applied to a pattern recognition problem about the classification of building materials.