2.0LGOct 3, 2023
Ranking a Set of Objects using Heterogeneous Workers: QUITE an Easy ProblemAlessandro Nordio, Alberto tarable, Emilio Leonardi
We focus on the problem of ranking $N$ objects starting from a set of noisy pairwise comparisons provided by a crowd of unequal workers, each worker being characterized by a specific degree of reliability, which reflects her ability to rank pairs of objects. More specifically, we assume that objects are endowed with intrinsic qualities and that the probability with which an object is preferred to another depends both on the difference between the qualities of the two competitors and on the reliability of the worker. We propose QUITE, a non-adaptive ranking algorithm that jointly estimates workers' reliabilities and qualities of objects. Performance of QUITE is compared in different scenarios against previously proposed algorithms. Finally, we show how QUITE can be naturally made adaptive.
1.2SPMay 15, 2024
An Initial Study of Human-Scale Blockage in sub-THz Radio Propagation with Application to Indoor Passive LocalizationF. Paonessa, G. Virone, S. Kianoush et al.
This paper empirically investigates the body induced electromagnetic (EM) effects, namely the human body blockage, by conducting indoor measurement campaigns in the unexplored sub-THz W-band (75-110 GHz) and G-band (170-260 GHz). The proposed analysis focuses on both the alterations of channel frequency response induced by body presence, fully or partially obstructing the line-of-sight (LoS) between transmitter and recevier, as well as on the channel impulse response (CIR) for selected movements of the target, i.e. crossing the LoS of the radio link. Modelling of large scale parameters is also presented using a phantom body object. The proposed study has applications in device-free radio localization and radio frequency (RF) sensing scenarios where the EM radiation or environmental radio signals are collected and processed to detect and locate people without requiring them to wear any electronic devices. Although preliminary, the study reveals that discrimination of the blockage micro-movements is possible, achieving higher precision compared to classical RF sensing and localization using cm-scale wavelengths (2.4-6GHz bands).
Information Retrieval in the Age of Generative AI: The RGB ModelMichele Garetto, Alessandro Cornacchia, Franco Galante et al.
The advent of Large Language Models (LLMs) and generative AI is fundamentally transforming information retrieval and processing on the Internet, bringing both great potential and significant concerns regarding content authenticity and reliability. This paper presents a novel quantitative approach to shed light on the complex information dynamics arising from the growing use of generative AI tools. Despite their significant impact on the digital ecosystem, these dynamics remain largely uncharted and poorly understood. We propose a stochastic model to characterize the generation, indexing, and dissemination of information in response to new topics. This scenario particularly challenges current LLMs, which often rely on real-time Retrieval-Augmented Generation (RAG) techniques to overcome their static knowledge limitations. Our findings suggest that the rapid pace of generative AI adoption, combined with increasing user reliance, can outpace human verification, escalating the risk of inaccurate information proliferation across digital resources. An in-depth analysis of Stack Exchange data confirms that high-quality answers inevitably require substantial time and human effort to emerge. This underscores the considerable risks associated with generating persuasive text in response to new questions and highlights the critical need for responsible development and deployment of future generative AI tools.
4.3IRFeb 26, 2020
Ranking a set of objects: a graph based least-square approachEvgenia Christoforou, Alessandro Nordio, Alberto Tarable et al.
We consider the problem of ranking $N$ objects starting from a set of noisy pairwise comparisons provided by a crowd of equal workers. We assume that objects are endowed with intrinsic qualities and that the probability with which an object is preferred to another depends only on the difference between the qualities of the two competitors. We propose a class of non-adaptive ranking algorithms that rely on a least-squares optimization criterion for the estimation of qualities. Such algorithms are shown to be asymptotically optimal (i.e., they require $O(\frac{N}{ε^2}\log \frac{N}δ)$ comparisons to be $(ε, δ)$-PAC). Numerical results show that our schemes are very efficient also in many non-asymptotic scenarios exhibiting a performance similar to the maximum-likelihood algorithm. Moreover, we show how they can be extended to adaptive schemes and test them on real-world datasets.
3.5HCMay 26, 2016
The Importance of Worker Reputation Information in Microtask-Based Crowd Work SystemsA. Tarable, A. Nordio, E. Leonardi et al.
This paper presents the first systematic investigation of the potential performance gains for crowd work systems, deriving from available information at the requester about individual worker reputation. In particular, we first formalize the optimal task assignment problem when workers' reputation estimates are available, as the maximization of a monotone (submodular) function subject to Matroid constraints. Then, being the optimal problem NP-hard, we propose a simple but efficient greedy heuristic task allocation algorithm. We also propose a simple "maximum a-posteriori" decision rule and a decision algorithm based on message passing. Finally, we test and compare different solutions, showing that system performance can greatly benefit from information about workers' reputation. Our main findings are that: i) even largely inaccurate estimates of workers' reputation can be effectively exploited in the task assignment to greatly improve system performance; ii) the performance of the maximum a-posteriori decision rule quickly degrades as worker reputation estimates become inaccurate; iii) when workers' reputation estimates are significantly inaccurate, the best performance can be obtained by combining our proposed task assignment algorithm with the message-passing decision algorithm.
5.8HCDec 23, 2015
Selecting the top-quality item through crowd scoringAlessandro Nordio, Alberto Tarable, Emilio Leonardi et al.
We investigate crowdsourcing algorithms for finding the top-quality item within a large collection of objects with unknown intrinsic quality values. This is an important problem with many relevant applications, for example in networked recommendation systems. The core of the algorithms is that objects are distributed to crowd workers, who return a noisy and biased evaluation. All received evaluations are then combined, to identify the top-quality object. We first present a simple probabilistic model for the system under investigation. Then, we devise and study a class of efficient adaptive algorithms to assign in an effective way objects to workers. We compare the performance of several algorithms, which correspond to different choices of the design parameters/metrics. In the simulations we show that some of the algorithms achieve near optimal performance for a suitable setting of the system parameters.
6.2HCNov 26, 2014
The Importance of Being Earnest in Crowdsourcing SystemsAlberto Tarable, Alessandro Nordio, Emilio Leonardi et al.
This paper presents the first systematic investigation of the potential performance gains for crowdsourcing systems, deriving from available information at the requester about individual worker earnestness (reputation). In particular, we first formalize the optimal task assignment problem when workers' reputation estimates are available, as the maximization of a monotone (submodular) function subject to Matroid constraints. Then, being the optimal problem NP-hard, we propose a simple but efficient greedy heuristic task allocation algorithm. We also propose a simple ``maximum a-posteriori`` decision rule. Finally, we test and compare different solutions, showing that system performance can greatly benefit from information about workers' reputation. Our main findings are that: i) even largely inaccurate estimates of workers' reputation can be effectively exploited in the task assignment to greatly improve system performance; ii) the performance of the maximum a-posteriori decision rule quickly degrades as worker reputation estimates become inaccurate; iii) when workers' reputation estimates are significantly inaccurate, the best performance can be obtained by combining our proposed task assignment algorithm with the LRA decision rule introduced in the literature.