Veljko Pejović

Matevž Fabjančič, Octavian Machidon, Hashim Sharif et al.

Runtime-tunable context-dependent network compression would make mobile deep learning (DL) adaptable to often varying resource availability, input "difficulty", or user needs. The existing compression techniques significantly reduce the memory, processing, and energy tax of DL, yet, the resulting models tend to be permanently impaired, sacrificing the inference power for reduced resource usage. The existing tunable compression approaches, on the other hand, require expensive re-training, do not support arbitrary strategies for adapting the compression and do not provide mobile-ready implementations. In this paper we present Mobiprox, a framework enabling mobile DL with flexible precision. Mobiprox implements tunable approximations of tensor operations and enables runtime-adaptable approximation of individual network layers. A profiler and a tuner included with Mobiprox identify the most promising neural network approximation configurations leading to the desired inference quality with the minimal use of resources. Furthermore, we develop control strategies that depending on contextual factors, such as the input data difficulty, dynamically adjust the approximation levels across a mobile DL model's layers. We implement Mobiprox in Android OS and through experiments in diverse mobile domains, including human activity recognition and spoken keyword detection, demonstrate that it can save up to 15% system-wide energy with a minimal impact on the inference accuracy.

Janez Božič, Amândio R. Faustino, Boris Radovič et al.

Progressing beyond centralized AI is of paramount importance, yet, distributed AI solutions, in particular various federated learning (FL) algorithms, are often not comprehensively assessed, which prevents the research community from identifying the most promising approaches and practitioners from being convinced that a certain solution is deployment-ready. The largest hurdle towards FL algorithm evaluation is the difficulty of conducting real-world experiments over a variety of FL client devices and different platforms, with different datasets and data distribution, all while assessing various dimensions of algorithm performance, such as inference accuracy, energy consumption, and time to convergence, to name a few. In this paper, we present CoLExT, a real-world testbed for FL research. CoLExT is designed to streamline experimentation with custom FL algorithms in a rich testbed configuration space, with a large number of heterogeneous edge devices, ranging from single-board computers to smartphones, and provides real-time collection and visualization of a variety of metrics through automatic instrumentation. According to our evaluation, porting FL algorithms to CoLExT requires minimal involvement from the developer, and the instrumentation introduces minimal resource usage overhead. Furthermore, through an initial investigation involving popular FL algorithms running on CoLExT, we reveal previously unknown trade-offs, inefficiencies, and programming bugs.

Boris Radovič, Veljko Pejović

Clustering clients into groups that exhibit relatively homogeneous data distributions represents one of the major means of improving the performance of federated learning (FL) in non-independent and identically distributed (non-IID) data settings. Yet, the applicability of current state-of-the-art approaches remains limited as these approaches cluster clients based on information, such as the evolution of local model parameters, that is only obtainable through actual on-client training. On the other hand, there is a need to make FL models available to clients who are not able to perform the training themselves, as they do not have the processing capabilities required for training, or simply want to use the model without participating in the training. Furthermore, the existing alternative approaches that avert the training still require that individual clients have a sufficient amount of labeled data upon which the clustering is based, essentially assuming that each client is a data annotator. In this paper, we present REPA, an approach to client clustering in non-IID FL settings that requires neither training nor labeled data collection. REPA uses a novel supervised autoencoder-based method to create embeddings that profile a client's underlying data-generating processes without exposing the data to the server and without requiring local training. Our experimental analysis over three different datasets demonstrates that REPA delivers state-of-the-art model performance while expanding the applicability of cluster-based FL to previously uncovered use cases.

Octavian Machidon, Jani Asprov, Tine Fajfar et al.

While the evolution of mobile computing is experiencing a considerable growth, it is at the same time seriously threatened by the limitations of the battery technology, which does not keep pace with the evergrowing increase in energy requirements of mobile applications. A novel approach for reducing the energy appetite of mobile apps comes from the approximate computing field, which proposes techniques that in a controlled manner sacrifice computation accuracy for higher energy savings. Building on this philosophy we propose a context-aware mobile video quality adaptation that reduces the energy needed for video playback, while ensuring that a user's quality expectations with respect to the mobile video are met. We confirm that the decoding resolution can play a significant role in reducing the overall power consumption of a mobile device and conduct two user studies to investigate how the context in which a video is played, its content, and the user's personality, modulate a user's quality expectations. We discover that a user's physical activity, the spatial/temporal properties of the video, and the user's personality traits interact and jointly influence the minimal acceptable playback resolution, paving the way for context-adaptable approximate mobile computing.

Christoph Anderson, Judith Simone Heinisch, Shohreh Deldari et al.

Pervasive and ubiquitous computing facilitates immediate access to information in the sense of always-on. Information such as news, messages, or reminders can significantly enhance our daily routines but are rendered useless or disturbing when not being aligned with our intrinsic interruptibility preferences. Attention management systems use machine learning to identify short-term opportune moments, so that information delivery leads to fewer interruptions. Humans' intrinsic interruptibility preferences - established for and across social roles and life domains - would complement short-term attention and interruption management approaches. In this article, we present our comprehensive results towards social role-based attention and interruptibility management. Our approach combines on-device sensing and machine learning with theories from social science to form a personalized two-stage classification model. Finally, we discuss the challenges of the current and future AI-driven attention management systems concerning privacy, ethical issues, and future directions.

Alina L. Machidon, Veljko Pejovic

Compressive sensing (CS) is a mathematically elegant tool for reducing the sampling rate, potentially bringing context-awareness to a wider range of devices. Nevertheless, practical issues with the sampling and reconstruction algorithms prevent further proliferation of CS in real world domains, especially among heterogeneous ubiquitous devices. Deep learning (DL) naturally complements CS for adapting the sampling matrix, reconstructing the signal, and learning form the compressed samples. While the CS-DL integration has received substantial research interest recently, it has not yet been thoroughly surveyed, nor has the light been shed on practical issues towards bringing the CS-DL to real world implementations in the ubicomp domain. In this paper we identify main possible ways in which CS and DL can interplay, extract key ideas for making CS-DL efficient, identify major trends in CS-DL research space, and derive guidelines for future evolution of CS-DL within the ubicomp domain.

Veljko Pejovic, Neal Lathia, Cecilia Mascolo et al.

The Experience Sampling Method (ESM) introduces in-situ sampling of human behaviour, and provides researchers and behavioural therapists with ecologically valid and timely assessments of a person's psychological state. This, in turn, opens up new opportunities for understanding behaviour at a scale and granularity that was not possible just a few years ago. The practical applications are many, such as the delivery of personalised and agile behaviour interventions. Mobile computing devices represent a revolutionary platform for improving ESM. They are an inseparable part of our daily lives, context-aware, and can interact with people at suitable moments. Furthermore, these devices are equipped with sensors, and can thus take part of the reporting burden off the participant, and collect data automatically. The goal of this survey is to discuss recent advancements in using mobile technologies for ESM (mESM), and present our vision of the future of mobile experience sampling.

Veljko Pejovic, Abhinav Mehrotra, Mirco Musolesi

The last two centuries saw groundbreaking advances in the field of healthcare: from the invention of the vaccine to organ transplant, and eradication of numerous deadly diseases. Yet, these breakthroughs have only illuminated the role that individual traits and behaviours play in the health state of a person. Continuous patient monitoring and individually-tailored therapies can help in early detection and efficient tackling of health issues. However, even the most developed nations cannot afford proactive personalised healthcare at scale. Mobile computing devices, nowadays equipped with an array of sensors, high-performance computing power, and carried by their owners at all time, promise to revolutionise modern healthcare. These devices can enable continuous patient monitoring, and, with the help of machine learning, can build predictive models of patient's health and behaviour. Finally, through their close integration with a user's lifestyle mobiles can be used to deliver personalised proactive therapies. In this article, we develop the concept of anticipatory mobile-based healthcare - anticipatory mobile digital health - and examine the opportunities and challenges associated with its practical realisation.

Veljko Pejovic, Mirco Musolesi

Today's mobile phones are far from mere communication devices they were ten years ago. Equipped with sophisticated sensors and advanced computing hardware, phones can be used to infer users' location, activity, social setting and more. As devices become increasingly intelligent, their capabilities evolve beyond inferring context to predicting it, and then reasoning and acting upon the predicted context. This article provides an overview of the current state of the art in mobile sensing and context prediction paving the way for full-fledged anticipatory mobile computing. We present a survey of phenomena that mobile phones can infer and predict, and offer a description of machine learning techniques used for such predictions. We then discuss proactive decision making and decision delivery via the user-device feedback loop. Finally, we discuss the challenges and opportunities of anticipatory mobile computing.