Nitish Nag

Nitish Nag, Vaibhav Pandey, Preston J. Putzel et al.

Individuals create and consume more diverse data about themselves today than any time in history. Sources of this data include wearable devices, images, social media, geospatial information and more. A tremendous opportunity rests within cross-modal data analysis that leverages existing domain knowledge methods to understand and guide human health. Especially in chronic diseases, current medical practice uses a combination of sparse hospital based biological metrics (blood tests, expensive imaging, etc.) to understand the evolving health status of an individual. Future health systems must integrate data created at the individual level to better understand health status perpetually, especially in a cybernetic framework. In this work we fuse multiple user created and open source data streams along with established biomedical domain knowledge to give two types of quantitative state estimates of cardiovascular health. First, we use wearable devices to calculate cardiorespiratory fitness (CRF), a known quantitative leading predictor of heart disease which is not routinely collected in clinical settings. Second, we estimate inherent genetic traits, living environmental risks, circadian rhythm, and biological metrics from a diverse dataset. Our experimental results on 24 subjects demonstrate how multi-modal data can provide personalized health insight. Understanding the dynamic nature of health status will pave the way for better health based recommendation engines, better clinical decision making and positive lifestyle changes.

Nitish Nag, Hyungik Oh, Mengfan Tang et al.

It is well understood that an individual's health trajectory is influenced by choices made in each moment, such as from lifestyle or medical decisions. With the advent of modern sensing technologies, individuals have more data and information about themselves than any other time in history. How can we use this data to make the best decisions to keep the health state optimal? We propose a generalized Personal Health Navigation (PHN) framework. PHN takes individuals towards their personal health goals through a system which perpetually digests data streams, estimates current health status, computes the best route through intermediate states utilizing personal models, and guides the best inputs that carry a user towards their goal. In addition to describing the general framework, we test the PHN system in two experiments within the field of cardiology. First, we prospectively test a knowledge-infused cardiovascular PHN system with a pilot clinical trial of 41 users. Second, we build a data-driven personalized model on cardiovascular exercise response variability on a smartwatch data-set of 33,269 real-world users. We conclude with critical challenges in health computing for PHN systems that require deep future investigation.

Ali Rostami, Vaibhav Pandey, Nitish Nag et al.

Food is central to life. Food provides us with energy and foundational building blocks for our body and is also a major source of joy and new experiences. A significant part of the overall economy is related to food. Food science, distribution, processing, and consumption have been addressed by different communities using silos of computational approaches. In this paper, we adopt a person-centric multimedia and multimodal perspective on food computing and show how multimedia and food computing are synergistic and complementary. Enjoying food is a truly multimedia experience involving sight, taste, smell, and even sound, that can be captured using a multimedia food logger. The biological response to food can be captured using multimodal data streams using available wearable devices. Central to this approach is the Personal Food Model. Personal Food Model is the digitized representation of the food-related characteristics of an individual. It is designed to be used in food recommendation systems to provide eating-related recommendations that improve the user's quality of life. To model the food-related characteristics of each person, it is essential to capture their food-related enjoyment using a Preferential Personal Food Model and their biological response to food using their Biological Personal Food Model. Inspired by the power of 3-dimensional color models for visual processing, we introduce a 6-dimensional taste-space for capturing culinary characteristics as well as personal preferences. We use event mining approaches to relate food with other life and biological events to build a predictive model that could also be used effectively in emerging food recommendation systems.

Daniel B. Azzam, Nitish Nag, Julia Tran et al.

Dry eye disease (DED) affects approximately half of the United States population. DED is characterized by dryness on the corena surface due to a variety of causes. This study fills the spatiotemporal gaps in DED epidemiology by using Google Trends as a novel epidemiological tool for geographically mapping DED in relation to environmental risk factors. We utilized Google Trends to extract DED-related queries estimating user intent from 2004-2019 in the United States. We incorporated national climate data to generate heat maps comparing geographic, temporal, and environmental relationships of DED. Multi-variable regression models were constructed to generate quadratic forecasts predicting DED and control searches. Our results illustrated the upward trend, seasonal pattern, environmental influence, and spatial relationship of DED search volume across US geography. Localized patches of DED interest were visualized along the coastline. There was no significant difference in DED queries across US census regions. Regression model 1 predicted DED searches over time (R^2=0.97) with significant predictors being control queries (p=0.0024), time (p=0.001), and seasonality (Winter p=0.0028; Spring p<0.001; Summer p=0.018). Regression model 2 predicted DED queries per state (R^2=0.49) with significant predictors being temperature (p=0.0003) and coastal zone (p=0.025). Importantly, temperature, coastal status, and seasonality were stronger risk factors of DED searches than humidity, sunshine, pollution, or region as clinical literature may suggest. Our work paves the way for future exploration of geographic information systems for locating DED and other diseases via online search query metrics.

Dhruv Upadhyay, Vaibhav Pandey, Nitish Nag et al.

Sleep is critical to leading a healthy lifestyle. Each day, most people go to sleep without any idea about how their night's rest is going to be. For an activity that humans spend around a third of their life doing, there is a surprising amount of mystery around it. Despite current research, creating personalized sleep models in real-world settings has been challenging. Existing literature provides several connections between daily activities and sleep quality. Unfortunately, these insights do not generalize well in many individuals. For these reasons, it is important to create a personalized sleep model. This research proposes a sleep model that can identify causal relationships between daily activities and sleep quality and present the user with specific feedback about how their lifestyle affects their sleep. Our method uses N-of-1 experiments on longitudinal user data and event mining to generate understanding between lifestyle choices (exercise, eating, circadian rhythm) and their impact on sleep quality. Our experimental results identified and quantified relationships while extracting confounding variables through a causal framework. These insights can be used by the user or a personal health navigator to provide guidance in improving sleep.

Vaibhav Pandey, Nitish Nag, Ramesh Jain

Knowing the state of our health at every moment in time is critical for advances in health science. Using data obtained outside an episodic clinical setting is the first step towards building a continuous health estimation system. In this paper, we explore a system that allows users to combine events and data streams from different sources to retrieve complex biological events, such as cardiovascular volume overload. These complex events, which have been explored in biomedical literature and which we call interface events, have a direct causal impact on relevant biological systems. They are the interface through which the lifestyle events influence our health. We retrieve the interface events from existing events and data streams by encoding domain knowledge using an event operator language.

Nitish Nag

Life's most valuable asset is health. Continuously understanding the state of our health and modeling how it evolves is essential if we wish to improve it. Given the opportunity that people live with more data about their life today than any other time in history, the challenge rests in interweaving this data with the growing body of knowledge to compute and model the health state of an individual continually. This dissertation presents an approach to build a personal model and dynamically estimate the health state of an individual by fusing multi-modal data and domain knowledge. The system is stitched together from four essential abstraction elements: 1. the events in our life, 2. the layers of our biological systems (from molecular to an organism), 3. the functional utilities that arise from biological underpinnings, and 4. how we interact with these utilities in the reality of daily life. Connecting these four elements via graph network blocks forms the backbone by which we instantiate a digital twin of an individual. Edges and nodes in this graph structure are then regularly updated with learning techniques as data is continuously digested. Experiments demonstrate the use of dense and heterogeneous real-world data from a variety of personal and environmental sensors to monitor individual cardiovascular health state. State estimation and individual modeling is the fundamental basis to depart from disease-oriented approaches to a total health continuum paradigm. Precision in predicting health requires understanding state trajectory. By encasing this estimation within a navigational approach, a systematic guidance framework can plan actions to transition a current state towards a desired one. This work concludes by presenting this framework of combining the health state and personal graph model to perpetually plan and assist us in living life towards our goals.

Nitish Nag, Bindu Rajanna, Ramesh Jain

With the exponential growth in the usage of social media to share live updates about life, taking pictures has become an unavoidable phenomenon. Individuals unknowingly create a unique knowledge base with these images. The food images, in particular, are of interest as they contain a plethora of information. From the image metadata and using computer vision tools, we can extract distinct insights for each user to build a personal profile. Using the underlying connection between cuisines and their inherent tastes, we attempt to develop such a profile for an individual based solely on the images of his food. Our study provides insights about an individual's inclination towards particular cuisines. Interpreting these insights can lead to the development of a more precise recommendation system. Such a system would avoid the generic approach in favor of a personalized recommendation system.

Nitish Nag, Vaibhav Pandey, Likhita Navali et al.

The health effects of air pollution have been subject to intense study in recent decades. Exposure to pollutants such as airborne particulate matter and ozone has been associated with increases in morbidity and mortality, especially with regards to respiratory and cardiovascular diseases. Unfortunately, individuals do not have readily accessible methods by which to track their exposure to pollution. This paper proposes how pollution parameters like CO, NO2, O3, PM2.5, PM10 and SO2 can be monitored for respiratory and cardiovascular personalized health during outdoor exercise events. Using location tracked activities, we synchronize them to public data sets of pollution sensors. For improved accuracy in estimation, we use heart rate data to understand breathing volume mapped with the local air quality sensors via constant GPS tracking.

Nitish Nag, Aditya Bharadwaj, Aditya Narendra Rao et al.

We propose a mechanism to use the features of flavour to enhance the quality of food recommendations. An empirical method to determine the flavour of food is incorporated into a recommendation engine based on major gustatory nerves. Such a system has advantages of suggesting food items that the user is more likely to enjoy based upon matching with their flavour profile through use of the taste biological domain knowledge. This preliminary intends to spark more robust mechanisms by which flavour of food is taken into consideration as a major feature set into food recommendation systems. Our long term vision is to integrate this with health factors to recommend healthy and tasty food to users to enhance quality of life.

Nitish Nag, Ramesh Jain

Lifestyle and environment interacting with our biological machine are primarily responsible for shaping our health and wellbeing. Continuous, multi-modal, and quantitative approaches to understanding and controlling these factors will allow each person to better reach their desired quality of life. A navigational paradigm can help users towards a specified health goal by using constantly captured measurements to feed estimations of how a user's health is continuously changing in order to provide actionable guidance. As various actions are taken by the user, measurements of the resulting effects loop back into the estimation and the next step of guidance. This perpetual cycle of measuring, estimating, guiding, and acting articulates a Personal Health Navigation information and actuation framework. Personal Health Navigation focuses on fulfilling a user's health goals by ensuring minimal deviation from healthy states, rather than treating disease or symptoms after derailment from proper biological function.

Nitish Nag, Vaibhav Pandey, Aishwarya Manjunath et al.

Road conditions affect both machine and human powered modes of transportation. In the case of human powered transportation, poor road conditions increase the work for the individual to travel. Previous estimates for these parameters have used computationally expensive analysis of satellite images. In this work, we use a computationally inexpensive and simple method by using only GPS data from a human powered cyclist. By estimating if the road taken by the user has high or low variations in their directional vector, we classify if the user is on a paved road or on an unpaved trail. In order to do this, three methods were adopted, changes in frequency of the direction of slope in a given path segment, fitting segments of the path, and finding the first derivative and the number of points of zero crossings of each segment. Machine learning models such as support vector machines, K-nearest neighbors, and decision trees were used for the classification of the path. We show in our methods, the decision trees performed the best with an accuracy of 86\%. Estimation of the type of surface can be used for many applications such as understanding rolling resistance for power estimation estimation or building exercise recommendation systems by user profiling as described in detail in the paper.

Vaibhav Pandey, Nitish Nag, Ramesh Jain

Advances in user interfaces, pattern recognition, and ubiquitous computing continue to pave the way for better navigation towards our health goals. Quantitative methods which can guide us towards our personal health goals will help us optimize our daily life actions, and environmental exposures. Ubiquitous computing is essential for monitoring these factors and actuating timely interventions in all relevant circumstances. We need to combine the events recognized by different ubiquitous systems and derive actionable causal relationships from an event ledger. Understanding of user habits and health should be teleported between applications rather than these systems working in silos, allowing systems to find the optimal guidance medium for required interventions. We propose a method through which applications and devices can enhance the user experience by leveraging event relationships, leading the way to more relevant, useful, and, most importantly, pleasurable health guidance experience.

Nitish Nag, Vaibhav Pandey, Ramesh C. Jain

People care more about the solutions to their problems rather than data alone. Inherently, this means using data to generate a list of recommendations for a given situation. The rapid growth of multi-modal wearables and sensors have not made this jump effectively in the domain of health. Modern user content consumption and decision making in both cyber (e.g. entertainment, news) and physical (eg. food, shopping) spaces rely heavily on targeted personalized recommender systems. The utility function is the primary ranking method to predict what a given person would explicitly prefer. In this work we describe two unique layers of user and context modeling that can be coupled to traditional recommender system approaches. The exogenous layer incorporates factors outside of the person's body (eg. location, weather, social context), while the endogenous layer integrates data to estimate the physiologic or innate needs of the user. This is accomplished through multi-modal sensor data integration applied to domain-specific utility functions, filters and re-ranking weights. We showcase this concept through a nutrition guidance system focused on controlling sodium intake at a personalized level, dramatically improving upon the fixed recommendations.

Nitish Nag, Mathias Lux, Ramesh C. Jain

Managing health lays the core foundation to enabling quality life experiences. Modern computer science research, and especially the field of recommender systems, has enhanced the quality of experiences in fields such as entertainment, shopping, and advertising; yet lags in the health domain. We are developing an approach to leverage multimedia for human health based on motivation modeling and recommendation of actions. Health is primarily a product of our everyday lifestyle actions, yet we have minimal health guidance on making everyday choices. Recommendations are the key to modern content consumption and decisions. Furthermore, long-term engagement with recommender systems is key for true effectiveness. Distinguishing intrinsic and extrinsic motivations from multi-modal data is key to provide recommendations that primarily fuel the intrinsic intentions, while using extrinsic motivation to further support intrinsic motivation. This understanding builds the foundation of sustainable behavioral adaptation for optimal personalized lifestyle health benefits.