Ramesh C. Jain

Nitish Nag, Vaibhav Pandey, Preston J. Putzel et al.

Individuals create and consume more diverse data about themselves today than any time in history. Sources of this data include wearable devices, images, social media, geospatial information and more. A tremendous opportunity rests within cross-modal data analysis that leverages existing domain knowledge methods to understand and guide human health. Especially in chronic diseases, current medical practice uses a combination of sparse hospital based biological metrics (blood tests, expensive imaging, etc.) to understand the evolving health status of an individual. Future health systems must integrate data created at the individual level to better understand health status perpetually, especially in a cybernetic framework. In this work we fuse multiple user created and open source data streams along with established biomedical domain knowledge to give two types of quantitative state estimates of cardiovascular health. First, we use wearable devices to calculate cardiorespiratory fitness (CRF), a known quantitative leading predictor of heart disease which is not routinely collected in clinical settings. Second, we estimate inherent genetic traits, living environmental risks, circadian rhythm, and biological metrics from a diverse dataset. Our experimental results on 24 subjects demonstrate how multi-modal data can provide personalized health insight. Understanding the dynamic nature of health status will pave the way for better health based recommendation engines, better clinical decision making and positive lifestyle changes.

Nitish Nag, Vaibhav Pandey, Ramesh C. Jain

People care more about the solutions to their problems rather than data alone. Inherently, this means using data to generate a list of recommendations for a given situation. The rapid growth of multi-modal wearables and sensors have not made this jump effectively in the domain of health. Modern user content consumption and decision making in both cyber (e.g. entertainment, news) and physical (eg. food, shopping) spaces rely heavily on targeted personalized recommender systems. The utility function is the primary ranking method to predict what a given person would explicitly prefer. In this work we describe two unique layers of user and context modeling that can be coupled to traditional recommender system approaches. The exogenous layer incorporates factors outside of the person's body (eg. location, weather, social context), while the endogenous layer integrates data to estimate the physiologic or innate needs of the user. This is accomplished through multi-modal sensor data integration applied to domain-specific utility functions, filters and re-ranking weights. We showcase this concept through a nutrition guidance system focused on controlling sodium intake at a personalized level, dramatically improving upon the fixed recommendations.

Nitish Nag, Mathias Lux, Ramesh C. Jain

Managing health lays the core foundation to enabling quality life experiences. Modern computer science research, and especially the field of recommender systems, has enhanced the quality of experiences in fields such as entertainment, shopping, and advertising; yet lags in the health domain. We are developing an approach to leverage multimedia for human health based on motivation modeling and recommendation of actions. Health is primarily a product of our everyday lifestyle actions, yet we have minimal health guidance on making everyday choices. Recommendations are the key to modern content consumption and decisions. Furthermore, long-term engagement with recommender systems is key for true effectiveness. Distinguishing intrinsic and extrinsic motivations from multi-modal data is key to provide recommendations that primarily fuel the intrinsic intentions, while using extrinsic motivation to further support intrinsic motivation. This understanding builds the foundation of sustainable behavioral adaptation for optimal personalized lifestyle health benefits.