Zona Kostic

Anita Mahinpei, Zona Kostic, Chris Tanner

Data visualization captions help readers understand the purpose of a visualization and are crucial for individuals with visual impairments. The prevalence of poor figure captions and the successful application of deep learning approaches to image captioning motivate the use of similar techniques for automated figure captioning. However, research in this field has been stunted by the lack of suitable datasets. We introduce LineCap, a novel figure captioning dataset of 3,528 figures, and we provide insights from curating this dataset and using end-to-end deep learning models for automated figure captioning.

Chris Kottmyer, Kevin Zhao, Zona Kostic et al.

A successful real estate search process involves locating a property that meets a user's search criteria subject to an allocated budget and time constraints. Many studies have investigated modeling housing prices over time. However, little is known about how a user's tastes influence their real estate search and purchase decisions. It is unknown what house a user would choose taking into account an individual's personal tastes, behaviors, and constraints, and, therefore, creating an algorithm that finds the perfect match. In this paper, we investigate the first step in understanding a user's tastes by building a system to capture personal preferences. We concentrated our research on real estate photos, being inspired by house aesthetics, which often motivates prospective buyers into considering a property as a candidate for purchase. We designed a system that takes a user-provided photo representing that person's personal taste and recommends properties similar to the photo available on the market. The user can additionally filter the recommendations by budget and location when conducting a property search. The paper describes the application's overall layout including frontend design and backend processes for locating a desired property. The proposed model, which serves as the application's core, was tested with 25 users, and the study's findings, as well as some key conclusions, are detailed in this paper.

Zona Kostic, Aleksandar Jevremovic

The attractiveness of a property is one of the most interesting, yet challenging, categories to model. Image characteristics are used to describe certain attributes, and to examine the influence of visual factors on the price or timeframe of the listing. In this paper, we propose a set of techniques for the extraction of visual features for efficient numerical inclusion in modern-day predictive algorithms. We discuss techniques such as Shannon's entropy, calculating the center of gravity, employing image segmentation, and using Convolutional Neural Networks. After comparing these techniques as applied to a set of property-related images (indoor, outdoor, and satellite), we conclude the following: (i) the entropy is the most efficient single-digit visual measure for housing price prediction; (ii) image segmentation is the most important visual feature for the prediction of housing lifespan; and (iii) deep image features can be used to quantify interior characteristics and contribute to captivation modeling. The set of 40 image features selected here carries a significant amount of predictive power and outperforms some of the strongest metadata predictors. Without any need to replace a human expert in a real-estate appraisal process, we conclude that the techniques presented in this paper can efficiently describe visible characteristics, thus introducing perceived attractiveness as a quantitative measure into the predictive modeling of housing.

Zona Kostic, Jared Jessup, Jeffrey Baglioni et al.

An innumerable number of individual choices go into discovering a new book. There are unmistakably two groups of booklovers: those who like to search online, follow other people's latest readings, or simply react to a system's recommendations; and those who love to wander between library stacks, lose themselves behind bookstore shelves, or simply hide behind piles of (un)organized books. Depending on which group a person may fall into, there are two distinct and corresponding mediums that inform his or her choices: digital, that provides efficient retrieval of information online, and physical, a more tactile pursuit that leads to unexpected discoveries and promotes serendipity. How could we possibly bridge the gap between these seemingly disparate mediums into an integrated system that can amplify the benefits they both offer? In this paper, we present the BookVIS application, which uses book-related data and generates personalized visualizations to follow users in their quest for a new book. In this new redesigned version, the app brings associative visual connections to support intuitive exploration of easily retrieved digital information and its relationship with the physical book in hand. BookVIS keeps track of the user's reading preferences and generates a dataSelfie as an individual snapshot of a personal taste that grows over time. Usability testing has also been conducted and has demonstrated the app's ability to identify distinguishable patterns in readers' tastes that could be further used to communicate personal preferences in new "shelf-browsing" iterations. By efficiently supplementing the user's cognitive information needs while still supporting the spontaneity and enjoyment of the book browsing experience, BookVIS bridges the gap between real and online realms, and maximizes the engagement of personalized mobile visual clues.