Yoshifumi Seki

Shunsuke Kitada, Hitoshi Iyatomi, Yoshifumi Seki

Discontinuing ad creatives at an appropriate time is one of the most important ad operations that can have a significant impact on sales. Such operational support for ineffective ads has been less explored than that for effective ads. After pre-analyzing 1,000,000 real-world ad creatives, we found that there are two types of discontinuation: short-term (i.e., cut-out) and long-term (i.e., wear-out). In this paper, we propose a practical prediction framework for the discontinuation of ad creatives with a hazard function-based loss function inspired by survival analysis. Our framework predicts the discontinuations with a multi-modal deep neural network that takes as input the ad creative (e.g., text, categorical, image, numerical features). To improve the prediction performance for the two different types of discontinuations and for the ad creatives that contribute to sales, we introduce two new techniques: (1) a two-term estimation technique with multi-task learning and (2) a click-through rate-weighting technique for the loss function. We evaluated our framework using the large-scale ad creative dataset, including 10 billion scale impressions. In terms of the concordance index (short: 0.896, long: 0.939, and overall: 0.792), our framework achieved significantly better performance than the conventional method (0.531). Additionally, we confirmed that our framework (i) demonstrated the same degree of discontinuation effect as manual operations for short-term cases, and (ii) accurately predicted the ad discontinuation order, which is important for long-running ad creatives for long-term cases.

Ryosuke Homma, Yoshifumi Seki, Mitsuo Yoshida et al.

Dwell time has been widely used in various fields to evaluate content quality and user engagement. Although many studies shown that content with long dwell time is good quality, contents with short dwell time have not been discussed in detail. We hypothesize that content with short dwell time is not always low quality and does not always have low user engagement, but is instead related to user interest. The purpose of this study is to clarify the meanings of short dwell time browsing in mobile news application. First, we analyze the relation of short dwell time to user interest using large scale user behavior logs from a mobile news application. This analysis was conducted on a vector space based on users click histories and then users and articles were mapped in the same space. The users with short dwell time are concentrated on a specific position in this space; thus, the length of dwell time is related to their interest. Moreover, we also analyze the characteristics of short dwell time browsing by excluding these browses from their click histories. Surprisingly, excluding short dwell time click history, it was found that short dwell time click history included some aspect of user interest in 30.87% of instances where the cluster of users changed. These findings demonstrate that short dwell time does not always indicate a low level of user engagement, but also level of user interest.

Kenshin Sekimoto, Yoshifumi Seki, Mitsuo Yoshida et al.

The purpose of this study is to clarify what kind of news is easily retweeted and what kind of news is easily Liked. We believe these actions, retweeting and Liking, have different meanings for users. Understanding this difference is important for understanding people's interest in Twitter. To analyze the difference between retweets (RT) and Likes on Twitter in detail, we focus on word appearances in news titles. First, we calculate basic statistics and confirm that tweets containing news URLs have different RT and Like tendencies compared to other tweets. Next, we compared RTs and Likes for each category and confirmed that the tendency of categories is different. Therefore, we propose metrics for clarifying the differences in each action for each category used in the $χ$-square test in order to perform an analysis focusing on the topic. The proposed metrics are more useful than simple counts and TF-IDF for extracting meaningful words to understand the difference between RTs and Likes. We analyzed each category using the proposed metrics and quantitatively confirmed that the difference in the role of retweeting and Liking appeared in the content depending on the category. Moreover, by aggregating tweets chronologically, the results showed the trend of RT and Like as a list of words and clarified how the characteristic words of each week were related to current events for retweeting and Liking.

Takeshi Yoneda, Shunsuke Kozawa, Keisuke Osone et al.

Personalization plays an important role in many services, just as news does. Many studies have examined news personalization algorithms, but few have considered practical environments. This paper provides algorithms and system architecture for generating immediate personalized news in a practical environment. Immediacy means changes in news trends and user interests are reflected in recommended news lists quickly. Since news trends and user interests rapidly change, immediacy is critical in news personalization applications. We develop algorithms and system architecture to realize immediacy. Our algorithms are based on collaborative filtering of user clusters and evaluate news articles using click-through rate and decay scores based on the time elapsed since the user's last access. Existing studies have not fully discussed system architecture, so a major contribution of this paper is that we demonstrate a system architecture and realize our algorithms and a configuration example implemented on top of Amazon Web Services. We evaluate the proposed method both offline and online. The offline experiments are conducted through a real-world dataset from a commercial news delivery service, and online experiments are conducted via A/B testing on production environments. We confirm the effectiveness of our proposed method and also that our system architecture can operate in large-scale production environments.

Yoshifumi Seki, Mitsuo Yoshida

In the process of information gathering on the web, confirmation bias is known to exist, exemplified in phenomena such as echo chambers and filter bubbles. Our purpose is to reveal how people consume news and discuss these phenomena. In web services, we are able to use action logs of a service to investigate these phenomena. However, many existing studies about these phenomena are conducted via questionnaires, and there are few studies using action logs. In this paper, we attempt to discover biases of information gathering due to differences in user demographic attributes, such as age and gender, from the behavior log of the news distribution service. First, we summarized the actions in the service for each user attribute and showed the difference of user behavior depending on the attributes. Next, the degree of correlation between the attributes was measured using the correlation coefficient, and a strong correlation was found to exist in the browsing tendency of the news articles between the attributes. Then, the bias of keywords between attributes was discovered, keywords with bias in behavior among the attributes were found using parameters of regression analysis. Since these discovered keywords are almost explainable by big news, our proposed method is effective in detecting biased keywords.

Yoshifumi Seki, Mitsuo Yoshida

Dwell time indicates how long a user looked at a page, and this is used especially in fields where ratings from users such as search engines, recommender systems, and advertisements are important. Despite the importance of this index, however, its characteristics are not well known. In this paper, we analyze the dwell time of news pages according to category in smartphone application. Our aim is to clarify the characteristics of dwell time and the relation between length of news page and dwell time, for each category. The results indicated different dwell time trends for each category. For example, the social category had fewer news pages with shorter dwell time than peaks, compared to other categories, and there were a few news pages with remarkably short dwell time. We also found a large difference by category in the correlation value between dwell time and length of news page. Specifically, political news had the highest correlation value and technology news had the lowest. In addition, we found that a user tends to get sufficient information about the news content from the news title in short dwell times.

Kojiro Iizuka, Takeshi Yoneda, Yoshifumi Seki

Personalization plays an important role in many services. To evaluate personalized rankings, online evaluation, such as A/B testing, is widely used today. Recently, multileaving has been found to be an efficient method for evaluating rankings in information retrieval fields. This paper describes the first attempt to optimize the multileaving method for personalization settings. We clarify the challenges of applying this method to personalized rankings. Then, to solve these challenges, we propose greedy optimized multileaving (GOM) with a new credit feedback function. The empirical results showed that GOM was stable for increasing ranking lengths and the number of rankers. We implemented GOM on our actual news recommender systems, and compared its online performance. The results showed that GOM evaluated the personalized rankings precisely, with significantly smaller sample sizes (< 1/10) than A/B testing.

Shunsuke Kitada, Hitoshi Iyatomi, Yoshifumi Seki

Accurately predicting conversions in advertisements is generally a challenging task, because such conversions do not occur frequently. In this paper, we propose a new framework to support creating high-performing ad creatives, including the accurate prediction of ad creative text conversions before delivering to the consumer. The proposed framework includes three key ideas: multi-task learning, conditional attention, and attention highlighting. Multi-task learning is an idea for improving the prediction accuracy of conversion, which predicts clicks and conversions simultaneously, to solve the difficulty of data imbalance. Furthermore, conditional attention focuses attention of each ad creative with the consideration of its genre and target gender, thus improving conversion prediction accuracy. Attention highlighting visualizes important words and/or phrases based on conditional attention. We evaluated the proposed framework with actual delivery history data (14,000 creatives displayed more than a certain number of times from Gunosy Inc.), and confirmed that these ideas improve the prediction performance of conversions, and visualize noteworthy words according to the creatives' attributes.