8.3CLApr 5
CAWN: Continuous Acoustic Wave Networks for Autoregressive Language ModelingDejan Äugalj, Aleksandar Jevremovic
Modern Large Language Models (LLMs) rely on Transformer self-attention, which scales quadratically with sequence length. Recent linear-time alternatives, like State Space Models (SSMs), often suffer from signal degradation over extended contexts. We introduce the Continuous Acoustic Wave Network (CAWN), a fully continuous sequence-mixing architecture. Instead of discrete matrix-based attention, CAWN projects hidden states into multi-headed complex-domain phasors, achieving sequence mixing through a causal, $O(L)$ Phase Accumulation mechanism. To prevent signal degradation over ultra-long contexts, we introduce a dual-gated Selective Phase Resonance mechanism incorporating Frequency-Dependent Retention, Hard-Threshold Gating via Straight-Through Estimation, and a Temporal Syntax Cache to capture short-term local dependencies. We also replace standard dense linear projections with Depth-wise Harmonic Convolutions for optimal spatial frequency mixing, augmented by Block Attention Residuals for depth-wise state routing. Scaled to a 150M-parameter model, CAWN utilizes custom Triton kernels for hardware-efficient, true-complex phase accumulation in float32. Trained via a continuous streaming loop on a 100-Billion-token corpus, the prototype is evaluated at a 5-Billion-token milestone. Empirical evaluations via a Targeted Semantic Retrieval protocol demonstrate robust vocabulary acquisition and extended explicitly learned contextual denoising. By leveraging $O(1)$ state-passing via chunked prefill, the model retrieves targeted information across 2,000,000 tokens while strictly plateauing at 8.72 GB of Peak VRAM, empirically overcoming the $O(L^2)$ context memory wall.
IRFeb 15, 2022
Roomsemble: Progressive web application for intuitive property searchChris Kottmyer, Kevin Zhao, Zona Kostic et al.
A successful real estate search process involves locating a property that meets a user's search criteria subject to an allocated budget and time constraints. Many studies have investigated modeling housing prices over time. However, little is known about how a user's tastes influence their real estate search and purchase decisions. It is unknown what house a user would choose taking into account an individual's personal tastes, behaviors, and constraints, and, therefore, creating an algorithm that finds the perfect match. In this paper, we investigate the first step in understanding a user's tastes by building a system to capture personal preferences. We concentrated our research on real estate photos, being inspired by house aesthetics, which often motivates prospective buyers into considering a property as a candidate for purchase. We designed a system that takes a user-provided photo representing that person's personal taste and recommends properties similar to the photo available on the market. The user can additionally filter the recommendations by budget and location when conducting a property search. The paper describes the application's overall layout including frontend design and backend processes for locating a desired property. The proposed model, which serves as the application's core, was tested with 25 users, and the study's findings, as well as some key conclusions, are detailed in this paper.
CVJul 15, 2021
What Image Features Boost Housing Market Predictions?Zona Kostic, Aleksandar Jevremovic
The attractiveness of a property is one of the most interesting, yet challenging, categories to model. Image characteristics are used to describe certain attributes, and to examine the influence of visual factors on the price or timeframe of the listing. In this paper, we propose a set of techniques for the extraction of visual features for efficient numerical inclusion in modern-day predictive algorithms. We discuss techniques such as Shannon's entropy, calculating the center of gravity, employing image segmentation, and using Convolutional Neural Networks. After comparing these techniques as applied to a set of property-related images (indoor, outdoor, and satellite), we conclude the following: (i) the entropy is the most efficient single-digit visual measure for housing price prediction; (ii) image segmentation is the most important visual feature for the prediction of housing lifespan; and (iii) deep image features can be used to quantify interior characteristics and contribute to captivation modeling. The set of 40 image features selected here carries a significant amount of predictive power and outperforms some of the strongest metadata predictors. Without any need to replace a human expert in a real-estate appraisal process, we conclude that the techniques presented in this paper can efficiently describe visible characteristics, thus introducing perceived attractiveness as a quantitative measure into the predictive modeling of housing.
HCMay 13, 2021
Applying information theory and entropy to eliminate errors in mouse-tracking resultsAleksandar Jevremovic, Panayiotis Zaphiris, Sasa Adamovic et al.
Mouse-tracking of computer system users represents a less expensive, but also a far more applicable alternative to eye-tracking. The main disadvantage of mouse-tracking are errors manifested as discrepancies between the actual eye-gaze position and the mouse cursor position. This paper presents a method for automated correction of errors arising in mouse-tracking. Our approach draws upon information theory and employs Shannon entropy. The method is based on calculating the entropy of a visual representation of a Web page, i.e., we quantify information potential values of various positions. Information obtained, thereby, is paired with cumulative time intervals, spent by the mouse cursor in each position. In this way, we identify cursor positions that do not match eye-gaze positions. To verify the effectiveness of our method, we compared the eye gaze and mouse cursor heat maps in the following ways: We calculated the coefficient of correlation between the two; we computed Euclidean distance between their centers of gravity; and we performed visual comparison.
HCMay 11, 2021
Intelligent interactive technologies for mental health and well-beingMladjan Jovanovic, Aleksandar Jevremovic, Milica Pejovic-Milovancevic
Mental healthcare has seen numerous benefits from interactive technologies and artificial intelligence. Various interventions have successfully used intelligent technologies to automate the assessment and evaluation of psychological treatments and mental well-being and functioning. These technologies include different types of robots, video games, and conversational agents. The paper critically analyzes existing solutions with the outlooks for their future. In particular, we: i)give an overview of the technology for mental health, ii) critically analyze the technology against the proposed criteria, and iii) provide the design outlooks for these technologies.