Staas de Jong

Staas de Jong

Inspired by the role sound and friction play in interactions with everyday objects, this work aims to identify some of the ways in which kinetic surface friction rendering can complement interactive sonification controlled by movable objects. In order to do this, a tactile system is presented which implements a movable physical object with programmable friction. Important aspects of this system include the capacity to display high-resolution kinetic friction patterns, the ability to algorithmically define interactions directly in terms of physical units, and the complete integration of audio and tactile synthesis. A prototype interaction spatially mapping arbitrary 1D signal data on a surface and directly converting these to sound and friction during movements across the surface is described. The results of a pilot evaluation of this interaction indicate how kinetic surface friction rendering can be a means for giving dynamically created virtual objects for sonification a tangible presence. Some specific possible roles for movement input and friction output are identified, as well as issues to be considered when applying and further developing this type of haptic feedback in the context of interactive sonification.

Staas de Jong

The cyclotactor is a novel platform for finger-based tactile interaction research. The operating principle is to track vertical fingerpad position above a freely approachable surface aperture, while directly projecting a force on the same fingerpad. The projected force can be specified in Newtons, with high temporal resolution. In combination with a relatively low overall latency between tactile input and output, this is used to work towards the ideal of instant programmable haptic feedback. This enables support for output across the continuum between static force levels and vibrotactile feedback, targeting both the kinesthetic and cutaneous senses of touch. The current state of the technology is described, and an overview of the research goals of the cyclotactor project is given.

Staas de Jong, Jeroen Jillissen, Dünya Kirkali et al.

This work presents One-press control, a tactile input method for pressure-sensitive keyboards based on the detection and classification of pressing movements on the already held-down key. To seamlessly integrate the added control input with existing practices for ordinary computer keyboards, the redefined notion of virtual modifier keys is introduced. A number of application examples are given, especially to point out a potential for simplifying existing interactions by replacing modifier key combinations with single key presses. Also, a new class of interaction scenarios employing the technique is proposed, based on an interaction model named "What You Touch Is What You Get (WYTIWYG)". Here, the proposed tactile input method is used to navigate interaction options, get full previews of potential outcomes, and then either commit to one or abort altogether - all in the space of one key depress / release cycle. The results of user testing indicate some remaining implementation issues, as well as that the technique can be learned within about a quarter of an hour of hands-on operating practice time.

Staas de Jong

We present Ghostfinger, a technology for highly dynamic up/down fingertip haptics and control. The overall user experience offered by the technology can be described as that of tangibly and audibly interacting with a small hologram. More specifically, Ghostfinger implements automatic visualization of the dynamic instantiation/parametrization of algorithmic primitives that together determine the current haptic conditions for fingertip action. Some aspects of this visualization are visuospatial: A floating see-through cursor provides real-time, to-scale display of the fingerpad transducer, as it is being moved by the user. Simultaneously, each haptic primitive instance is represented by a floating block shape, type-colored, variably transparent, and possibly overlapping with other such block shapes. Further aspects of visualization are symbolic: Each instance is also represented by a type symbol, lighting up within a grid if the instance is providing output to the user. We discuss the system's user interface, programming interface, and potential applications. This is done from a general perspective that articulates and emphasizes the uniquely enabling role of the principle of computation in the implementation of new forms of instrumental control of musical sound. Beyond the currently presented technology, this also reflects more broadly on the role of Digital Musical Instruments (DMIs) in NIME.

Staas de Jong

This paper proposes a new research direction for the large family of instrumental musical interfaces where sound is generated using digital granular synthesis, and where interaction and control involve the (fine) operation of stiff, flat contact surfaces. First, within a historical context, a general absence of, and clear need for, tangible output that is dynamically instantiated by the grain-generating process itself is identified. Second, to fill this gap, a concrete general approach is proposed based on the careful construction of non-vibratory and vibratory force pulses, in a one-to-one relationship with sonic grains. An informal pilot psychophysics experiment initiating the approach was conducted, which took into account the two main cases for applying forces to the human skin: perpendicular, and lateral. Initial results indicate that the force pulse approach can enable perceivably multidimensional, tangible display of the ongoing grain-generating process. Moreover, it was found that this can be made to meaningfully happen (in real time) in the same timescale of basic sonic grain generation. This is not a trivial property, and provides an important and positive fundament for further developing this type of enhanced display. It also leads to the exciting prospect of making arbitrary sonic grains actual physical manipulanda.

Staas de Jong

This paper presents developments in the technology underlying the cyclotactor, a finger-based tactile I/O device for musical interaction. These include significant improvements both in the basic characteristics of tactile interaction and in the related (vibro)tactile sample rates, latencies, and timing precision. After presenting the new prototype's tactile output force landscape, some of the new possibilities for interaction are discussed, especially those for musical interaction with zero audio/tactile latency.

Staas de Jong

This paper reports on work in progress on a finger-based tactile I/O device for musical interaction. Central to the device is the ability to set up cyclical relationships between tactile input and output. A direct practical application of this to musical interaction is given, using the idea to multiplex two degrees of freedom on a single tactile loop.

Staas de Jong

This paper presents a device implementing a closed tactile loop for musical interaction, based on a small freely held magnet which serves as the medium for both input and output. The component parts as well as an example of its programmable behaviour are described.