Michinari Kono

Naoki Kimura, Michinari Kono, Jun Rekimoto

The availability of digital devices operated by voice is expanding rapidly. However, the applications of voice interfaces are still restricted. For example, speaking in public places becomes an annoyance to the surrounding people, and secret information should not be uttered. Environmental noise may reduce the accuracy of speech recognition. To address these limitations, a system to detect a user's unvoiced utterance is proposed. From internal information observed by an ultrasonic imaging sensor attached to the underside of the jaw, our proposed system recognizes the utterance contents without the user's uttering voice. Our proposed deep neural network model is used to obtain acoustic features from a sequence of ultrasound images. We confirmed that audio signals generated by our system can control the existing smart speakers. We also observed that a user can adjust their oral movement to learn and improve the accuracy of their voice recognition.

Kohei Toyoda, Michinari Kono, Jun Rekimoto

Contributions of recent deep-neural-network (DNN) based techniques have been playing a significant role in human-computer interaction (HCI) and user interface (UI) domains. One of the commonly used DNNs is human pose estimation. This kind of technique is widely used for motion capturing of humans, and to generate or modify virtual avatars. However, in order to gain accuracy and to use such systems, large and precise datasets are required for the machine learning (ML) procedure. This can be especially difficult for extreme/wild motions such as acrobatic movements or motions in specific sports, which are difficult to estimate in typically provided training models. In addition, training may take a long duration, and will require a high-grade GPU for sufficient speed. To address these issues, we propose a method to improve the pose estimation accuracy for extreme/wild motions by using pre-trained models, i.e., without performing the training procedure by yourselves. We assume our method to encourage usage of these DNN techniques for users in application areas that are out of the ML field, and to help users without high-end computers to apply them for personal and end use cases.

Michinari Kono, Jun Rekimoto

There has been a long history in electrical muscle stimulation (EMS), which has been used for medical and interaction purposes. Human-computer interaction (HCI) researchers are now working on various applications, including virtual reality (VR), notification, and learning. For the electric signals applied to the human body, various types of waveforms have been considered and tested. In typical applications, pulses with short duration are applied, however, many perspectives are required to be considered. In addition to the duration and polarity of the pulse/waves, the wave shapes can also be an essential factor to consider. A problem of conventional EMS toolkits and systems are that they have a limitation to the variety of signals that it can produce. For example, some may be limited to monophonic pulses. Furthermore, they are usually limited to rectangular pulses and a limited range of frequencies, and other waveforms cannot be produced. These kinds of limitations make us challenging to consider variations of EMS signals in HCI research and applications. The purpose of "{\it wavEMS}" is to encourage testing of a variety of waveforms for EMS, which can be manipulated through audio output. We believe that this can help improve HCI applications, and to open up new application areas.