Sang–Rok Oh

Woomin Song, Jihoon Tack, Sangwoo Mo et al.

State-space models (SSMs) offer a promising architecture for sequence modeling, providing an alternative to Transformers by replacing expensive self-attention with linear recurrences. In this paper, we propose a simple yet effective trick to enhance SSMs within given computational budgets by sparsifying them. Our intuition is that tokens in SSMs are highly redundant due to gradual recurrent updates, and dense recurrence operations block the delivery of past information. In particular, we observe that upper layers of SSMs tend to be more redundant as they encode global information, while lower layers encode local information. Motivated by this, we introduce Simba, a hierarchical sparsification method for SSMs based on token pruning. Simba sparsifies upper layers more than lower layers, encouraging the upper layers to behave like highways. To achieve this, we propose a novel token pruning criterion for SSMs, measuring the global impact of tokens on the final output by accumulating local recurrences. We demonstrate that Simba outperforms the baseline model, Mamba, with the same FLOPS in various natural language tasks. Moreover, we illustrate the effect of highways, showing that Simba not only enhances efficiency but also improves the information flow across long sequences. Code is available at https://github.com/woominsong/Simba.

Tapomayukh Bhattacharjee, Yonghwan Oh, Sang-Rok Oh

This technical report gives an overview of our work on control algorithms dealing with redundant robot systems for achieving human-like motion characteristics. Previously, we developed a novel control law to exhibit human-motion characteristics in redundant robot arm systems as well as arm-trunk systems for reaching tasks [1], [2]. This newly developed method nullifies the need for the computation of pseudo-inverse of Jacobian while the formulation and optimization of any artificial performance index is not necessary. The time-varying properties of the muscle stiffness and damping as well as the low-pass filter characteristics of human muscles have been modeled by the proposed control law to generate human-motion characteristics for reaching motion like quasi-straight line trajectory of the end-effector and symmetric bell shaped velocity profile. This report focuses on the experiments performed using a 7-DOF redundant robot-arm system which proved the effectiveness of this algorithm in imitating human-like motion characteristics. In addition, we extended this algorithm to a 19-DOF Hand-Arm System for a reach-to-grasp task. Simulations using the 19-DOF Hand-Arm System show the effectiveness of the proposed scheme for effective human-like hand-arm coordination in reach-to-grasp tasks for pinch and envelope grasps on objects of different shapes such as a box, a cylinder, and a sphere.

Tapomayukh Bhattacharjee, Yonghwan Oh, Sang-Rok Oh

Many day-to-day activities require the dexterous manipulation of a redundant humanoid arm in complex 3D environments. However, position regulation of such robot arm systems becomes very difficult in presence of non-linear uncertainties in the system. Also, perturbations exist due to various unwanted interactions with obstacles for clumsy environments in which obstacle avoidance is not possible, and this makes position regulation even more difficult. This report proposes a non-linear task-space disturbance observer by virtue of which position regulation of such robotic systems can be achieved in spite of such perturbations and uncertainties. Simulations are conducted using a 7-DOF redundant robot arm system to show the effectiveness of the proposed method. These results are then compared with the case of a conventional mass-damper based task-space disturbance observer to show the enhancement in performance using the developed concept. This proposed method is then applied to a controller which exhibits human-like motion characteristics for reaching a target. Arbitrary perturbations in the form of interactions with obstacles are introduced in its path. Results show that the robot end-effector successfully continues to move in its path of a human-like quasi-straight trajectory even if the joint trajectories deviated by a considerable amount due to the perturbations. These results are also compared with that of the unperturbed motion of the robot which further prove the significance of the developed scheme.