O. M. Kiselev

O. M. Kiselev

We propose a minimal dynamical model of adaptive softmax routing for a two-expert Mixture-of-Experts (MoE) layer. The model is obtained as a mean-field limit of a discrete reinforcement rule: the selected expert receives a small score increment, while all scores undergo regularizing decay. In the symmetric case the limiting system has a supercritical pitchfork bifurcation: for weak feedback there is a unique stable balanced state, whereas above a critical feedback strength two stable asymmetric states appear. When an external asymmetry is added, the pitchfork unfolds into a pair of fold bifurcations forming a cusp in the control-parameter plane. We derive exact parametric equations for the bifurcation set and the local normal form of the cusp catastrophe. Numerical experiments connect this picture to empirical expert load, a small trainable MoE model, hard top-1 PyTorch routing, and a small classification experiment on digits. The results provide a controlled low-dimensional mechanism for abrupt transitions to load imbalance in adaptive MoE routers.

O. M. Kiselev

We study dynamics of the inverted pendulum on the wheel on a soft surface and under a proportional-integral-derivative controller. The behaviour of such pendulum is modelled by a system with a differential inclusion. If the the system has a sensor for the rotational velocity of the pendulum, the tilt sensor and the encoder for the wheel then this system is observable. The using of the observed data for the controller brings stochastic perturbations into the system. The properties of the differential inclusion under stochastic control is studied for upper position of the pendulum. The formula for the time, which the pendulum spends near the upper position, is derived.

O. M. Kiselev

We study dynamics of an wheeled inverted pendulum under a proportional-integral-derivative controller on horizontal, inclined and soft surfaces. An oscillatory area and conditions of the stability for the control are shown on the phase portraits of the dynamical systems. Particularly, we study a differential inclusion for moving on the soft surface, and we find semi-stable stationary solutions in our mathematical model. Due to rounding errors of the numerical modelling or external perturbations of robotics equipment the semistability looks as a limit cycle in simulations.