Natesh Ganesh

Adam N. McCaughan, Bakhrom G. Oripov, Natesh Ganesh et al.

We present multiplexed gradient descent (MGD), a gradient descent framework designed to easily train analog or digital neural networks in hardware. MGD utilizes zero-order optimization techniques for online training of hardware neural networks. We demonstrate its ability to train neural networks on modern machine learning datasets, including CIFAR-10 and Fashion-MNIST, and compare its performance to backpropagation. Assuming realistic timescales and hardware parameters, our results indicate that these optimization techniques can train a network on emerging hardware platforms orders of magnitude faster than the wall-clock time of training via backpropagation on a standard GPU, even in the presence of imperfect weight updates or device-to-device variations in the hardware. We additionally describe how it can be applied to existing hardware as part of chip-in-the-loop training, or integrated directly at the hardware level. Crucially, the MGD framework is highly flexible, and its gradient descent process can be optimized to compensate for specific hardware limitations such as slow parameter-update speeds or limited input bandwidth.

Natesh Ganesh

In this paper, we take a brief look at the advantages and disadvantages of dominant frameworks in consciousness studies -- functionalist and causal structure theories, and use it to motivate a new non-equilibrium thermodynamic framework of consciousness. The main hypothesis in this paper will be two thermodynamic conditions obtained from the non-equilibrium fluctuation theorems -- TCC 1 and 2, that the author proposes as necessary conditions that a system will have to satisfy in order to be 'conscious'. These descriptions will look to specify the functions achieved by a conscious system and restrict the physical structures that achieve them without presupposing either of the two. These represent an attempt to integrate consciousness into established physical law (without invoking untested novel frameworks in quantum mechanics and/or general relativity). We will also discuss it's implications on a wide range of existing questions, including a stance on the hard problem. The paper will also explore why this framework might offer a serious path forward to understanding consciousness (and perhaps even realizing it in artificial systems) as well as laying out some problems and challenges that lie ahead.