2.4NEFeb 3, 2020
Exponential discretization of weights of neural network connections in pre-trained neural networksMagomed Yu. Malsagov, Emil M. Khayrov, Maria M. Pushkareva et al.
To reduce random access memory (RAM) requirements and to increase speed of recognition algorithms we consider a weight discretization problem for trained neural networks. We show that an exponential discretization is preferable to a linear discretization since it allows one to achieve the same accuracy when the number of bits is 1 or 2 less. The quality of the neural network VGG-16 is already satisfactory (top5 accuracy 69%) in the case of 3 bit exponential discretization. The ResNet50 neural network shows top5 accuracy 84% at 4 bits. Other neural networks perform fairly well at 5 bits (top5 accuracies of Xception, Inception-v3, and MobileNet-v2 top5 were 87%, 90%, and 77%, respectively). At less number of bits, the accuracy decreases rapidly.
4.5NEMay 3, 2012
Discretization of a matrix in the problem of quadratic functional binary minimizationBoris Kryzhanovsky, Mikhail Kryzhanovsky, Magomed Malsagov
The capability of discretization of matrix elements in the problem of quadratic functional minimization with linear member built on matrix in N-dimensional configuration space with discrete coordinates is researched. It is shown, that optimal procedure of replacement matrix elements by the integer quantities with the limited number of gradations exist, and the efficient of minimization does not reduce. Parameter depends on matrix properties, which allows estimate the capability of using described procedure for given type of matrix, is found. Computational complexities of algorithm and RAM requirements are reduced by 16 times, correct using of integer elements allows increase minimization algorithm speed by the orders.