Using the Projected Belief Network at High Dimensions
This work addresses scalability issues for researchers using PBNs in domains like audio processing, though it is incremental as it builds on existing PBN frameworks.
The paper tackles the limitations of the Projected Belief Network (PBN) in high-dimensional settings by developing techniques to avoid matrix inversion and dimension-reduction constraints, applying these to classify and auto-encode high-dimensional acoustic spectrograms.
The projected belief network (PBN) is a layered generative network (LGN) with tractable likelihood function, and is based on a feed-forward neural network (FFNN). There are two versions of the PBN: stochastic and deterministic (D-PBN), and each has theoretical advantages over other LGNs. However, implementation of the PBN requires an iterative algorithm that includes the inversion of a symmetric matrix of size M X M in each layer, where M is the layer output dimension. This, and the fact that the network must be always dimension-reducing in each layer, can limit the types of problems where the PBN can be applied. In this paper, we describe techniques to avoid or mitigate these restrictions and use the PBN effectively at high dimension. We apply the discriminatively aligned PBN (PBN-DA) to classifying and auto-encoding high-dimensional spectrograms of acoustic events. We also present the discriminatively aligned D-PBN for the first time.