Analysis of Branch Specialization and its Application in Image Decomposition
This provides a methodological analysis for researchers in computer vision and neural networks, though it is incremental as it builds on known qualitative observations.
The paper tackled the problem of understanding Branch Specialization in branched neural networks, showing that branched generative networks can decompose animal images into meaningful channels like fur and whiskers, and face images into components like illumination and face parts.
Branched neural networks have been used extensively for a variety of tasks. Branches are sub-parts of the model that perform independent processing followed by aggregation. It is known that this setting induces a phenomenon called Branch Specialization, where different branches become experts in different sub-tasks. Such observations were qualitative by nature. In this work, we present a methodological analysis of Branch Specialization. We explain the role of gradient descent in this phenomenon. We show that branched generative networks naturally decompose animal images to meaningful channels of fur, whiskers and spots and face images to channels such as different illumination components and face parts.