DNNs May Determine Major Properties of Their Outputs Early, with Timing Possibly Driven by Bias
This provides a new perspective on bias mitigation and efficient inference for machine learning researchers, though it appears incremental as it builds on existing understanding of DNN behavior.
The paper argues that deep neural networks (DNNs) often determine their outputs early in inference, influenced by model biases, drawing parallels to human intuitive decision-making, and uses diffusion models as a case study to demonstrate this effect.
This paper argues that deep neural networks (DNNs) mostly determine their outputs during the early stages of inference, where biases inherent in the model play a crucial role in shaping this process. We draw a parallel between this phenomenon and human decision-making, which often relies on fast, intuitive heuristics. Using diffusion models (DMs) as a case study, we demonstrate that DNNs often make early-stage decision-making influenced by the type and extent of bias in their design and training. Our findings offer a new perspective on bias mitigation, efficient inference, and the interpretation of machine learning systems. By identifying the temporal dynamics of decision-making in DNNs, this paper aims to inspire further discussion and research within the machine learning community.