Luiz Angelo Steffenel

Lilian Hollard, Lucas Mohimont, Nathalie Gaveau et al.

The paper investigates the performance of state-of-the-art low-parameter deep neural networks for computer vision, focusing on bottleneck architectures and their behavior using superlinear activation functions. We address interference in feature maps, a phenomenon associated with superposition, where neurons simultaneously encode multiple characteristics. Our research suggests that limiting interference can enhance scaling and accuracy in very low-scaled networks (under 1.5M parameters). We identify key design elements that reduce interference by examining various bottleneck architectures, leading to a more efficient neural network. Consequently, we propose a proof-of-concept architecture named NoDepth Bottleneck built on mechanistic insights from our experiments, demonstrating robust scaling accuracy on the ImageNet dataset. These findings contribute to more efficient and scalable neural networks for the low-parameter range and advance the understanding of bottlenecks in computer vision. https://caiac.pubpub.org/pub/3dh6rsel

Lilian Hollard, Lucas Mohimont, Nathalie Gaveau et al.

Efficient computation in deep neural networks is crucial for real-time object detection. However, recent advancements primarily result from improved high-performing hardware rather than improving parameters and FLOP efficiency. This is especially evident in the latest YOLO architectures, where speed is prioritized over lightweight design. As a result, object detection models optimized for low-resource environments like microcontrollers have received less attention. For devices with limited computing power, existing solutions primarily rely on SSDLite or combinations of low-parameter classifiers, creating a noticeable gap between YOLO-like architectures and truly efficient lightweight detectors. This raises a key question: Can a model optimized for parameter and FLOP efficiency achieve accuracy levels comparable to mainstream YOLO models? To address this, we introduce two key contributions to object detection models using MSCOCO as a base validation set. First, we propose LeNeck, a general-purpose detection framework that maintains inference speed comparable to SSDLite while significantly improving accuracy and reducing parameter count. Second, we present LeYOLO, an efficient object detection model designed to enhance computational efficiency in YOLO-based architectures. LeYOLO effectively bridges the gap between SSDLite-based detectors and YOLO models, offering high accuracy in a model as compact as MobileNets. Both contributions are particularly well-suited for mobile, embedded, and ultra-low-power devices, including microcontrollers, where computational efficiency is critical.