Real-time Multi Task Network based Object Detection, Semantic Segmentation and Soiling Detection for Automated Driving English Free

  • Category Technical paper
  • Related event International Congress : SIA VISION Digital 2021 - 17 & 18 MARCH 2021
  • Edition SIA
  • Date 03/29/2021
  • Author F. Dellinger, T. Boulay, D. Mendoza Barrenechea, S. El-Hachimi, I. Leang, F. Bürger | Valeo Bobigny, France
  • Language English
  • Type PDF file (2.32 Mo)
    (Downloadable immediately on receipt of online payment)
  • Number of pages 10
  • Code R-2021-11-0604
  • Fee Free

Camera-based Deep Learning algorithms are increasingly needed for perception in Automated Driving systems.
However, constraints from the automotive industry challenge the deployment of CNNs by imposing embedded systems with limited computational resources. In this paper, we propose an approach to embed a multi-task CNN network under such conditions on a commercial prototype platform, i.e. a low power System on Chip (SoC) processing four surround-view fisheye cameras at 10 FPS.
The first focus is on designing an efficient and compact multi-task network architecture. To ensure robustness against adverse weather, we integrate a soiling detection task in addition to the more common perception tasks that are object detection and semantic segmentation.
Secondly, a pruning method is applied to compress the CNN, helping to reduce the runtime and memory usage by a factor of 2 without lowering the performances significantly. Finally, several embedded optimization techniques such as mixed-quantization format usage and efficient data transfers between different memory areas are proposed to ensure real-time execution and avoid bandwidth bottlenecks. The approach is evaluated on the hardware platform, considering embedded detection performances, runtime and memory bandwidth. Unlike most works from the literature that focus on classification
task, we aim here to study the effect of pruning and quantization on a compact multi-task network with
object detection, semantic segmentation and soiling detection tasks.