| Title: |
Repercussions of Using DNN Compilers on Edge GPUs for Real Time and Safety Critical Systems: A Quantitative Audit |
| Authors: |
Shafi, Omais; Pandit, Mohammad Khalid; Saini, Amarjeet; Ananthanarayanan, Gayathri; Sen, Rijurekha |
| Contributors: |
Science and Engineering Research Board |
| Source: |
ACM Journal on Emerging Technologies in Computing Systems ; volume 20, issue 1, page 1-25 ; ISSN 1550-4832 1550-4840 |
| Publisher Information: |
Association for Computing Machinery (ACM) |
| Publication Year: |
2024 |
| Description: |
Rapid advancements in edge devices have led to a large deployment of deep neural network (DNN) based workloads. To utilize the resources at the edge effectively, many DNN compilers are proposed that efficiently map the high level DNN models developed in frameworks like PyTorch, Tensorflow, Caffe, and so on into minimum deployable lightweight execution engines. For real time applications like ADAS, these compiler optimized engines should give precise, reproducible, and predictable inferences, both in-terms of runtime and output consistency. This article is the first effort in empirically auditing state-of-the-art DNN compilers viz TensorRT, AutoTVM, and AutoScheduler. We characterize the NN compilers based on their performance predictability w.r.t inference latency, output reproducibility, hardware utilization, and so on and based on that provide various recommendations. Our methodology and findings can potentially help the application developers, in making informed decision about the choice of DNN compiler, in a real time safety critical setting. |
| Document Type: |
article in journal/newspaper |
| Language: |
English |
| DOI: |
10.1145/3611016 |
| Availability: |
https://doi.org/10.1145/3611016; https://dl.acm.org/doi/10.1145/3611016; https://dl.acm.org/doi/pdf/10.1145/3611016 |
| Rights: |
https://www.acm.org/publications/policies/copyright_policy#Background |
| Accession Number: |
edsbas.EF8C641C |
| Database: |
BASE |