Conference item
Neural response interpretation through the lens of critical pathways
- Abstract:
- Is critical input information encoded in specific sparse pathways within the neural network? In this work, we discuss the problem of identifying these critical pathways and subsequently leverage them for interpreting the network's response to an input. The pruning objective - selecting the smallest group of neurons for which the response remains equivalent to the original network - has been previously proposed for identifying critical pathways. We demonstrate that sparse pathways derived from pruning do not necessarily encode critical input information. To ensure sparse pathways include critical fragments of the encoded input information, we propose pathway selection via neurons' contribution to the response. We proceed to explain how critical pathways can reveal critical input features. We prove that pathways selected via neuron contribution are locally linear (in an `2-ball), a property that we use for proposing a feature attribution method: “pathway gradient”. We validate our interpretation method using mainstream evaluation experiments. The validation of pathway gradient interpretation method further confirms that selected pathways using neuron contributions correspond to critical input features. The code1 2 is publicly available.
- Publication status:
- Published
- Peer review status:
- Peer reviewed
Actions
Access Document
- Files:
-
-
(Preview, Accepted manuscript, pdf, 3.4MB, Terms of use)
-
- Publisher copy:
- 10.1109/CVPR46437.2021.01332
Authors
- Publisher:
- IEEE
- Host title:
- Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2021)
- Pages:
- 13523-13533
- Publication date:
- 2021-11-02
- Event title:
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2021)
- Event location:
- Online
- Event website:
- https://cvpr2021.thecvf.com/
- Event start date:
- 2021-06-19
- Event end date:
- 2021-06-25
- DOI:
- EISSN:
-
2575-7075
- ISSN:
-
1063-6919
- EISBN:
- 978-1-6654-4509-2
- ISBN:
- 978-1-6654-4510-8
- Language:
-
English
- Keywords:
- Pubs id:
-
1237079
- Local pid:
-
pubs:1237079
- Deposit date:
-
2022-07-21
- ARK identifier:
Terms of use
- Copyright holder:
- IEEE
- Copyright date:
- 2021
- Rights statement:
- © IEEE 2021
- Notes:
- This paper was presented at the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2021), 19th-25th June 2021. This is the accepted manuscript version of the article. The final version is available online from IEEE at: https://doi.org/10.1109/CVPR46437.2021.01332
If you are the owner of this record, you can report an update to it here: Report update to this record