Conference item
On the impact of the activation function on deep neural networks training
- Abstract:
-
The weight initialization and the activation function of deep neural networks have a crucial impact on the performance of the training procedure. An inappropriate selection can lead to the loss of information of the input during forward propagation and the exponential vanishing/exploding of gradients during back-propagation. Understanding the theoretical properties of untrained random networks is key to identifying which deep networks may be trained successfully as recently demonstrated by Sa...
Expand abstract
- Publication status:
- Published
- Peer review status:
- Peer reviewed
Actions
Authors
Funding
+ Engineering and Physical Sciences Research Council
More from this funder
Grant:
EP/R013616/1
56726
Bibliographic Details
- Publisher:
- Journal of Machine Learning Research Publisher's website
- Host title:
- Proceedings of Machine Learning Research
- Journal:
- Proceedings of Machine Learning Research Journal website
- Publication date:
- 2019-06-12
- Acceptance date:
- 2019-04-30
- ISSN:
-
2640-3498
Item Description
- Pubs id:
-
pubs:1043248
- UUID:
-
uuid:1e9a7c39-7519-4796-9784-537b14ba1941
- Local pid:
- pubs:1043248
- Source identifiers:
-
1043248
- Deposit date:
- 2019-08-13
Terms of use
- Copyright holder:
- Hayou, S et al
- Copyright date:
- 2019
- Notes:
- © The Author(s) 2019. Conference paper presented at the 36th International Conference on Machine Learning (ICML 2019), Long Beach, California, USA, June 2019. The final published version and supplementary materials are available online from Proceedings of Machine Learning Research at: http://proceedings.mlr.press/v97/hayou19a.html
Metrics
If you are the owner of this record, you can report an update to it here: Report update to this record