Inductive bias from layerwise structure and backpropagation in neural networks: analysis through simplified models and empirical frameworks

Thesis

Deep Neural Networks (DNNs) are revolutionizing industries, pushing the boundaries of image and language processing. Despite their remarkable performance, we still have little understanding of how they achieve it. Unlocking their potential and ensuring their safe use requires exploring their inductive bias — the preferences that guide models in selecting solutions from countless possibilities.

Another mystery lies in their dynamics. Trained via gradient descent, DNNs often find gen...

Expand abstract

Authors: Nam, Y

• DOI: 10.5287/ora-jvp7o4pkr
• Type of award: DPhil
• Level of award: Doctoral
• Awarding institution: University of Oxford
• Language: English
• Keywords: machine learning; dynamics; inductive bias; neural networks; feature learning; grokking
• Subjects: Deep learning (Machine learning)