Soufiane Hayou

I am a Peng Tsu Ann Assistant Professor of Mathematics at the National University of Singapore. This is a joint position between the Department of Mathematics and Institute for Mathematical Sciences. Previously, I obtained my PhD in Statistics & Machine learning at the University of Oxford where I was advised by Arnaud Doucet, and Judith Rousseau. I am also a member of the OxCSML Group. Before starting my PhD, I graduated from Ecole Polytechnique in France with a major in Applied Mathematics.

My Research Interests include:

• Foundations of Deep Learning
• Bayesian Neural Networks
• Uncertainty in Deep Learning
• Model Compression
• Stochastic Processes

I am also interested in the applications of Machine Learning in Healthcare and Finance.

Education

• Doctor of Philosophy - PhD University of Oxford (2017.10 — 2021.01)
• Engineer's degree & MSc in Applied Mathematics Ecole polytechnique (2013 — 2017)
• Classes Préparatoires Classes préparatoires Moulay Youssef (2011 — 2013)

Companies

• PTA Assistant Professor of Mathematics National University of Singapore (2021)
• PhD candidate in Statistics/Machine Learning University of Oxford (2017 — 2021)
• Quantitative Research Bloomberg LP (2017 — 2017)
• Quant Research JPMorgan Chase & Co. (2016 — 2016)
• FX Analyst (Summer Intern) Kantox (2015 — 2015)
• Data Analyst (Part-time Intern) Snecma (2014 — 2015)
• Teaching Assistant Académie de Toulouse (2013 — 2014)

Full list of Publications and Preprints

♦ The Equilibrium Hypothesis: Rethinking implicit regularization in Deep Neural Networks (2021, Preprint). Yizhang Lou, Chris Mingard, Soufiane Hayou Link

♦ Regularization in ResNet with Stochastic Depth. NeurIPS 2021. Soufiane Hayou, Fadhel Ayed. Link

♦ The Curse of Depth in Kernel Regime. Neurips 2021 ICBINB workshop (Spotlight). To appear in Proceedings of Machine Learning Research. Soufiane Hayou, Arnaud Doucet, Judith Rousseau

♦ Probabilistic fine-tuning of pruning masks and PAC-Bayes self-bounded learning (2021, Preprint). Soufiane Hayou, Bobby He, Gintare Karolina Dziugaite. Link.

♦ Stochastic Pruning: Fine-Tuning, and PAC-Bayes bound optimization. NeurIPS 2021 Bayesian Deep Learning Workshop. Soufiane Hayou, Bobby He, Gintare Karolina Dziugaite.

♦ Robust Pruning at Initialization. ICLR 2021. Soufiane Hayou, Jean-Francois Ton, Arnaud Doucet, Yee Whye Teh. Link.

♦ Stable ResNet. AISTATS 2021 (Oral presentation).Soufiane Hayou, Eugenio Clerico, Bobby He, George Deligiannidis, Arnaud Doucet, Judith Rousseau. Link.

♦ On the Impact of the Activation function on Deep Neural Networks Training. ICML 2019. Soufiane Hayou, Arnaud Doucet, Judith Rousseau. Link.

♦ Mean-field Behaviour of Neural Tangent Kernel for Deep Neural Networks (2020, submitted). Soufiane Hayou, Arnaud Doucet, Judith Rousseau. Link.

♦ On the Overestimation of the Largest Eigenvalue of a Covariance Matrix, 2017 (Bloomberg). Soufiane Hayou

♦ Cleaning the Correlation Matrix with a Denoising AutoEncoder, 2017 (Bloomberg). Soufiane Hayou