I work as a post-doctoral scientist within the IMPACT project, for "Intelligent Machine Perception Project", at the České vysoké učení technické v Praze, or Czech Technical University in Prague, in English.
My works concern the construction and justification of stochastic optimisation algorithms applied to deep learning models.
As part of my contract, I often carry on scientific trips to INRIA Paris.
Contact : pierre-yves.masse [at] lri.fr
With Yann Ollivier. HAL version, and arxiv version. More details about this work may be found here.
We prove the convergence of several online learning algorithms applied to recurrent systems, such as recurrent neural networks. These include the real time recurrent learning (RTRL) algorithm, truncated backpropagation through time (TBPTT) on well-chosen time intervals, as well as the low-rank, probabilistic approximations of RTRL No Back Track and UORO. We also prove convergence of adaptive algorithms such as Adam. Since we prove convergence from a general set of assumptions applied to some abstract model of a learning algorithm, our framework can cover other variants of the algorithms considered.
2015. "Speed learning on the fly", preprint.
With Yann Ollivier. arxiv version.
We describe an algorithm which adapts online the step size of a gradient descent. The algorithm was constructed by Yann Ollivier. We carry on experiments on synthetic models.
2014. "Adaptive confidence bands in the nonparametric fixed design regression model", Journal of Nonparametric Statistics.
With William Meiniel. Version on the JNPS website, and version on arxiv.
We prove that adaptive confidence bands exist in the nonparametric fixed design regression model. In the course of the proof, we show that sup-norm adaptive estimators exist as well in regression.
2017. "About the Use of Gradients in Machine Learning", Phd. thesis.
Manuscript, in French.
The manuscript presents proofs of convergence for the "Real Time Recurrent Learning" and "No Back Track" algorithms. It also contains the "Speed learning on the fly" article.
2018- Post-Doctorate. IMPACT Project, České vysoké učení technické v Praze, Prague.
2014-2017. Phd in Mathematics and Computer Science, under the supervision of Yann Ollivier. Laboratoire de Recherche en Informatique, Université Paris-Sud, Orsay.
2012-2013. Master Probabilités et Modèles Aléatoires ("Probabilities and Random Models"), Université Pierre et Marie Curie, Paris.
2010-2014. Studies. Mathematics Department, École Nationale Supérieure de Cachan (ENS Cachan), Cachan.