Dr. Théo Galy-Fajou

Bio

Théo Galy-Fajou is a doctor in Machine Learning specialized in Bayesian inference methods. He is particularly interested in approximate Bayesian inference methods like sampling, applications to Gaussian processes and online learning. Throughout his Ph.D., Théo developed numerous open source projects in Julia and dived deep into software development. He is currently working for PlantingSpace as a Julia software developer and Bayesian researcher.

You can find me on Github, Mastodon , ResearchGate and Twitter.

News

I finally defended my thesis on the 07th of July! I will upload the final version of the thesis soon.
I joined the PlantingSpace team in April as Julia software developer and Bayesian researcher!
A new paper got accepted at the Entropy journal! Check out "Flexible and Efficient Inference with Particles for the Variational Gaussian Approximation" with Valerio Perrone and Manfred Opper.
I gave a short poster presentation on my work on Turkie.jl for JuliaCon 2021, you can find the video here.
I gave a talk at the Cambridge Machine Learning Reading Group and it's been recorded, check it out on here on Youtube
Two papers were accepted for the AABI Symposium: Gaussian Density Parametrization Flow: Particle and Stochastic Approaches with Valerio Perrone and Manfred Opper and Evidence Estimation by Kullback-Leibler Integration for Flow-Based Methods with Nikolai Zaki and Manfred Opper.
JuliaCon 2020 is happening this week and here are the two talks I have given about my ongoing work on KernelFunctions.jl (presentation here) and AugmentedGaussianProcesses.jl (presentation here).
I started a new blog called "Bayesian Tribulations", it's very experimental at the moment but if you want have a look! I used Franklin.jl to generate it!
My paper "Adaptive Inducing Point Selection for Gaussian Processes" has been accepted for the Continual Learning Workshop of ICML 2020. You can check this introduction video for more info. Note that this is still a work in progress!
Our paper "Automated Augmented Conjugate Inference for Non-conjugate Gaussian Process Models" got accepted at the AISTATS 2020 conference, check it out! Here is a short explaining Twitter thread.
Our paper "Multi-Class Gaussian Process Classification Made Conjugate: Efficient Inference via Data Augmentation" got accepted for the UAI conference 2019, see you in Tel Aviv
Recently gave a short talk at Berlin Julia Meetup (join!), here are the slides and the notebook is here
Our paper "Efficient Gaussian Process Classification Using Polya-Gamma Data Augmentation" got accepted at the AAAI conference 2019!
We have a journal club to speak about Bayesian inference and Machine Learning papers. You can find more informations here

Publications

Flexible and Efficient Inference with Particles for the Variational Gaussian Approximation

Entropy Journal 21' T. Galy-Fajou, V. Perrone, M. Opper

We solve the Gaussian Variational Approximation cubic complexity by defining a deterministic flow. It allows to use low-rank approximation (without diagonal) and obtain exact guarantees. Additionally, we show that using particles helps getting a more stable convergence and a higher accuracy.

Paper:

Code:

Adaptive Inducing Points Selection For Gaussian Processes

Continual Learning Workshop ICML 20' T. Galy-Fajou, M. Opper

We propose a new way to select inducing points locations both in an online and offline setting. The algorithm is fully data-centric but takes fully into account the sparse Gaussian process framework.

Paper:

Code:

Video:

Automated Augmented Conjugate Inference for Non-conjugate Gaussian Process Models

AISTATS 20' T. Galy-Fajou, F. Wenzel, M. Opper

We generalize the theory around latent variable augmentation for Gaussian models. From the Schoenberg theorem, we identify a new class of functions that can be augmented and provide methods to perform inference out of the box.

Paper:

Code:

Multi-Class Gaussian Process Classification Made Conjugate: Efficient Inference via Data Augmentation

UAI 19' T. Galy-Fajou, F. Wenzel, C. Donner, M. Opper

Using new augmentation methods, we derive an augmented likelihood for the logistic-softmax link. It allows us to infer categorical data in only a few iterations and obtain uncertainty estimates for each of the class.

Paper:

Code:

Video:

Efficient Gaussian Process Classification Using Polya-Gamma Data Augmentation

AAAI 19' F. Wenzel* T. Galy-Fajou*, C. Donner, M. Kloft, M. Opper

By applying the Pòlya-Gamma augmentation to Gaussian processes and sparse Gaussian Processes, we scale inference to billion of inducing points for the binary classification problem.

Paper:

Bayesian Nonlinear Support Vector Machines for Big Data

ECML 17' F. Wenzel, T. Galy-Fajou, M. Deutsch, M. Kloft

The Bayesian SVM is a probabilistic interpretation of the hinge loss. By applying an augmentation on it we are able to scale Gaussian processes using this likelihood to large datasets and converge extremely quickly.

Paper:

Code:

Open Source Projects

AugmentedGaussianProcesses.jl, package for working with Gaussian Processes and the augmentation developed in all the publications.
AugmentedGPLikelihoods.jl, package interfacing the work in AugmentedGaussianProcesses.jl and the JuliaGP ecosystem.
KernelFunctions.jl, kernel functions for machine learning in Julia.
AbstractGPs.jl, basic implementation for Gaussian Processes in Julia with an emphasis on probabilistic programming.
Turkie.jl, live visualization of sampling statistics with Turing.jl and Makie.jl.
BayesianQuadrature.jl, different Bayesian quadrature methods.
ThermodynamicIntegration.jl, thermodynamic integration with support for Turing models.

Education

2022 : PhD in Machine Learning under the supervision of Professor Manfred Opper (pdf to be uploaded soon) Technische Universität Berlin, Berlin, Germany.
2015 : MsC of Physics EPFL (École Polytechnique Fédérale de Lausanne) master thesis (pdf here) at University of Edinburgh/CERN with Minor in Computational Science and Engineering.
2012 : BsC of Physics EPFL (École Polytechnique Fédérale de Lausanne), Lausanne, Switzerland
2009 : Baccalauréat Lycée Vincent Auriol, Revel, France

Work experience

2017-2022: Research Assistant @ Technische Universität Berlin
2016-2017 : Research intern @ Humboldt-Universität zu Berlin
2015-2016 : C++ Software developper in MRI development @ Siemens Healthcare in Lausanne
2012-2015 : Teaching assistant for Particle Physics, General Physics @ University of Edinburgh & EPFL
2015 : Internship in Particle Physics @ Lancaster University

You can find a more complete CV here