Hello, I just recently finished a Msc in Statistics at The University of Toronto,
where I worked with
David Duvenaud.
Before that I was Graduate student in the Atmospheric Physics group under the supervision of
Dylan Jones.
I am widely interested in statistics and machine learning.
Some current things i'm thinking about include:

● principled priors for bayesian neural networks

● limits of mean field variational inference

● hypernetworks and their bayesian variants

● bayesian optimization for the large scale exploration of chemical space

● generative models of graphs for automatic chemical design

● Bayesian approaches to fair machine learning

If you're interested in any of these and want to work together please send me an email at
danielfs@utstat.toronto.edu. You can also check out my resume if you want to.

In this work we develop a simple method to construct priors for Bayesian neural networks that incorporates meaningful prior information about functions. We fit neural nets to samples of functions using a hypernetwork, in order to use the empirical moments of the learned weights for our prior parameters. This method allows us to characterize the relationship between weight space and function space.

Daniel Flam-Shepherd, James Requeima, David Duvenaud

Neural processes define a class of neural latent variable models. We extend this class to an infinite dimensional space by imposing a stick-breaking prior on the latent space. Using Stochastic Gradient Variational Bayes, we perform posterior inference for the weights of the stick-breaking process and develop the stick-breaking neural process (SB-NP). SB-NPs are able to learn the dimensionality of the latent space and have improved posterior uncertainty.

Daniel Flam-Shepherd, Yuxiang Gao, Zhaoyu Guo

What defines a reasonable prior to use in Bayesian models? Often, normal distributions are placed over the weights for convenience and are interpreted as a bias toward less complex functions via smaller weights. Gaussian processes, on the other hand, have a elegant mechanism for incorporating prior beliefs about the underlying function - specifying the mean and covariance functions. In this work, we present an approach to specify a more principled prior for Bayesian Neural Networks that can leverage the well studied kernel design techniques from Gaussian process regression.

Daniel Flam-Shepherd, James Requeima, David Duvenaud

The back-propagation algorithm is one of the main tools for credit assignment in neural networks where the loss gradient is computed to back-propagate error from the output layer to the hidden layers. A method called feedback alignment performs almost as well and is more biologically plausible since it avoids using the weights from the forward pass in the the backwards pass by replacing them with random feedback weights. In this work, a general feedback alignment strategy for training neural networks is proposed and experimented with in an supervised and unsupervised setting

During my M SC. of Physics I conducted research in Atmospheric Physics. Specifically, I used bayesian neural networks to learn a functional relationship between the planetary boundary layer depth and physical quantities commonly measured at weather stations such as temperature, wind speeds, surface pressure speciﬁc humidity and solar intensity.