Learn how you might be bottlenecking your training because of the dataset

Bad data practices WILL slow down your training (Photo credit: )

If you’re using machine learning or deep learning then you’ve likely obsessed over making sure all your code can run on GPUs or, for the brave souls, even TPUs.

I hate to be the bearer of bad news, but your models are already likely pretty optimal for GPUs! (especially if…

Iterate your way from baseline to custom models to ship products faster or to publish your research faster.

With Flash build PyTorch baselines in minutes

Whether you’re a data scientist, research engineer, AI researcher, or machine learning engineer, baselines are non-negotiable. Don’t build a fancy GAN or try a complex idea before setting up a good foundation.

In this tutorial, we’ll use to build two baselines in minutes. After that, we’ll iterate on…

Sharded is a new technique that helps you save over 60% memory and train models twice as large.

Giving it scale (Photo by on )

Deep learning models have been shown to improve with more data and more parameters. Even with the latest GPT-3 model from Open AI which uses 175B parameters, we have yet to see models plateau as the number of parameters grow.

For some domains like NLP, the workhorse model has been…

Hands-on Tutorials

This tutorial implements a variational autoencoder for non-black and white images using PyTorch.

Generated images from cifar-10 (author’s own)

It’s likely that you’ve searched for VAE tutorials but have come away empty-handed. Either the tutorial uses MNIST instead of color images or the concepts are conflated and not explained clearly.

You’re in luck!

This tutorial covers all aspects of VAEs including the matching math and implementation on a realistic…

Opensource is the key to advancing AI and has been the driver of the majority of innovation in the field. This is the story from an insider’s perspective.

Credit: (with permission)

As an the question I hear the most is “why would you want to give that away for free.?”

In the field of AI, there are many reasons why opensource is key. First, the code for building models does not give away any competitive advantage because the value…

In this tutorial, I’ll show you how to use gradient ascent to figure out how to misclassify an input.

Using gradient ascent to figure out how to change an input to be classified as a 5. (All images are the author’s own with all rights reserved).

Neural networks get a bad reputation for being black boxes. And while it certainly takes creativity to understand their decision making, they are really not as opaque as people would have you believe.

In this tutorial, I’ll show you how to use backpropagation to change the input as to classify…

In a new paper, we discuss the key ideas driving performance in self-supervised learning and show what matters.

Contrastive learning: Batch of inputs.

This is the partner blog matching our new paper: (by William Falcon and Kyunghyun Cho).

In the last year, a stream of “novelself-supervised learning algorithms have set new state-of-the-art results in AI research: AMDIM, CPC, SimCLR, BYOL, Swav…

William Falcon

⚡️PyTorch Lightning Creator • PhD Student, AI (NYU, Facebook AI research).

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store