23
Mar
2020
/
Lukas Biewald, CEO of Weights & Biases

Machine Learning Experiment Tracking

Why is experiment tracking so important for doing real world machine learning?

At first glance, building and deploying machine learning models looks a lot like writing code. But there are some key differences that make machine learning harder:

  1. Machine Learning projects have far more branching and experimentation than a typical software project.
  2. Machine Learning code generally doesn’t throw errors, it just underperforms, making debugging extra difficult and time consuming.
  3. A single small change in training data, training code or hyperparameters can wildly change a model’s performance, so reproducing earlier work often requires exactly matching the prior setup.
  4. Running machine learning experiments can be time consuming and just the compute costs can get expensive.

Tracking experiments in an organized way helps with all of these core issues. Weights and Biases (wandb) is a simple tool that helps individuals to track their experiments — I talked to several machine learning leaders of different size teams about how they use wandb to track their experiments.

The essential unit of progress in an ML project is an experiment, so most people track what they’re doing somehow — generally I see practitioners start with a spreadsheet or a text file to keep track of what they’re doing.

Spreadsheets and docs are incredibly flexible — what’s wrong with this approach? See more at Machine Learning Experiment Tracking.


Newsletter

Enter your email to get updates about new features and blog posts.

Weights & Biases

We're building lightweight, flexible experiment tracking tools for deep learning. Add a couple of lines to your python script, and we'll keep track of your hyperparameters and output metrics, making it easy to compare runs and see the whole history of your progress. Think of us like GitHub for deep learning.

Partner Program

We are building our library of deep learning articles, and we're delighted to feature the work of community members. Contact Lavanya to learn about opportunities to share your research and insights.

Try our free tools for experiment tracking →