If you have an existing W&B project, it’s easy to start using Sweeps for hyperparameter optimization. I’ll work through the steps with a working example— you can open my W&B Dashboard and example code.
I open my project page. Here are a couple of runs I’ve done already.
I open the sweep tab and click “Create sweep” in the upper right corner.
These steps take me through running my first sweep. To make sure I have the latest version of wandb I run pip install --upgrade wandb first.
The auto-generated config guesses values to sweep over based on the runs I’ve done already. In the “Parameters” tab, I remove channels_one and channels_two from my sweep config. I don’t want to sweep over those hyperparameters. Once I’m happy with the ranges of parameters to sweep over, I download the file.
I move the generated config file to my training script repo.
I run wandb sweep sweep.yaml to start a sweep on the W&B server. This is a centralized service sends out the next set of hyperparameters to agents that I run on my own machines.
Next I launch an agent locally. I can launch dozens of agents in parallel if I want to distribute the work and finish the sweep more quickly. The agent will print out the set of parameters it’s trying next.
That’s it! Now I’m running a sweep. Here’s what my dashboard looks like as the sweep begins to explore the space of hyperparameters.
Enter your email to get updates about new features and blog posts.
Thanks! We look forward to sharing news with you.
Oops! Please refresh the page and try again.
Weights & Biases
We're building lightweight, flexible experiment tracking tools for deep learning. Add a couple of lines to your python script, and we'll keep track of your hyperparameters and output metrics, making it easy to compare runs and see the whole history of your progress. Think of us like GitHub for deep learning.
We are building our library of deep learning articles, and we're delighted to feature the work of community members. Contact Lavanya to learn about opportunities to share your research and insights.