--- name: weights-biases-a7f94d943379 description: Expert Weights & Biases (W&B) assistance covering experiment tracking, artifact versioning, hyperparameter sweeps, and model monitoring. Use when tracking deep learning experiments and collaborating on ML workflows. --- # Weights & Biases (W&B) Weights & Biases (W&B) provides MLOps infrastructure for tracking experiments, versioning datasets and models, evaluating LLM applications, and orchestrating distributed training runs. ## When to Use - **Enterprise ML Experiment Tracking**: Tracking hyperparameters, training loss curves, GPU utilization, and system metrics. - **Hyperparameter Tuning with W&B Sweeps**: Automated Bayesian, grid, and random sweeps across distributed compute nodes. - **Model & Dataset Versioning (Artifacts)**: Storing, versioning, and tracking data lineage for datasets and model checkpoints. - **Production Model Evaluation & LLM Tracing**: Evaluating LLM prompts and agentic tool calls with Weave / W&B Prompts. ## Quick Start ```python import wandb # Initialize experiment run wandb.init(project="transformer-pretraining", config={"lr": 0.001, "batch_size": 32}) for epoch in range(10): # Simulated training step train_loss = 0.5 / (epoch + 1) val_acc = 0.75 + (epoch * 0.02) wandb.log({"epoch": epoch, "loss": train_loss, "val_accuracy": val_acc}) wandb.finish() ``` ## Core Concepts ### Experiment Tracking with wandb.init & wandb.log Logging metrics, hyperparameters, and console output: ```python import wandb import numpy as np # Initialize W&B run run = wandb.init( project="transformer-pretraining", name="llama-3.2-finetune-v1", config={ "learning_rate": 2e-5, "batch_size": 32, "epochs": 5, "optimizer": "AdamW", "weight_decay": 0.01, "architecture": "Llama-3.2-3B" } ) # Access hyperparameter config config = wandb.config # Training loop simulation for epoch in range(config.epochs): train_loss = 2.5 / (epoch + 1) + np.random.normal(0, 0.05) val_loss = 2.7 / (epoch + 1) + np.random.normal(0, 0.05) val_accuracy = 0.6 + 0.07 * epoch # Log metrics to dashboard in real-time wandb.log({ "epoch": epoch, "train/loss": train_loss, "val/loss": val_loss, "val/accuracy": val_accuracy, }) wandb.finish() ``` ### Dataset & Model Checkpoint Versioning with Artifacts Tracking data lineage and model weights: ```python import wandb run = wandb.init(project="model-registry-pipeline", job_type="train") # Log model artifact with versioning model_artifact = wandb.Artifact( name="fraud_detection_model", type="model", description="Fine-tuned XGBoost model with calibrated probabilities", metadata={"roc_auc": 0.942, "framework": "xgboost"} ) model_artifact.add_file("model_weights.json") run.log_artifact(model_artifact) # In deployment pipeline: consume specific artifact version deployed_artifact = run.use_artifact("fraud_detection_model:latest") artifact_dir = deployed_artifact.download() print(f"Downloaded model to {artifact_dir}") run.finish() ``` ### Automated Hyperparameter Sweeps with wandb.sweep Running Bayesian optimization across parameters: ```python import wandb sweep_config = { "method": "bayes", "metric": {"name": "val/loss", "goal": "minimize"}, "parameters": { "learning_rate": {"min": 1e-5, "max": 1e-2}, "batch_size": {"values": [16, 32, 64]}, "dropout": {"uniform": [0.1, 0.5]} } } sweep_id = wandb.sweep(sweep_config, project="hyperparameter-sweeps") def train(): with wandb.init() as run: config = wandb.config loss = (config.learning_rate - 1e-3)**2 + config.dropout * 0.1 wandb.log({"val/loss": loss}) # Run agent across 5 trials # wandb.agent(sweep_id, function=train, count=5) ``` ## Common Patterns ### Dataset and Model Checkpoint Versioning via Artifacts **Problem**: Untracked dataset updates causing silent regressions across model training runs. **Solution**: Log and consume versioned W&B Artifacts: ```python run = wandb.init(project="nlp-classifier") # Log versioned training dataset artifact = wandb.Artifact("customer_reviews", type="dataset") artifact.add_file("data/reviews_v2.parquet") run.log_artifact(artifact) # In training run: link model artifact directly to input dataset lineage ``` ## Best Practices **Do**: - Always set `config=...` in `wandb.init()` to record all training hyperparameters for full experiment reproducibility. - Call `wandb.finish()` at the end of runs to ensure logs, checkpoints, and system metrics finish syncing. - Use W&B Artifacts to track datasets and models, establishing end-to-end data lineage and auditability. - Group related runs using `group="experiment_group_name"` for cleaner team dashboards. **Don't**: - Call `wandb.log()` in tight microsecond inner loops; log aggregated metrics periodically per step or epoch. - Hardcode your API key; configure via environment variable `WANDB_API_KEY`. - Log sensitive credentials or unmasked PII data to W&B dashboards. ## Troubleshooting | Error | Cause | Solution | | :----------------------------------------------------------- | :----------------------------------------------------------------------- | :----------------------------------------------------------------------------- | | `wandb.errors.CommError: It appears your API key is invalid` | Missing or invalid `WANDB_API_KEY`. | Run `wandb login` or export `WANDB_API_KEY`. | | `Network timeout during artifact upload` | Uploading massive multi-gigabyte checkpoint without multipart upload. | Verify stable internet connection or save model checkpoint reference pointers. | | `Multiple runs logging to the same dashboard line` | Forgetting to call `wandb.finish()` between consecutive loop iterations. | Ensure `wandb.finish()` is called inside a `try..finally` block. | ## References - [Weights & Biases](https://wandb.ai/) --- Source attribution: G1Joshi Original project: https://github.com/G1Joshi/Agent-Skills/blob/6a74cf03a8a86dc7d993ead3091b41123dbd0181/skills/ai-ml/weights-biases/SKILL.md License: MIT License information: https://opensource.org/license/mit Original upstream instructions; downloaded from Mega AI Hub. ## Upstream license notice MIT License Copyright (c) 2026 G1Joshi Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. # Mega AI Hub resource metadata Creator / publisher: G1Joshi Original project: https://github.com/G1Joshi/Agent-Skills/blob/6a74cf03a8a86dc7d993ead3091b41123dbd0181/skills/ai-ml/weights-biases/SKILL.md Attribution basis: Source repository record Fields of work: Data & Analytics, Development & Engineering Roles: Data Analyst, Data Scientist, ML / AI Engineer Specialties: API, Cross Platform Tools: Weights Biases Country applicability: Country not specified Classification basis: Source catalog labels Instruction SHA-256: 41eb08621b3bb6143a37fccb367e1246d66a92c7d9975310794c5f8d3c242db1