---
name: ml-experiment-c100340ce58c
description: "Design and run machine learning experiments with proper evaluation using jupyter_execute, including training, benchmarking, and ablation studies. Use when the user wants to train models, compare algorithms, run ablation studies, evaluate ML performance, or reproduce paper results."
---

# ML Experiment Skill

## Description
Design, implement, and evaluate machine learning experiments with reproducible workflows, proper baselines, and statistical analysis.

## Tools Used
- `jupyter_execute` - Execute ML code in Python (auto-switches to Jupyter)
- `jupyter_notebook` - Manage experiment notebooks
- `update_notebook` - Set up experiment cells
- `update_latex` - Write experiment results to papers
- `latex_compile` - Compile CS conference papers (auto-switches to LaTeX)
- `arxiv_to_prompt` - Read related work from arXiv papers
- `update_notes` - Write experiment logs and analysis summaries

## Capabilities

### Experiment Design
- Proper train/validation/test splits
- Cross-validation and bootstrap confidence intervals
- Ablation study design
- Hyperparameter search (grid, random, Bayesian)

### Implementation
- PyTorch and TensorFlow model building
- Data loading and augmentation pipelines
- Training loops with logging and checkpointing
- Distributed training setup

### Evaluation
- Standard metrics per task (accuracy, F1, BLEU, mAP, etc.)
- Statistical significance testing (paired t-test, bootstrap)
- Comparison with baselines
- Error analysis and visualization

## Usage Patterns

### Run an Experiment
When user says: "Train a model for [task]"
1. Clarify dataset, metrics, and baselines
2. Implement data loading and preprocessing
3. Build model architecture
4. Train with proper logging
5. Evaluate and compare to baselines
6. Report results with confidence intervals

### Reproduce a Paper
When user says: "Reproduce [paper title/arXiv ID]"
1. Fetch paper using arxiv_to_prompt
2. Extract key method details
3. Implement core algorithm
4. Run experiments matching paper setup
5. Compare results to reported numbers

## Tool Examples

### Train and evaluate a classifier
```python
# via jupyter_execute
import torch
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
# ... train model ...
print(classification_report(y_test, predictions))
```

### Run ablation study
```python
# via jupyter_execute
configs = [
    {"name": "full", "use_augmentation": True, "use_dropout": True},
    {"name": "no_aug", "use_augmentation": False, "use_dropout": True},
    {"name": "no_dropout", "use_augmentation": True, "use_dropout": False},
]
results = {c["name"]: train_and_eval(**c) for c in configs}
```

### Validation checkpoints
- Verify data shapes match expected dimensions before training
- Check that loss is decreasing after the first few epochs
- Confirm test set has no overlap with training data


---
Source attribution: Prismer-AI
Original project: https://github.com/Prismer-AI/Prismer
License: MIT
License information: https://opensource.org/license/mit
Original upstream instructions; downloaded from Mega AI Hub.

## Upstream license notice

MIT License

Copyright (c) 2026 Gabriel Moreira

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.



# Mega AI Hub resource metadata

Creator / publisher: Prismer-AI
Original project: https://github.com/Prismer-AI/Prismer
Attribution basis: Upstream repository explicitly recorded in package mirror path

Fields of work: Data & Analytics, Development & Engineering
Roles: Data Analyst, Data Scientist, ML / AI Engineer
Specialties: API
Tools: Tool not specified
Country applicability: Country not specified
Classification basis: Source catalog labels
Instruction SHA-256: d5de712a999edc02ab9db27715a08d5ec1d9c7560235348fd26c5b49354056b1