aim-experiment-guide
安装方式
先用 Prompt 核对来源,也可切换到安装命令或下载可用的安装包。
用 AI 助手安装
复制提示词,让助手先核对固定版本和依赖。
查看 Prompt收起 Prompt
先检查当前 Agent 是否已安装 SkillPI Skill Finder。如果尚未安装,请按官方仓库的说明安装并启用:https://github.com/Findskills/skillpi。然后使用 Skill Finder 打开此 SkillPI 详情页并定位 "aim-experiment-guide":https://skillpi.com/zh/skills/deepskill-511a909fa3d9603c0d150d3c149d9531c4dfce7cc94aeedfbdb856ebae761666。安装前核对上游来源和版本:https://github.com/brycewang-stanford/Auto-Empirical-Research-Skills/blob/bfa0a6e5a843babbf45a1ef1663fd5dc4f07c0ba/skills/43-wentorai-research-plugins/skills/research/automation/aim-experiment-guide/SKILL.md。来源不一致时停止并说明差异。核对通过后,用 Skill Finder 安装目标 Skill,并报告安装路径和来源版本。
此命令从 GitHub 来源仓库获取指定技能。运行前请先查看源文件。
npx skills add https://github.com/brycewang-stanford/Auto-Empirical-Research-Skills/tree/bfa0a6e5a843babbf45a1ef1663fd5dc4f07c0ba/skills/43-wentorai-research-plugins/skills/research/automation/aim-experiment-guide --skill aim-experiment-guide来源与配置
查看 GitHub 源码brycewang-stanford/Auto-Empirical-Research-Skills固定提交 bfa0a6e5a843babbf45a1ef1663fd5dc4f07c0ba
配置方式待核实
请查看原始来源中的适用工具、依赖和权限要求。
安装实测待补充
本条目面向分析数据与指标归因场景,提供“aim-experiment-guide”技能的目录入口。公开标签:learning、实验追踪、Aim、超参数。请通过固定上游版本核对实际功能、依赖与权限;此页为 Skillπ 独立编写的用途介绍和原站链接,不代表运行安全保证。原文与附件依各自许可开放。
文件浏览
---
name: aim-experiment-guide
description: "Track and compare research experiments with Aim experiment tracker"
metadata:
openclaw:
emoji: "🎯"
category: "research"
subcategory: "automation"
keywords: ["experiment-tracking", "visualization", "mlops", "reproducibility", "metrics", "hyperparameters"]
source: "https://github.com/aimhubio/aim"
---
# Aim Experiment Tracker Guide
## Overview
Aim is an open-source experiment tracking platform designed for researchers and ML engineers who need to log, compare, and analyze large numbers of experiments. Unlike cloud-based tracking services that require sending data to external servers, Aim runs entirely on your own infrastructure, making it suitable for research environments with data privacy requirements or institutional restrictions on external services.
The core problem Aim solves is experiment management at scale. A typical research project involves hundreds or thousands of training runs with different hyperparameters, data splits, model architectures, and random seeds. Without systematic tracking, researchers lose track of which configurations produced which results, leading to wasted computation and unreproducible findings. Aim provides a high-performance storage backend and a rich web UI for logging, querying, and visualizing experiment metadata and metrics.
With over 6,000 GitHub stars, Aim has established itself as a compelling self-hosted alternative to tools like Weights and Biases and MLflow. Its Python-native API integrates with minimal friction into existing training loops, and the query language enables sophisticated filtering across thousands of runs.
## Installation and Setup
Install Aim via pip:
```bash
pip install aim
```
Initialize an Aim repository in your project directory:
```bash
cd /path/to/research-project
aim init
```
This creates a `.aim` directory that stores all experiment data locally. Launch the web UI:
```bash
aim up
```
The dashboard becomes available at `http://localhost:43800`, providing interactive visualizations of all tracked experiments.
For remote server deployment:
```bash
aim up --host 0.0.0.0 --port 43800
```
## Core Features
**Experiment Logging**: Integrate Aim tracking into your training scripts with minimal code changes:
```python
from aim import Run
# Initialize a tracked run
run = Run(experiment="protein_folding_v2")
# Log hyperparameters
run["hparams"] = {
"learning_rate": 0.001,
"batch_size": 64,
"model": "transformer",
"num_layers": 6,
"hidden_dim": 256,
"dropout": 0.1,
"optimizer": "adamw",
"weight_decay": 0.01,
"seed": 42,
}
# Log dataset information
run["dataset"] = {
"name": "protein_benchmark_v3",
"train_size": 50000,
"val_size": 5000,
"test_size": 5000,
}
# Track metrics during training
for epoch in range(num_epochs):
train_loss = train_one_epoch(model, train_loader)
val_loss, val_accuracy = evaluate(model, val_loader)
run.track(train_loss, name="loss", context={"subset": "train"})
run.track(val_loss, name="loss", context={"subset": "val"})
run.track(val_accuracy, name="accuracy", context={"subset": "val"})
```
**Framework Integrations**: Aim provides built-in callbacks for popular training frameworks:
```python
# PyTorch Lightning integration
from aim.pytorch_lightning import AimLogger
aim_logger = AimLogger(experiment="lightning_exp")
trainer = pl.Trainer(logger=aim_logger, max_epochs=100)
# Hugging Face Transformers integration
from aim.hugging_face import AimCallback
aim_callback = AimCallback(experiment="hf_training")
trainer = Trainer(
model=model,
args=training_args,
callbacks=[aim_callback],
)
# Keras integration
from aim.keras import AimCallback as KerasAimCallback
model.fit(
x_train, y_train,
callbacks=[KerasAimCallback(experiment="keras_exp")],
epochs=50,
)
```
**Powerful Query Language**: Filter and retrieve experiments programmatically:
```python
from aim import Repo
repo = Repo("/path/to/research-project")
# Query runs matching specific criteria
query = """
run.experiment == "protein_folding_v2"
and run.hparams.learning_rate < 0.01
and run.hparams.model == "transformer"
"""
for run in repo.query_runs(query).iter_runs():
print(f"Run: {run.hash}")
print(f" LR: {run['hparams']['learning_rate']}")
print(f" Final val loss: {run['loss']}")
```
**Rich Visualizations**: The web UI provides interactive charts for comparing experiments:
- Line charts for metric trajectories across epochs
- Parallel coordinates plots for hyperparameter exploration
- Scatter plots correlating hyperparameters with final metrics
- Distribution plots for metric analysis across run groups
- Image and audio tracking for multimedia experiments
## Research Workflow Integration
**Hyperparameter Search Analysis**: After running grid search or random search experiments, use Aim to identify the best configurations:
```python
from aim import Repo
repo = Repo(".")
# Find the best run by validation accuracy
best_run = None
best_acc = 0.0
for run_metrics in repo.query_metrics(
"metric.name == 'accuracy' and metric.context.subset == 'val'"
).iter_runs():
for metric in run_metrics:
final_val = list(metric.values.values())[-1]
if final_val > best_acc:
best_acc = final_val
best_run = metric.run.hash
print(f"Best run: {best_run} with accuracy {best_acc:.4f}")
```
**Reproducibility Documentation**: Every tracked run captures the full hyperparameter configuration, making it straightforward to include exact experimental details in paper methods sections and supplementary materials.
**Ablation Studies**: Tag runs with ablation group identifiers and use the comparison UI to visualize the impact of each component:
```python
run = Run(experiment="ablation_study")
run["hparams"] = config
run["ablation"] = {
"group": "attention_mechanism",
"variant": "multi_head",
"description": "Standard multi-head attention vs. linear attention",
}
```
**Lab Notebook Integration**: Export experiment summaries for inclusion in electronic lab notebooks. The query API enables automated report generation:
```python
import pandas as pd
from aim import Repo
repo = Repo(".")
records = []
for run_metrics in repo.query_metrics(
"metric.name == 'accuracy'"
).iter_runs():
run = run_metrics.run
for metric in run_metrics:
values = list(metric.values.values())
records.append({
"run_hash": run.hash[:8],
"model": run["hparams"].get("model"),
"lr": run["hparams"].get("learning_rate"),
"final_accuracy": values[-1] if values else None,
})
df = pd.DataFrame(records)
df.to_csv("experiment_summary.csv", index=False)
```
## Storage and Performance
Aim uses a custom high-performance storage engine optimized for time-series metrics data. The storage scales to millions of tracked values across thousands of runs without significant degradation in query performance.
Data is stored locally in the `.aim` directory. Back up this directory to preserve your experiment history. For team settings, the Aim server can be deployed as a shared service accessible to multiple researchers.
```bash
# Check storage usage
du -sh .aim/
# Export data for archival
aim storage --repo . upgrade 3.0
```
## References
- Aim repository: https://github.com/aimhubio/aim
- Aim documentation: https://aimstack.readthedocs.io/
- Aim UI demo and screenshots in the repository wiki
- Comparison with MLflow and Weights and Biases in the documentation
版本历史
-
v2026.07
GitHub ↗
原版更新说明
## Highlights - **First tagged release.** Every gate is green: `make check` (validate + 183 unit tests + eval-harness + numeric benchmarks), validate-catalog, quality-evals, and OpenSSF Scorecard on `main`. - **Rigor double loop at 13 method families.** Each family closes taxonomy tag → eval scenario (does the agent avoid the method's known traps?) → deterministic numeric benchmark (does it recover the true effect?). New this cycle: heterogeneous treatment effects (CATE) and quantile treatment effects (QTE). - **Paper-WorkFlow flagship at 29/29 executable rigor gates**, now merged to its `main` with a drift-gated README badge — the credibility artifact is checked by CI, so it can never silently go stale ([RIGOR.md](https://github.com/brycewang-stanford/Paper-WorkFlow/blob/main/RIGOR.md)). - **1,150 vendored skills / 69 collections**, all license-audited, security-scanned, tracked in a machine-readable catalog with CI-enforced freshness, and importable as a single whole-repo router skill. - **Hardened weekly upstream syncs**: synced trees must pass the full gate before merge; this cycle both sync PRs were verified, patched (StatsPAI's SkillOpt execution-gate card restored), and merged. - **First-party plugin marketplace** (Python / R / Stata empirical-analysis plugins + guide) shipped in the previous cycle, now under a tagged version. ## Known Follow-Ups - Ecosystem benchmark comparison (AERS vs econometrics agents on the shared benchmark suite) — [PLAN-2026-07 week 3](https://github.com/brycewang-stanford/Auto-Empirical-Research-Skills/blob/main/docs/PLAN-2026-07.md). - Decomposition (Oaxaca–Blinder) and shift-share (Bartik) method families (weeks 3–4). - GitHub Pages catalog site, README demo screenshots, five-language README consistency script (week 4). ## Catalog - Top-level collections: **69** - `SKILL.md` files: **1150** - Cataloged agent tools: **335** ([`tools/CATALOG.md`](../tools/CATALOG.md)) ## Methodological rigor - Method families in the coverage map: **13** ([`RIGOR_COVERAGE.md`](RIGOR_COVERAGE.md)) - Eval scenarios: **26** (138 rubric items, 122 auto-checkable) ([`../eval-harness/`](../eval-harness/README.md)) - Benchmark tasks: **13** (70 gold checks, 58 required) ([`../benchmark/`](../benchmark/README.md)) ### Method families (skills tagged) - `iv`: 27 - `rdd`: 25 - `panel-fe`: 19 - `staggered-did`: 17 - `bayesian`: 13 - `did`: 10 - `event-study`: 10 - `matching`: 10 - `synthetic-control`: 10 - `cate`: 8 - `dml`: 8 - `survival`: 3 - `quantile`: 2 ### Eval scenarios by category - `causal-identification`: 14 - `citation-hygiene`: 1 - `reproducibility`: 2 - `research-integrity`: 1 - `runtime-safety`: 1 - `statistical-validity`: 2 - `writing-compliance`: 3 - `writing-style`: 2 ## License buckets - AGPL-3.0: 1 - Apache-2.0: 1 - CC-BY-NC-4.0 (non-commercial): 1 - CC-BY-SA-4.0: 1 - CC-BY-SA-4.0 (repository default): 4 - GPL-3.0: 1 - MIT: 35 - MIT Non-Commercial: 1 - Mixed (per-skill; see folder): 2 - UNKNOWN - check upstream: 22 ## Quality gates - `make check` — catalog/provenance/audit/eval freshness, repo hygiene, Paper-WorkFlow gates, stdlib unit tests, eval-harness lint + graded smoke, and the strict numeric benchmark (reference-gold drift fails the build). - External links and OpenSSF Scorecard run on their scheduled workflows.
当前收录版本
0.0.1
质量评测
满分 100。前四项来自 DeepSkill 收录的质量报告。
评分依据
提供 Aim 实验跟踪器的完整端到端使用指南,覆盖安装、配置、集成和高级查询
评分依据
description 清晰准确描述 Skill 功能为实验跟踪与比较
评分依据
文档结构完整,包含 Overview、Installation、Core Features、Workflow Integration 等逻辑清晰的章节
评分依据
提供完整的安装、初始化和启动命令,参数正确且可执行
计算方式
取前四项原始分值的算术平均,四舍五入取整。可通过收录报告核对各项评分与说明,运行记录需单独查证。
权限与使用前检查
根据当前主文档自动整理,命中内容可能是示例或防护说明;不是实际权限声明,也未覆盖附件、外部依赖或运行行为。
-
安装或获取外部依赖
使用官方来源并锁定版本,在隔离环境检查安装脚本。当前文档检查不覆盖后续下载的代码。
主文档提及 · 第 28 行
文档引用的域名或地址
这些地址可能用于文档链接或示例,不是已批准的联网清单。使用前核对实际目的地及发送的数据。
- aimstack.readthedocs.io
- github.com
- localhost
安全评级与访问范围
risk · 静态证据核验,不代表运行安全保证。
使用前请由宿主核对并授权实际文件读写、外部服务和联网目的地;本站的核验不会授予任何运行权限。
用户讨论
还没有已核实的点评。