Sacred : Lightweight experiment tracking for machine learning

Sacred is an open-source Python library designed to facilitate reproducible machine learning experiments by helping researchers and developers organize, configure, log, and track experiments in a lightweight and flexible way. Originally developed by the Swiss AI lab IDSIA, Sacred is used in academic and research contexts where structured experiment management, traceability, and minimal setup overhead are important.

Unlike full-featured platforms, Sacred provides a code-centric, dependency-free approach to experiment monitoring, with optional integrations for storage and visualization (e.g., with MongoDB and Sacredboard).

Key benefits:

• Simple, code-based way to log configurations, results, and metadata
• Designed for reproducibility and minimal external dependencies
• Suitable for researchers and developers working in Python environments

What are the main features of Sacred?

Configuration management and reproducibility

• Tracks all configurable parameters of an experiment via decorators
• Uses named configurations and ingredients to manage complex setups
• Automatically captures source code versions, command-line arguments, and dependencies
• Ensures that experiments can be re-executed identically

Logging and result tracking

• Logs metrics, status, artifacts, and exceptions during execution
• Supports structured result output and custom observers
• Records start/end time, host information, and exit codes
• Integrates with MongoDB to persist experiment runs and metadata

Observers and extensibility

• Uses observer classes to send experiment data to different backends
• Built-in observers: MongoDB, file storage, Slack (notifications), SQL, and more
• Developers can create custom observers for new storage or notification systems
• Modular architecture allows easy extension for specific needs

Minimalistic and framework-agnostic

• Does not depend on any specific ML library or data pipeline tool
• Can be integrated with any training loop, model, or data source
• Lightweight and suitable for academic and scripting-based workflows
• Maintains high compatibility with standard Python workflows

Optional visualization with Sacredboard

• Sacredboard provides a web interface to browse, search, and compare experiments
• Displays configurations, logs, metrics, and outputs
• Helps analyze and navigate experiment history from MongoDB storage
• Useful for collaborative research and reviewing long-running experiments

Why choose Sacred?

• Designed for clarity, simplicity, and reproducibility in ML experiments
• Lightweight, open-source, and easy to integrate into existing workflows
• Highly flexible thanks to custom observers and code-centric configuration
• Ideal for academic research, rapid prototyping, and offline experiment tracking
• Enables transparent documentation of all experiment settings and outcomes