Experimental Launcher

The Experimental Launcher is an automated system that manages the complete lifecycle of running Bonsai experiments, from repository setup to experiment execution. It ensures reproducible, reliable experiment execution with automatic package management and error handling.

Overview

The launcher provides a unified interface for running Bonsai experiments with the following key capabilities:

Automated Repository Management: Downloads and syncs experiment code from GitHub
Bonsai Installation & Package Management: Automatically installs and verifies Bonsai packages
Experiment Execution: Launches Bonsai workflows with proper parameter handling
Error Recovery: Automatically fixes common package and dependency issues
Data Management: Handles experiment data storage and session tracking

Architecture

Core Components

BonsaiExperiment Class (bonsai_experiment_launcher.py)

Main orchestrator for experiment execution
Handles all phases of experiment lifecycle
Provides error handling and recovery mechanisms

Parameter Management

JSON-based configuration system
Environment-specific settings (camstim config integration)
Session metadata and UUID generation

Process Management

Windows job object integration for proper cleanup
Process monitoring and timeout handling
Signal handling for graceful shutdown

Experiment Lifecycle

Phase 1: Repository Setup

Clone/update repository from GitHub
Checkout specific commit or branch
Verify repository integrity
Set up local working directory

Phase 2: Bonsai Installation

Locate Bonsai executable in repository
Verify Bonsai installation
Run setup scripts if needed
Validate Bonsai version compatibility

Phase 3: Package Verification

Parse Bonsai.config for required packages
Scan installed packages directory
Compare versions and dependencies
Auto-reinstall if mismatches detected

Phase 4: Stimulus Table Generation

Generate session UUID and metadata
Create session-specific stimulus CSV using on-the-fly generation
Use session-specific random seed for unique stimulus sequences
Place stimulus table in session output folder

Phase 5: Experiment Execution

Prepare experiment parameters with generated stimulus table path
Launch Bonsai workflow with --start --no-editor
Monitor process and capture output
Capture and log experiment output

Phase 6: Cleanup & Data Management

Ensure proper process termination
Save experiment data and metadata
Generate session reports
Clean up temporary resources

Configuration

Parameter File Structure

The launcher uses JSON parameter files with the following structure:

{
    "repository_url": "https://github.com/AllenNeuralDynamics/openscope-community-predictive-processing.git",
    "commit_hash": "main", 
    "local_repository_path": "C:/BonsaiDataPredictiveProcessingTest",
    "bonsai_path": "code/stimulus-control/src/Mindscope/generic_oddball.bonsai",
    "session_type": "sensorimotor_mismatch",
    "mouse_id": "test_mouse",
    "user_id": "test_user"
}

Key Parameters

Parameter	Description	Required	Default
`repository_url`	GitHub repository URL	Yes	-
`commit_hash`	Specific commit or branch name	Yes	-
`local_repository_path`	Local directory for repository	Yes	-
`bonsai_path`	Relative path to workflow file	Yes	-
`session_type`	Experimental session type for stimulus generation	Yes	-
`mouse_id`	Subject identifier	Yes	-
`user_id`	Experimenter identifier	Yes	-
`bonsai_exe_path`	Relative path to Bonsai executable	No	`tools/Bonsai.startstop/Bonsai.exe`
`bonsai_setup_script`	Path to package installation script	No	`code/stimulus-control/bonsai/setup.cmd`

Session Types

The launcher supports the following session types for automatic stimulus table generation:

Session Type	Description	Typical Duration
`visual_mismatch`	Standard oddball with orientation deviants	~66 minutes
`sensorimotor_mismatch`	Motor coupling with mismatch blocks	~66 minutes
`sequence_mismatch`	Sequential learning with pattern violations	~66 minutes
`duration_mismatch`	Temporal oddball with duration deviants	~66 minutes
`sequence_no_oddball`	Long blocks without oddball trials	~66 minutes
`sensorimotor_no_oddball`	Long motor coupling without mismatches	~66 minutes

Each session type generates a unique stimulus table with appropriate control blocks and RF mapping.

Stimulus Table Generation

On-the-Fly Generation

The launcher automatically generates stimulus tables for each experimental session using the integrated CSV generator. This approach provides several advantages:

Unique Stimulus Sequences: Each session uses a session-specific random seed, ensuring no two sessions have identical stimulus presentations
Reproducible Results: The random seed is logged with session metadata, allowing exact replication when needed
Simplified Workflow: No need to pre-generate or manage static CSV files
Reduced Storage: Only the active session's stimulus table is stored, eliminating large collections of unused CSV files

Generation Process

Session Initialization: When a session starts, the launcher extracts the session_type parameter
Seed Generation: A unique random seed is created based on the session UUID, session type, and current time
CSV Creation: The generator script is called with the session type, output path (in session folder), and unique seed
Parameter Update: The stimulus_table_path parameter is updated to point to the generated CSV file
Bonsai Launch: The workflow is launched with the path to the newly generated stimulus table

Logging and Traceability

All stimulus generation is fully logged: - Generator script execution and output - Random seed used for generation - Number of trials generated - Output file location and verification - Any errors or warnings during generation

This ensures complete traceability and reproducibility of experimental conditions.

Package Management

Automatic Package Verification

The launcher automatically verifies that installed Bonsai packages match the requirements specified in Bonsai.config. This includes:

Version Matching: Ensures exact version compatibility
Dependency Resolution: Verifies all package dependencies
Missing Package Detection: Identifies packages that need installation

Auto-Reinstall System

The launcher automatically fixes package issues when detected:

Detection: Identifies version mismatches or missing packages
Cleanup: Removes the entire Packages directory for a clean slate
Reinstallation: Runs the setup script to install all packages fresh
Verification: Re-checks packages to confirm the fix worked

This ensures experiments always run with the correct package versions.

Usage Examples

Basic Usage

from bonsai_experiment_launcher import BonsaiExperiment

# Create experiment instance
experiment = BonsaiExperiment()

# Run experiment with parameter file
success = experiment.run('experiment_params.json')

if success:
    print("Experiment completed successfully!")
else:
    print("Experiment failed - check logs for details")

Testing and Development

# Test the launcher with sample parameters
python test_bonsai_launcher.py

# Use custom parameter file
python test_bonsai_launcher.py --param-file my_params.json

Integration with Camstim Agent

The launcher is designed to be executed by the camstim agent - one of the Allen Institute's internal session management system. The integration works as follows:

Agent-Based Execution

Agent receives: A script path and YAML parameter file from the experiment scheduling system
Agent calls: The experimental launcher with the provided parameters
Agent monitors: Experiment execution and handles results

Parameter Flow

1. Experiment scheduling system → YAML parameters → camstim agent
2. camstim agent → experimental launcher
3. experimental launcher → experiment execution → results back to agent

Camstim Infrastructure Integration

Configuration: Reads from C:/ProgramData/AIBS_MPE/camstim/config/stim.cfg
Data Storage: Saves experiment data to camstim-compatible pickle files
Session Management: Uses camstim session UUID format and metadata structure
Logging: Compatible with camstim logging standards and tracking formats
Process Management: Integrates with camstim's process monitoring and cleanup

Expected Usage Pattern

# This is typically called by the camstim agent, not directly by users
from bonsai_experiment_launcher import BonsaiExperiment

# Agent provides the YAML file path
experiment = BonsaiExperiment()
success = experiment.run('agent_provided_params.json')  # Agent converts YAML→JSON

# Results are automatically saved in camstim-compatible format
# Agent handles success/failure reporting back to scheduling system

Camstim-Compatible Output Format

The launcher generates pickle files with the same structure as standard camstim experiments: - Platform and session metadata - Experiment parameters and checksums - Behavioral data integration points - LIMS-compatible data structure