ML Model Training Management

Select Model for Evaluation

Evaluation Metrics (MedHELM LLM-Jury)

Based on MedHELM (Bedi et al., 2025) - arXiv:2505.23802. Uses LLM-jury evaluation with three criteria. Final score is the mean of all three metrics (equal weighting).

📚 Available Human-Annotated Benchmarks

Standard medical Q&A benchmarks available for evaluation. These datasets are human-annotated and clinically validated.

Run Evaluation

Evaluation Status

Evaluation Results

Extraction Method

Clinical Note Input

Extraction Status

📊 Create FactEHR Dataset from PhysioNet

Generate FactEHR-style datasets from PhysioNet clinical notes. Select a dataset source and extraction method.

📚 Human-Annotated Medical Benchmarks

Standard medical Q&A benchmarks available for evaluation. These datasets are human-annotated and clinically validated.

✨ Enhanced Features

Select a completed training job to view results.

📚 Quick Links

Base URL: https://your-backend-server.com

API Version: v1

Content-Type: application/json

import requests

API_BASE_URL = "https://your-backend-server.com"
API_KEY = "YOUR_API_KEY"

# Start training with billing allocation
response = requests.post(
    f"{API_BASE_URL}/api/training/start",
    params={
        "instance_type": "48vcpu",
        "datasets": "MIMICIII,MIMIC4"
    },
    headers={
        "Authorization": f"Bearer {API_KEY}",
        "Content-Type": "application/json"
    },
    json={
        "user_id": "external_user_123",
        "billing_account": "account_abc"
    }
)

result = response.json()
job_id = result["job_id"]
print(f"Training started: {job_id}")

💡 Note: All costs are automatically tracked and allocated to the provided user_id and billing_account. You can query costs at any time using the billing endpoints.

📞 Support

For API access, billing questions, or technical support:

• Email: api-support@summithealth.ai
• Documentation: View Full Documentation