Automated Processing Pipeline

Automated Processing Pipeline

To maintain scientific rigor and ensure the integrity of the trackers, the Quantum Advantage Tracker uses an automated pipeline to validate and ingest new submissions. This process is triggered whenever a contributor submits a new "Advantage Candidate" via the GitHub Issue templates.

Submission Workflow

The pipeline bridges the gap between a community-submitted issue and the live data displayed on the website. The workflow follows these steps:

1. Issue Creation: A contributor selects one of the three pathways (Observable Estimations, Variational Problems, or Classically Verifiable Problems) and fills out the corresponding YAML-based issue template.
2. Automated Verification: Upon submission, a GitHub Action (verify-issue.yml) is triggered. This action invokes a parsing script (parse-submission.js) to process the issue content.
3. Data Validation: The pipeline checks the submission against specific requirements for the chosen pathway (e.g., presence of error bars, valid circuit instances, or Hamiltonian references).
4. Ingestion & Deployment: Once validated, the data is structured into the project's internal JSON storage (located in the /data directory), and the site is rebuilt and deployed to GitHub Pages.

Verification Checks

The parse-submission.js script (internal) performs several automated checks to ensure data consistency. While contributors do not run this script directly, understanding its checks helps ensure a successful submission:

• Schema Validation: Ensures all required fields from the issue template (e.g., institutions, qubits, circuit_link) are present and correctly formatted.
• Path Verification: Validates that any referenced circuit models or Hamiltonians exist within the repository's data/ subfolders.
• Institutional Mapping: Normalizes institution names to ensure they are correctly represented in the Contributors marquee on the homepage.
• Type Enforcement: Converts string inputs from the GitHub issue into the appropriate numeric or boolean types required by the src/types definitions.

Interacting with the Pipeline

As a user or contributor, you interact with the automated pipeline primarily through the GitHub interface.

Submitting a Claim

To trigger the pipeline, visit the Participate page or go directly to the GitHub "New Issue" screen and select a template:

# Example Submission Fields (YAML Template)
pathway: "Observable estimations"
circuit_model: "Random Circuit Sampling"
qubits: 53
error_bars: "0.001"
publication_url: "https://arxiv.org/..."

Feedback and Errors

If a submission fails the automated validation, the pipeline will:

1. Post a Comment: An automated comment is added to your GitHub issue detailing the validation errors (e.g., "Missing required field: qubits").
2. Assign Labels: The issue may be labeled as invalid or needs-info until the contributor updates the issue description to meet the requirements.

Once the verification script passes, the issue is typically labeled as verified or active, and the new candidate will automatically appear in the corresponding tracker table after the next build cycle.

Local Development and Testing

If you are a developer looking to test how data is rendered after the pipeline processes it, you can manually add an entry to the local JSON files to see the results in your development environment:

# 1. Manually add a mock entry to data/observable-estimations/submissions.json
# 2. Run the development server
npm run dev

The website logic (specifically src/utils.ts) will automatically handle sorting and formatting the new data, mimicking the behavior of the production pipeline.