Automated Verification

Overview

The Quantum Advantage Tracker (QAT) uses an automated verification pipeline to maintain the integrity and scientific rigor of the data displayed on the trackers. This system acts as a gatekeeper, ensuring that every submission conforms to the required data schemas and scientific criteria before it is merged into the public repository.

The verification process is triggered automatically whenever a new submission is made via the GitHub Issue Templates.

The Submission Lifecycle

The automated workflow follows a structured path from initial submission to site deployment:

1. Issue Creation: A contributor fills out a pathway-specific issue template (Observable Estimations, Variational Problems, or Classically Verifiable Problems).
2. Schema Parsing: A GitHub Action parses the structured form data from the issue.
3. Automated Validation: The system runs a series of checks against the submitted values and associated circuit/Hamiltonian files.
4. Feedback Loop: The bot provides immediate feedback via issue comments.
   • Success: If the submission passes all checks, it is labeled as verified and prepared for inclusion.
   • Failure: If checks fail, the bot lists specific errors (e.g., missing error bars, invalid circuit IDs) for the contributor to fix.
5. Data Integration: Once approved, the data is appended to the corresponding submissions.json file in the data/ directory.

Automated Validation Checks

The verification engine performs different levels of validation based on the chosen pathway.

1. Structural Validation

Every submission must pass basic structural requirements:

• Institution Mapping: Validates that participating organizations are correctly formatted to populate the Contributors marquee.
• Required Metadata: Ensures createdAt, qubits, and gates (for circuits) or energy_bounds (for variational) are present.
• Path Integrity: Verifies that referenced circuit models or Hamiltonians exist within the repository's data structure.

2. Pathway-Specific Logic

The automation applies logic unique to each tracker:

| Pathway | Primary Verification Focus | | :--- | :--- | | Observable Estimations | Validates the presence of mathematically grounded error bars and expectation values. | | Variational Problems | Verifies that the submitted ground-state energy respects the variational principle (upper-bounding). | | Classically Verifiable | Cross-references quantum outputs against known classical witnesses or exact solutions. |

Data Schema Example

The automation validates submissions against the internal CircuitModels and Submissions types. A valid submission entry generally follows this JSON structure:

{
  "id": "example-submission-id",
  "path": "path/to/results.json",
  "qubits": 127,
  "gates": 5000,
  "institutions": "University of Quantum, Q-Corp",
  "createdAt": "2025-05-20"
}

Troubleshooting Bot Feedback

When a submission is flagged by the automated verifier, the bot will post a comment detailing the reason. Common issues include:

• Invalid Instance ID: The id provided does not match any existing instance in the circuit-models.json or hamiltonians.json files.
• Formatting Errors: The submission contains non-standard characters or incorrectly formatted dates (expected format: YYYY-MM-DD).
• Missing Files: The issue references a circuit file or data log that has not been uploaded or linked correctly in the repository.

To resolve these, contributors should edit the original issue description. The automation will re-run validation once the issue is updated.