TASK 1 / SUBMISSION

Submit Task 1 predictions on the web.

All train, development, and test files are available from the participant hub. No pre-registration or access code is required; submit directly through the separate development and test pages.

Access: Open · no team code
Development: Scores + receipt + rank
Test: Format check + receipt

Test resubmissions: unlimited before the deadline

You may submit revised Test predictions as often as needed before 25 October 2026, 23:59 Anywhere on Earth. There is no daily or total Test submission quota. Keep the same Team Name and Contact Email for all revisions.

Your latest accepted, valid submission before the deadline replaces earlier accepted submissions for final evaluation. An invalid or rejected upload does not replace your last accepted submission. Save each acceptance receipt.

Uploading an identical previously accepted file returns its original receipt and acceptance time; it does not create a new revision or change which submission is latest. Earlier records are retained privately for the competition audit trail. Test scores and ranks remain hidden until final results.

To restore earlier predictions, generate a new valid ZIP containing the earlier predictions.jsonl and upload it with the same Team Name and Contact Email before the deadline. The ZIP must differ from the previous archive and receive a new acceptance receipt; the predictions may stay unchanged. A replay returning the original receipt does not restore the earlier version. Restoration clarification published 12 October 2026.

Policy updated 16 September 2026, replacing the previous Test submission-count limits.

Task 1 rules clarification · 12 October 2026

Final ranking and ties

The frozen scorer is finreason.task1.scorer/2.0.0. Final ranking uses descending lexicographic order of (CombinedFAC, HiddenFAC, CombinedCheckpoint), with each key quantized to six decimal places using round-half-even. CombinedFAC and CombinedCheckpoint each weight the seen and hidden strata within the Test evaluation equally; they do not average development leaderboard scores with Test scores. Teams identical on all three keys share the same rank. Submission time is not an additional tie-breaker.

Use of public FinChain templates

Subject to the resource licence, teams may generate additional training, few-shot or local evaluation data from public FinChain templates, use those templates to categorize questions for analysis, and match questions to public templates to compute predictions. Computing predictions with a public symbolic template is permitted. Describe the FinChain version, matching and execution procedure, and contribution to predictions in the Working Notes. Accessing private evaluation answers, bypassing submission limits or exploiting the evaluation service is prohibited.

Development submission limit

Each normalized Team Name may make two accepted submissions per UTC day and 40 accepted submissions in total. The daily window resets at 00:00 UTC. Rejected uploads and identical re-uploads of an accepted ZIP do not consume another submission. Keep one consistent Team Name. Test submissions remain unlimited before the deadline.

Restoring earlier Test predictions

To restore earlier predictions, generate a new valid ZIP containing that earlier predictions.jsonl and upload it under the same Team Name and Contact Email before the deadline. The new ZIP must differ from the previously accepted archive and receive a new acceptance receipt; its prediction contents may remain unchanged. If the portal returns the original receipt and acceptance time, the upload is a replay and the latest submission has not changed. The latest accepted, valid submission before the deadline is used for final evaluation.

Organizer model baselines

Lite, Standard and Advanced are model-based references evaluated on the same 580 development questions with the frozen scorer. They generate final-answer and checkpoint predictions from public questions without gold-answer inputs. Their Final answer / Reasoning steps scores are Lite 0.234048 / 0.555354, Standard 0.980556 / 0.987725, and Advanced 0.984444 / 0.990000. Model identities and full configurations are not currently public. Cite the published baseline labels and development leaderboard. Publication of Test scores for these three systems has not been confirmed.

HOW TO SUBMIT

Six steps from data to receipt

Follow the phase-specific filenames exactly. Competition predictions and the challenge paper use different submission routes.

Step 1 of 6: Choose a consistent team name.
No pre-registration, approval, access code, or account is required. Choose one Team Name and use it consistently for development and test submissions. The test page also asks for a Contact Email for submission identification, submission-related support, matching final results to the related challenge paper, and selection of the latest accepted test submission and handling repeat uploads through a non-public pseudonymous identifier; the email is not a login and is never published.
Step 2 of 6: Download the correct phase files.
Development: leaderboard_questions.jsonl and leaderboard_expected_ids.json. Do not use dev_questions.jsonl for the 580-row leaderboard upload.
Test: test_questions.jsonl and test_expected_ids.json.
Use the matching expected-IDs file as your checklist: include exactly one prediction for every listed case_id, with no missing, duplicate, or additional IDs.
03
Step 3 of 6: Create predictions.jsonl.
Write one JSON object for every expected case ID: 580 rows for development or 928 rows for test. Every row uses exactly schema_version, dataset_version, case_id, final_answer, and steps.
Start from the sample predictions and check the prediction schema. If useful, generate a separate blank template below, then save your completed system output as predictions.jsonl. The blank template is a format example, not a competitive prediction file.
Optional development template
```
python3 scripts/task1_cli.py baseline-b0 --questions public/task1/data/development/leaderboard_questions.jsonl > blank_predictions.jsonl
```
Optional test template
```
python3 scripts/task1_cli.py baseline-b0 --questions public/task1/data/test/test_questions.jsonl > blank_predictions.jsonl
```

Step 4 of 6: Validate and package one canonical ZIP.

Clone or download the public participant toolkit, run the commands for your phase, and continue only when both validation commands report valid: true and the package command reports status: PASS. The ZIP must contain exactly one root-level file named predictions.jsonl. If an old submission.zip exists, remove or rename it before running the package command.

Development · 580 rows

python3 scripts/task1_cli.py validate --questions public/task1/data/development/leaderboard_questions.jsonl --predictions predictions.jsonl
python3 scripts/task1_cli.py package --questions public/task1/data/development/leaderboard_questions.jsonl --predictions predictions.jsonl --output submission.zip
python3 scripts/task1_cli.py validate-zip --questions public/task1/data/development/leaderboard_questions.jsonl --submission-zip submission.zip

Test · 928 rows

python3 scripts/task1_cli.py validate --questions public/task1/data/test/test_questions.jsonl --predictions predictions.jsonl
python3 scripts/task1_cli.py package --questions public/task1/data/test/test_questions.jsonl --predictions predictions.jsonl --output submission.zip
python3 scripts/task1_cli.py validate-zip --questions public/task1/data/test/test_questions.jsonl --submission-zip submission.zip

Step 5 of 6: Upload submission.zip on the matching webpage.
Development: enter Team Name and select the single submission.zip file. Test: enter the same Team Name, provide a Contact Email, and select the single submission.zip file. Choose Submit once and do not upload individual JSONL files.
Development upload (opens in a new tab)Test upload (opens in a new tab)
Step 6 of 6: Check the correct result.
Development:wait for the result to show the Final answer score, Reasoning steps score, receipt ID, and current rank; the team's best eligible result also appears on the public development leaderboard. Test: check the format feedback and acceptance status, then save the receipt ID when accepted; no test score, rank, answer-correctness feedback, or score-derived signal is shown before the final results are released.

Task 1 required submission materials.

By 25 October 2026, 23:59 Anywhere on Earth, submit a ZIP containing exactly one root-level predictions.jsonl for all 928 Test questions through the Test submission portal linked above. Keep the same Team Name and Contact Email and save the acceptance receipt. The latest accepted, valid Test submission determines the final competition result; Development submissions do not qualify for final ranking.

This requirement applies to every Task 1 team seeking final ranking, not only the highest-ranked teams. No separate source-code archive, model weights, repository link, or additional report is required by the competition deadline. Code and reproducibility materials may be shared voluntarily with your Working Notes.

Teams seeking final ranking and awards must also submit Working Notes through CyberChair SC03 by 2 November 2026, 23:59 Anywhere on Earth. The paper is submitted separately from the prediction ZIP.

Challenge paper is separate.

Submit the paper PDF through CyberChair SC03, using the IEEE two-column conference template. The FinReason Cup requirement is no more than ten pages total, including references. Working Notes are due by 2 November 2026 at 23:59 Anywhere on Earth; final competition predictions are due earlier, on 25 October 2026 at 23:59 Anywhere on Earth. Follow the organizer schedule updated on 18 October if CyberChair displays a different date. Paper feedback and organizer acceptance decisions are scheduled for 9 November; camera-ready papers are due on 16 November 2026 at 23:59 Anywhere on Earth. Keep author names and affiliations in the PDF, following the conference's single-blind review policy. Task 1 Working Notes must cite the FinReason Cup overview, the Task 1 overview, and the FinChain benchmark paper. Include the overviews for any other tasks you enter; see the required references and available official BibTeX. Do not upload a paper PDF to either prediction workspace.

Direct web upload availableVerified live links

SUBMISSION PAGES

Development and test submission

Development and test use separate submission pages. An accepted development upload shows the Final answer score, Reasoning steps score, receipt ID, and current rank immediately, and updates the public best-per-team leaderboard. Test shows format-check feedback and an acceptance receipt, with no score, rank, answer-correctness feedback, or score-derived signal before the final results are released.

Open development submission Open test submission

No pre-registration, approval, access code, or account is required. The development page asks for Team Name and a ZIP. The test page asks for Team Name, Contact Email, and a ZIP. Contact Email is private and is used only for submission identification, submission-related support, matching final results to the related challenge paper, and selecting the latest accepted test submission and handling repeat uploads through a non-public pseudonymous identifier, not as a login. Review the Terms of Participation and Privacy Notice. Participant support: zhuohan.xie@mbzuai.ac.ae.

01 / TRAIN

Build with public answers.

Train questions, gold answers, and canonical target examples are already available from the participant hub. Train results are not submitted to a leaderboard.

02 / DEVELOPMENT

Receive scores immediately.

Upload the 580-row predictions ZIP. Each accepted submission immediately returns the Final answer score, Reasoning steps score, receipt ID, and current rank, and updates the public best-per-team leaderboard.

03 / TEST

Check format and acceptance.

Upload predictions for the public 928-question test release and retain the receipt identifier. The portal shows format feedback and acceptance status. Test submissions receive no online score or rank. Official test evaluation is performed by the organizers after submissions close.

Open Test submission status