All pull requests

Pull requests

• Add DSV4.1 threshold and reference implementation example

  size/normal
  #519·zihaok opened Sep 23, 2026Collaborator·

  ·
  3

• Integrate steady state into report

  priority: P0size/large
  #514·nv-alicheng opened Sep 21, 2026Collaborator·

  ·
  69

• fix(ci): allow automatic review requests for fork PRs

  size/normal
  #513·YibaiMeng opened Sep 21, 2026Collaborator·

  ·
  2

• fix(metrics): clarify chat-template fallback logging

  size/normal
  #509·YibaiMeng opened Sep 18, 2026Collaborator·

  ·
  2

• feat(docker): provision ROUGE scorer deps and corpora in dev image

  size/normal
  #507·anandhu-eng opened Sep 15, 2026Contributor·

  ·
  1

• feat(deps): add rouge extra for ROUGE scoring deps

  size/normal
  #506·anandhu-eng opened Sep 15, 2026Contributor·

  ·
  3

• feat(swebench): add trajectory-stable session routing

  size/normal
  #503·tianmu-li opened Sep 10, 2026Collaborator·

  ·
  7

• Add submission cli test for create run

  size/large
  #502·anandhu-eng opened Sep 10, 2026Contributor·

  ·
  1

• document output folder structure

  size/normal
  #498·anandhu-eng opened Sep 9, 2026Contributor·

  ·
  1

• feat(swe-bench): add persistent Pyxis runtime

  size/very-large
  #496·leopck opened Sep 5, 2026Collaborator·

  ·
  1

• feat(provenance): resolve source git SHA via a validated multi-channel chain

  size/normal
  #491·arekay-nv opened Sep 1, 2026Collaborator·

  ·
  3

• feat(report): add events.jsonl SHA-256 to result_summary.json

  size/normal
  #490·arekay-nv opened Aug 31, 2026Collaborator·

  ·
  2

• feat(agentic): auto-download official dataset

  size/normal
  #489·tianmu-li opened Aug 27, 2026Collaborator·

  ·
  21

• feat(swe-bench): add the distributed fleet scorer

  size/large
  #485·leopck opened Aug 27, 2026Collaborator·6/6·

  ·
  6

• feat(swe-bench): gate dispatch on endpoint readiness

  size/large
  #484·leopck opened Aug 27, 2026Collaborator·5/6·

  ·
  7

• feat(swe-bench): classify evaluation infrastructure failures

  size/normal
  #483·leopck opened Aug 27, 2026Collaborator·4/6·

  ·
  1

• feat(swe-bench): recover conservatively from infrastructure failures

  size/large
  #482·leopck opened Aug 27, 2026Collaborator·3/6·

  ·
  4

• feat(swe-bench): enforce run-scoped all-or-nothing merging

  size/normal
  #481·leopck opened Aug 27, 2026Collaborator·2/6·

  ·
  1

• feat(swe-bench): add a durable distributed work queue

  size/large
  #480·leopck opened Aug 27, 2026Collaborator·1/6·

  ·
  13

• [docs] Benchmark suite and ruleset proposal

  size/normal
  #468·arekay-nv opened Aug 25, 2026Collaborator·

  ·
  42

• fix(swebench-service): give Pyxis container creation its own deadline

  size/normal
  #456·leopck opened Aug 19, 2026Collaborator·

  ·
  2

• fix(benchmark): an accuracy run that produces no number must fail

  size/normal
  #454·leopck opened Aug 19, 2026Collaborator·

  ·
  6

• fix(benchmark): don't reject an accuracy-only run for listing many endpoints

  size/normal
  #453·leopck opened Aug 19, 2026Collaborator·

  ·
  2

• Added Conversational Aware Routing for Dynamo support

  #434·leopck opened Jul 30, 2026Collaborator·

  ·
  3

• Add License Scanning and Reporting for Endpoints Changes on Main.

  #412·arav-agarwal2 opened Jul 14, 2026Collaborator·

  ·
  18