Where did GPT-6 Astra lead other models on SNAP policy?

Percentage-point difference in answer-key agreement by category, comparing original GPT-6 Astra runs with the equally weighted average of 11 other models.

Dot plot of GPT-6 Astra’s percentage-point difference from the equally weighted average of 11 other models across all 29 tagged SNAP policy categories in the original no-search runs. Each category label includes its question count, ranging from 4 to 60. Astra led most on transitional benefits, by 25.3 points across 4 questions, and trailed most on self-employment, by 3.1 points across 19 questions. Four categories were tied.

(Please use a modern browser to see the interactive version of this visualization)

Question counts appear beside each category. Gray dots mark no difference; green dots show Astra’s result. Categories contain 4 to 60 questions, so results for smaller categories are less stable. Original no-search runs used the same 500 questions; eight untagged questions are excluded.

Chart: Nava LabsSource: Nava Labs analysis of the Better Government Lab SNAP scenarios datasetGet the dataDownload image