Run Categories
Selected categories filter every dashboard view before model and batch selections are applied.
Dashboard Status Definitions
These status labels are used by every dashboard view except Key metrics. Key metrics uses separate detection TP/FN/FP/TN rules and treats a wrong attack label on an attacked PCAP as a detection true positive.
| Status | Meaning outside Key metrics |
|---|---|
| SUCCESS | The final answer exactly matches the configured expected answer: the correct Attack X detected, or No attack detected for a PCAP with no attack. |
| NOT DETECTED | The final answer is No attack detected while the configured expected answer is an attack. |
| WRONG | The final answer is an Attack X detected label, but it does not exactly match the configured expected answer. This includes wrong attack type and attack reported on a no-attack PCAP. |
| NO CONCLUSION | The run did not produce one of the strict accepted final answers, so it is kept separate from success, missed detection, and wrong-label counts. |
Model Ranking
Model x Prompt Variant Matrix
Each cell shows status rates within the selected model and prompt variant. The highlighted cell is the winning prompt for that model using success rate, then fewer not-detected, wrong, and no-conclusion outcomes.
Calculation: status count / investigations in the model-prompt cell. Tie-break: higher success rate, then lower not detected, lower wrong, lower no conclusion.
Selected Model Totals
Calculation: all investigations for each selected model across prompt variants, reasoning modes, batches, and configured attack runs.
Prompt Variant Totals
Aggregated across selected models and sorted by success rate with deterministic tie-breaks.
Calculation: all selected-model investigations for each prompt variant. Sorting uses higher success rate, then lower not detected, lower wrong, lower no conclusion.
Attack Totals
All selected models combined, sorted by success rate with deterministic tie-breaks.
Calculation: all selected-model investigations grouped by configured run and expected attack. Sorting uses higher success rate, then lower not detected, lower wrong, lower no conclusion.
Model x Attack Matrix
Calculation: status count / investigations for the selected model and attack. Highlight marks the winning attack cell for each model row by the dashboard tie-break rule.
Prompt Variant x Attack Matrix
Calculation: status count / investigations for the selected prompt variant and attack. Highlight marks the winning attack cell for each prompt row by the dashboard tie-break rule.
Category Comparison
Compares selected run categories after the current model and batch filters. The first table uses the dashboard's standard tie-break rule.
Attack Confusion Matrix
Rows are ground-truth attacks. Columns are strict final model outputs. Wrong predictions are counted on the ground-truth row and the predicted-attack column.
Calculation: each cell is count and row percentage for selected investigations with the same expected attack and predicted output. `No attack detected` and `No conclusion` are explicit prediction columns.
Wrong Attack Pair Leaderboard
Only attack-to-attack mistakes are shown here. Missed detections and no conclusions are excluded from this pair list because they are not confused with another attack label.
Calculation: selected investigations where expected attack differs from predicted Attack X. Sorted by count, then pair rate.
Duration Filters
These controls affect only request/time calculations. Accuracy, attack totals, and confusion scores remain unchanged.
Model Call And Time Histograms
Histograms use all selected-model investigations and do not remove rows hidden by the duration filter.
Number Of Calls
Average OpenCode model calls per investigation. Matrices and leaderboards are sorted from lowest to highest consumption.
Model x Prompt Variant
Model x Attack
Global Leaderboards
Calculation: average modelCallCount recovered from session artifacts for the selected investigations in each cell or leaderboard group.
Estimated Investigation Time
Average estimated processing seconds per investigation, derived from batch seconds per vLLM request and task model-call count.
Model x Prompt Variant
Model x Attack
Global Leaderboards
Calculation: average estimatedProcessingSeconds, where estimatedProcessingSeconds = modelCallCount * batchAverageSecondsPerModelRequest.
Token And Cost Controls
For Google, OpenAI, and Anthropic, token counts and cost use OpenCode session totals when present. Other providers use OpenCode step tokens. A pricing preset applies to every selected investigation, so select the matching model before estimating its cost.
Calculation: commercial rows prefer OpenCode `info.tokens` and `info.cost`. If OpenCode reports zero reasoning tokens but the export contains reasoning blocks, output tokens are split with a text-derived reasoning share while preserving the OpenCode total and cost. Non-commercial rows use step-finish token records; manual output pricing applies to output + reasoning tokens.
Token Averages By Model
Calculation: exact token fields averaged across tokenized investigations for each selected model.
Token Averages By Prompt Variant
Calculation: exact token fields averaged across selected models for each prompt variant.
Batch Token Totals
Calculation: one row per selected batch, summing exact investigation token counts from that batch.
High-Call And Excluded Investigations
Rows shown here are excluded from duration averages by unavailable timing, no-conclusion filtering, or the selected model-call range.
Calculation: selected-model investigations where durationMetricsExcluded is true, scoreStatus is NO CONCLUSION while that checkbox is enabled, or modelCallCount is outside the selected call range.
Excluded Model x Attack Calls
Calculation: average modelCallCount for excluded investigations grouped by model and ground-truth attack.
Batches
Calculation: one row per benchmark batch. Average seconds/request = batch wall-clock seconds / total model calls.
Matrix
Calculation: one row per completed run artifact after selected-model filtering.