OpenParable
Skip to content
← Back to skills

Pairwise Ranking

Productivity
pairwise-ranking

Pairwise Ranking is an Agent Skill for productivity published in the yogsoth-ai/de-anthropocentric-research-engine repository on GitHub: a SKILL.md instruction file an AI agent can load to do this task.

Pairwise Ranking Campaign — produce global rankings through pairwise comparisons and voting aggregation using Bradley-Terry, Elo, TrueSkill, Condorcet, Borda, Schulze methods.

View on GitHub ↗
QUICK START

How to use this skill

Bring this guide into your coding agent with a prompt tailored to the tool you use.

  1. Open your project in Codex.
  2. Copy and paste the prompt below.
  3. Review files and risks before approving any action.
Prompt to paste
Inspect the Agent Skill at https://github.com/yogsoth-ai/de-anthropocentric-research-engine/blob/HEAD/skills/pairwise-ranking/SKILL.md and all supporting files. Report provenance, permissions, scripts, dependencies, and security risks. Propose an exact installation into .agents/skills/pairwise-ranking, but wait for my approval before writing files or running scripts.

Copying this prompt does not install or run the skill. Review third-party files before use.

Pairwise Ranking

Produce global rankings from pairwise comparisons. This campaign orchestrates comparison collection, rating computation, multi-judge aggregation, and consistency verification to yield robust ordinal rankings with confidence estimates.

Strategy Routing

SignalStrategy
Small N precise comparison / 5-15 options / careful calibrationdeliberative-calibration
Large N sparse / 100+ options / can only compare subsetefficient-exploration
Multi-judge / committee / multi-judge / LLM judge aggregationcollective-adjudication
Continuous update / Elo / live rating / A/B testingdynamic-tracking
Consistency audit / cycle detection / transitivity checkcoherence-diagnosis

Manifest

Strategies

StrategyMethodsWhen
deliberative-calibrationBradley-Terry, Thurstone, AHP pairwise, BordaSmall N complete comparison
efficient-explorationBT incomplete, TrueSkill, Active learning, Rank CentralityLarge N sparse matrix
collective-adjudicationCondorcet/Schulze, Borda, Kemeny-Young, CopelandMulti-judge aggregation
dynamic-trackingElo, Glicko-2, TrueSkill 2, Whole-History RatingContinuous rating update
coherence-diagnosisConsistency Ratio, cycle enumeration, mEloPreference consistency check

Tactics

TacticPurpose
adaptive-pair-selectionIteratively select maximally informative pairs, compare, update ratings, check convergence
multi-judge-aggregationCollect independent ballots from multiple judges, aggregate, identify disagreement
consistency-audit-loopDetect cycles, localize inconsistencies, request corrections, recompute

SOPs

SOPInputOutput
pair-selectorcurrent_ratings, comparison_historynext_pairs[]
comparison-executorpair, contextjudgment(winner, confidence, reasoning)
rating-updatejudgment, current_ratings, methodupdated_ratings
convergence-checkrating_historyconverged(bool), stability_score
ballot-collectioncandidates[], perspectives[]ballots[]
aggregation-methodballots[], methodconsensus_ranking
cycle-detectioncomparison_matrixcycles[], transitivity_score
inconsistency-localizationcomparison_matrix, cycles[]problematic_pairs[]
ranking-synthesisratings, consistency_reportfinal_ranking

Budget Table (M tier)

DimensionThreshold
Comparison pairs>= N*log(N) pairs (N=candidate count)
Judge count (collective)>=3 independent perspectives
Consistency checkCR < 0.1 or equivalent threshold
Convergence criterionranking stability >= 90%

MCP Tools

  • mcp__wiki-vault__vault_search — retrieve candidate descriptions and prior rankings
  • mcp__wiki-vault__vault_add_edge — record ranking relationships
  • mcp__wiki-vault__vault_query_graph — check existing preference edges

Context Management

  • State is maintained in a ranking_state ledger passed between tactics
  • Each SOP receives only its required inputs (no full state leakage)
  • Convergence check gates iteration termination
  • Final synthesis produces the deliverable ranking artifact

Available Strategies

Optional, no fixed order; the final leaf is always a sop.

StrategyWhen to use
coherence-diagnosisStrategy for auditing preference consistency using Consistency Ratio, cycle enumeration, and mElo to detect and resolve intransitivities.
collective-adjudicationStrategy for multi-judge ranking aggregation using Condorcet, Schulze, Borda, Kemeny-Young, and Copeland methods to produce consensus rankings from diverse perspectives.
deliberative-calibrationStrategy for small-N complete pairwise comparison using Bradley-Terry, Thurstone, AHP, and Borda methods to produce calibrated rankings.
dynamic-trackingStrategy for continuous rating updates using Elo, Glicko-2, TrueSkill 2, and Whole-History Rating for live ranking systems and A/B testing.
efficient-explorationStrategy for large-N sparse pairwise comparison using TrueSkill, active learning, and rank centrality to rank 100+ candidates from limited comparisons.

Available SOPs

Optional, no fixed order; the final leaf is always a sop.

SOPWhen to use
context-checkpointAppend research process and results to the current Phase's context file. Each append MUST contain >=500 lines of markdown covering both process and results. Use this skill at plan-designated checkpoint points — typically after each strategy completes or at key decision nodes within a research Phase.
context-initCreate a new context file for a research Phase. Called once at Phase start to initialize the file that subsequent context-checkpoint calls will append to. Use this skill whenever a new research Phase begins and a fresh context file is needed.
convergence-saturation-detectionDetermines when to stop iterating — coverage threshold met or marginal returns diminishing. Shared across all campaigns.