Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to content

closedloop-technologies/triumvirate

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

145 Commits
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

Triumvirate

Triumvirate Logo

npm version

CI

Triumvirate

License: MIT

Multi-model AI code reviews with consensus detection

Triumvirate is a CLI tool and GitHub Action that runs your codebase through multiple AI models (OpenAI, Claude, Gemini) simultaneously, then synthesizes their findings to surface issues with cross-model consensus. When multiple AI reviewers agree on a problem, you can be more confident it's a real issue worth fixing.

Triumvirate gives vibe-coders, agentic coders, and lean dev teams a whole-repo AI reviewβ€”uncovering subtle design flaws, logic slips, and cross-file inconsistencies that slip past linters and traditional static-analysis tools.

Who Is This For?

  • Vibe-coders & AI-assisted developers – Get a second (and third) opinion on AI-generated code
  • Solo developers – Professional-grade code reviews without a team
  • Lean teams – Automated quality gates that scale with your codebase
  • CI/CD pipelines – Integrate multi-model reviews into pull request workflows

How It Works

flowchart TB
    subgraph Entry["πŸš€ Entry Points"]
        CLI["πŸ’» CLI<br/><code>tri review</code>"]
        PR["πŸ”€ Pull Request<br/>GitHub Action"]
        CI["βš™οΈ CI/CD Pipeline<br/>Scheduled/Manual"]
    end

    subgraph Prepare["πŸ“¦ Preparation"]
        Repomix["Repomix<br/>Package Codebase"]
        Diff["Git Diff<br/>(--diff mode)"]
    end

    subgraph Review["πŸ€– Multi-Model Review"]
        OpenAI["OpenAI<br/>GPT-4 / O3"]
        Claude["Anthropic<br/>Claude"]
        Gemini["Google<br/>Gemini"]
    end

    subgraph Analyze["πŸ” Analysis"]
        Consensus["Cross-Model<br/>Consensus Detection"]
        Categorize["Categorize<br/>Findings"]
    end

    subgraph Output["πŸ“Š Output"]
        Report["Enhanced Report<br/>JSON + Markdown"]
        Badge["README Badge<br/>(--badge)"]
        PRComment["PR Comments<br/>Summary + Inline"]
        Plan["Task Plan<br/><code>tri plan</code>"]
    end

    CLI --> Repomix
    CLI --> Diff
    PR --> Diff
    CI --> Repomix

    Repomix --> OpenAI & Claude & Gemini
    Diff --> OpenAI & Claude & Gemini

    OpenAI & Claude & Gemini --> Consensus
    Consensus --> Categorize

    Categorize --> Report
    Report --> Badge
    Report --> PRComment
    Report --> Plan

    style CLI fill:#4CAF50,color:#fff
    style PR fill:#2196F3,color:#fff
    style CI fill:#9C27B0,color:#fff
    style OpenAI fill:#10a37f,color:#fff
    style Claude fill:#d97706,color:#fff
    style Gemini fill:#4285F4,color:#fff
    style Consensus fill:#FF5722,color:#fff
Loading

Features

  • Multi-model Analysis - Compare insights from OpenAI, Claude, and Gemini models
  • Cross-model Consensus - Identify findings that multiple models agree on with clear agreement indicators (🚨 high, ❗ partial, ⚠️ low)
  • Specialized Reviews - Conduct focused reviews for security, performance, architecture, and documentation
  • Actionable Tasks - Generate prioritized improvement tasks with dependencies
  • CI/CD Integration - Use as a GitHub Action in your workflow
  • Cost Transparency - View detailed API usage and cost breakdown for each model
  • Comprehensive Reports - Get categorized findings with detailed explanations

Installation

# Install globally
npm install -g @justbuild/triumvirate@latest

# Or use directly with npx
npx @justbuild/triumvirate

# For development/contributing
git clone https://github.com/closedloop-technologies/triumvirate.git
cd triumvirate
npm install
npm run build

After installing globally, you can run the tool using either tri or triumvirate commands.

Quick Start

Set up API Keys

  1. Create a .env file in your project root:

    cp .env.example .env
  2. Add your API keys:

    OPENAI_API_KEY=your-openai-key
    ANTHROPIC_API_KEY=your-anthropic-key
    GEMINI_API_KEY=your-google-key

Basic Usage

tri review

# Get list of models from running
tri models
# Run a review with specific models
tri review --models openai/o3,anthropic/claude-opus-4-20250514,gemini/gemini-2.5-pro-preview-06-05

# For local development/testing
npm run dev review

Default output location: Review artifacts (JSON, Markdown) are saved in the .triumvirate/ directory within your project root, named with a timestamp (e.g., .triumvirate/tri-review-2024-08-15T103000Z.md). Use the -o option to specify a different file or directory.

Output format: The review process provides real-time progress indicators and generates a comprehensive report with:

  • Categories of findings (e.g., Code Quality, Error Handling, Security)
  • Specific findings with agreement levels across models (🚨 high, ❗ partial, ⚠️ low)
  • Distribution of findings by category
  • API usage summary with detailed cost breakdown

Cost information: The tool provides transparency about API usage costs, showing:

  • Total cost across all models
  • Token usage (input and output)
  • Per-model cost breakdown

CLI Reference

Triumvirate provides a command-line interface for running code reviews.

tri <command> [options]
Command Description
review Run code reviews across selected models
summarize Create a summary from raw review output
plan Break a summary into tasks with dependencies
next Display the next available task
models List all supported models and estimated prices
install Install shell completion

Run tri --help to see all available options.

Review Command Options

tri review [options]

Model Options

  • -m, --models <models> - Comma-separated list of models (default: openai,claude,gemini)
  • --fail-on-error - Exit with non-zero code if any model fails
  • --skip-api-key-validation - Skip API key validation check
  • --enhanced-report - Generate enhanced report with model agreement analysis (default: true)
  • --summary-only - Only include summary in results

Output & Formatting Options

  • -o, --output <file> - Specify the output file or directory (defaults to .triumvirate/)
  • --style <type> - Specify the output style (xml, markdown, plain)
  • --output-show-line-numbers - Add line numbers to each line in the output

Filter Options

  • --include <patterns> - List of include patterns (comma-separated)
  • -i, --ignore <patterns> - Additional ignore patterns (comma-separated)
  • --diff - Only review files changed in git diff

Task & Context Options

  • --docs <paths...> - List of documentation file paths to include as context
  • --task <description> - Specific task or question to guide the review

Processing & Review Options

  • --token-limit <number> - Maximum tokens to send to the model
  • --token-count-encoding <encoding> - Specify token count encoding
  • --compress - Perform code compression to reduce token count
  • --remove-comments - Remove comments from code
  • --remove-empty-lines - Remove empty lines from code
  • --top-files-len <number> - Specify the number of top files to include

Model & Threshold Options

  • --agent-model <model> - Specify the LLM for report analysis and planning (default: claude)
  • --output-dir <dir> - Specify the output directory (default: ./.triumvirate)
  • --pass-threshold <threshold> - Set review pass/fail threshold (strict, lenient, none)

Summarize Command Options

tri summarize [options]
  • -i, --input <file> - Input file containing raw reports
  • -o, --output <file> - Output file for the summary
  • --enhanced-report - Generate enhanced report with model agreement analysis

Plan Command Options

tri plan [options]
  • -i, --input <file> - Input file containing the summary (if not specified, will use the latest output from tri review)
  • -o, --output <file> - Output file for the plan
  • --agent-model <model> - Specify the LLM for task generation (default: claude)
  • --task <description> - Specific task or focus to guide the task generation
  • --output-dir <dir> - Specify the output directory (default: ./.triumvirate)

Next Command Options

tri next [options]
  • -i, --input <file> - Input file containing the plan

Global Options

  • -v, --version - Show version information
  • --verbose - Enable verbose logging for detailed output
  • --quiet - Disable all output to stdout

Advanced Examples

Focused Security Review

tri review --task "security focused code review" --output security-review.json

Only Review Changed Files

tri review --diff --models openai

Focus on Specific Files with Compression

tri review --include "src/**/*.js,src/**/*.ts" --compress

Generate Plan from Existing Summary

# Basic plan generation using the latest review summary
tri plan

# Basic plan generation with explicit input and output files
tri plan --input summary.md --output plan.json

# Generate plan with a specific LLM and task focus
tri plan --agent-model openai --task "Improve error handling and add tests"

Understanding the Output

When running tri review, you'll see output similar to this:

πŸ“¦ Triumvirate v0.4.0

Checking API keys for models: openai, claude, gemini
βœ… API key validation passed.

... [processing indicators] ...

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚          β–ˆβ–“β–’β–‘  21 FINDINGS EXTRACTED β–‘β–’β–“β–ˆ           β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
Key Findings:          08 βœ… | 13 ❌
Improvement Agreement: 03 🚨 | 07 ❗ | 03 ⚠️
🚨 3 findings have high agreement across models
1. Inconsistent Error Handling
2. Lack of Global Configuration System
3. Path Sanitization and Security Risks

... [additional findings] ...

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚             β–ˆβ–“β–’β–‘ API USAGE SUMMARY β–‘β–’β–“β–ˆ             β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
 Total API Calls:           9  
 Total Cost:           $0.5028          
 Total Tokens:         200568  (183630 input, 16938 output)

The findings are categorized by agreement level:

  • 🚨 High Agreement: Issues identified by all models
  • ❗ Partial Agreement: Issues identified by multiple but not all models
  • ⚠️ Low Agreement: Issues identified by only one model

The API usage summary provides transparency about the cost of the review process.

Get Next Task

tri next --input plan.json

GitHub Actions Integration

Triumvirate can automatically post PR summary comments and inline review comments on your pull requests.

Basic Setup with PR Comments

name: Code Review

on:
  pull_request:
    branches: [main]

jobs:
  review:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      pull-requests: write  # Required for posting comments
    steps:
      - uses: actions/checkout@v3
      - uses: actions/setup-node@v3
        with:
          node-version: '20'
      - run: npm install -g @justbuild/triumvirate
      - run: |
          tri review --models openai,claude,gemini \
            --diff \
            --output-dir .triumvirate \
            --fail-on-error \
            --pass-threshold lenient
        env:
          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
          ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
          GEMINI_API_KEY: ${{ secrets.GEMINI_API_KEY }}

Using the Triumvirate Action (with PR Comments)

For automatic PR comments, use the built-in action:

- uses: closedloop-technologies/triumvirate/.github/actions/triumvirate-review@main
  with:
    mode: strict                    # 'normal' or 'strict'
    post_summary: 'true'            # Post summary comment on PR
    post_inline_comments: 'true'    # Post inline comments on specific lines
    github_token: ${{ secrets.GITHUB_TOKEN }}
  env:
    OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
    ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
    GEMINI_API_KEY: ${{ secrets.GEMINI_API_KEY }}

PR Comment Features

Feature Description
Summary Comment Posts a summary of findings with agreement levels, top issues, and cost breakdown
Inline Comments Posts comments directly on the lines where issues were found
Agreement Indicators 🚨 High (all models), ❗ Partial (2 models), ⚠️ Low (1 model)
Auto-update Re-running the action updates existing comments instead of creating duplicates

Action Inputs

Input Description Default
mode Execution mode: normal or strict normal
post_summary Post a summary comment on the PR true
post_inline_comments Post inline comments on specific lines true
github_token GitHub token for posting comments ${{ github.token }}

Roadmap

See ROADMAP.md for planned features and improvements.

License

MIT

About

Let the Triumvirate judge your code. Entire repo code review by 3 large language models

Resources

Contributing

Stars

Watchers

Forks

Packages

 
 
 

Contributors