Wingetlywingetly
← All apps

LLM Test Bench

by aleksandrreva-engv1.0.0

Last updated

Desktop test bench that runs test-case suites against a local llama.cpp server and compares the runs

Install with winget

$ winget install --id aleksandrreva-eng.llm-test-bench --exact --version 1.0.0

Run in Command Prompt, PowerShell, or Windows Terminal. Prompts for any agreements.

Silent install command for LLM Test Bench

LLM Test Bench uses MSI (WiX). The silent install switches are /quiet /norestart.

One-line silent install (x64, user scope)
msiexec.exe /i LLMTestBench-1.0.0.msi /quiet /norestart

See the full silent install reference for LLM Test Bench →

Built by Pckgr

For Intune admins

Stop chasing app updates. Pckgr patches them for you.

Automated application patching for Microsoft Intune. Pckgr keeps a curated library of 1,000+ apps continuously up-to-date in your tenant via Microsoft Graph - no manual repackaging, no chasing vendor sites.

Start free 30-day trial

From $49/month. No credit card required.

Not on Intune? Pckgr RMM deploys and patches apps like this one on any Windows fleet. Free for up to 5 devices.

About

A Windows desktop application for evaluating local GGUF models served by llama.cpp. It scans a models folder, starts or attaches to a llama-server on 127.0.0.1:8080, and runs configurable test-case suites covering logic, code, format compliance, instruction following, generation speed and long-context handling. Each answer is graded by an automatic check - substring, regex, JSON schema, code execution, LLM-as-judge or embedding similarity - with skipped checks kept out of the score instead of counted as failures. Runs are stored in a SQLite history with per-case drill-down and side-by-side comparison, and can be exported as an HTML report or printed. Also includes a local agent mode that uses the tested model as a subagent, and reports prefill and generation speed, time to first token, model memory usage and layer placement from the server log.

Installers · v1.0.0

ArchitectureTypeScopeInstallDownload
x64MSI
WiX
userDirect

Copy a command tailored to that specific architecture, type, and scope - useful when winget would otherwise pick a different default.

Security

No known CVEs for LLM Test Bench.

Coverage is best-effort and depends on a winget package mapping to an NVD CPE entry. Absence here is not a guarantee of safety.

Related apps

  • Claude CodeAnthropic PBC
    Anthropic.ClaudeCodev2.1.330

    Unleash Claude’s raw power directly in your terminal. Search million-line codebases instantly. Turn hours-long workflows into a single command. Your tools. Your workflow. Your codebase, evolving at thought speed.

  • ClaudeAnthropic, PBC
    Anthropic.Claudev6.28899.6

    Your AI partner on desktop. Fast, focused, and designed for deep work.

  • O
    Ollama (Portable)Ollama
    Ollama.Ollama.Portablev0.40.17

    Start building with open models.

  • LM StudioLM Studio
    ElementLabs.LMStudiov0.4.28+2

    Discover, download, and run local LLMs

  • C
    Codex CLIOpenAI, Inc.
    OpenAI.Codexv0.171.0

    Codex CLI is an open‑source local coding agent that runs in your terminal, letting you write, edit, and understand code without leaving the command line.

  • R
    rtkrtk-ai
    rtk-ai.rtkv1.2.1

    CLI proxy that reduces LLM token consumption by 60-90% on common dev commands. Single Rust binary, zero dependencies

More from aleksandrreva-eng or browse llm, llama-cpp, gguf.

Frequently asked questions

How do I install LLM Test Bench on Windows?
Open Windows Terminal, PowerShell, or Command Prompt and run: winget install --id aleksandrreva-eng.llm-test-bench --exact --version 1.0.0. winget downloads the installer from aleksandrreva-eng and runs it. Requires Windows 10 (1809+) or Windows 11.
How do I install LLM Test Bench silently for unattended deployment?
Add --silent and accept the agreements upfront: winget install --id aleksandrreva-eng.llm-test-bench --exact 1.0.0 --silent --accept-package-agreements --accept-source-agreements. This is the variant Intune, Configuration Manager, and other deployment tools should use.
What are the silent install switches for LLM Test Bench?
LLM Test Bench uses MSI (WiX). Run the downloaded installer with: msiexec.exe /i LLMTestBench-1.0.0.msi /quiet /norestart. The silent switches are /quiet /norestart.
How do I uninstall LLM Test Bench via winget?
Run: winget uninstall --id aleksandrreva-eng.llm-test-bench --exact. Add --silent for unattended uninstalls. winget will use the registered uninstaller from LLM Test Bench's Apps & Features entry.
Is LLM Test Bench free?
LLM Test Bench is distributed under MIT. Refer to the publisher (https://github.com/aleksandrreva-eng/llm-test-bench) for the full license terms - Wingetly itself does not charge for installation.
Does LLM Test Bench work on Windows 10?
Yes, as long as your Windows 10 build supports winget (1809 or newer). winget ships with App Installer on Windows 10/11 and pulls LLM Test Bench directly from the publisher.
How do I keep LLM Test Bench up to date?
Run winget upgrade --id aleksandrreva-eng.llm-test-bench --exact, or winget upgrade --all to update everything winget tracks. We index 1 version of LLM Test Bench from microsoft/winget-pkgs.