Methodology

Every test is automated and scripted; the harness is in our public repo so anyone can rerun it.

TestPointsHow
Install20npm: npm install <pkg> in a clean directory, Node 20. PyPI: uv pip install <pkg> into a fresh venv (Python 3.13). Linux, 4-minute timeout.
MCP handshake30Start the package's bin over stdio with no credentials, send initialize (protocol 2025-06-18) then tools/list, 45 s timeout. 30 = tools listed (working); 5 = auth-gated: the only output was a missing-credential message, so it is listed separately and not counted as working; 0 = exited or timed out without a clear reason, or needs a backing service.
Maintenance25GitHub pushed_at (or latest release date when no repo is found): ≤30 days 25, ≤90 days 15, ≤365 days 5. 0 if deprecated on npm, latest PyPI release yanked, or GitHub repo archived.
Docs15Repo README: over 3,000 characters (5), install instructions (5), an mcpServers config example (5). We check the package's own README (the README shipped in the npm tarball, or the PyPI long description), so monorepo packages are judged on their own subdirectory README, not the repo root.
License10OSI-approved license detected 10, otherwise 3.

Functional task tests (not scored)

For browser, filesystem, memory and fetch servers we also run key-free tasks and record pass/fail per task: write, read and list a temp file; store and retrieve a memory entity; fetch https://example.com/; navigate to https://example.com/ and read the page title. Tasks blocked by our test box (sandbox DNS, no bundled browser download, no backing database, browser extension required) are recorded as not run, not as failures. These results are shown on each listing but do not change the 100-point score.

Small-model friendliness (not scored)

Computed from each server's real tools/list response: tool count, total parameters (including nested object properties), required parameters, maximum nesting depth (1 = flat top-level params), and total/average tool description length in characters. Rating: low if more than 40 tools, nesting depth 4 or more, more than 150 total params, or more than 15,000 description characters; high if at most 15 tools, depth at most 2, at most 60 total params and average description at most 300 characters; otherwise medium. Servers without a tools/list response are unknown. These are heuristics for context-window and schema load on small (≤8B) models, not a measured model result.

What our score does not mean

What we do not test yet

Behavior with real credentials, security review, and performance. Functional tests cover only four categories. Scores therefore measure installability, availability and upkeep, not output quality.

Independence

Payment never changes a score. Featured placement is labeled. Vendors can request a retest.