LangSmith MCP Server

70/100

Capped at 70: no passing tool call yet; the tests add up to 75. Why scores are capped

Last tested 2026-11-13 UTC (Fresh: tested within 30 days) Connects · tools listed DeprecatedNo tool call tested yet Connects without a key; tool calls need one

Advisories: not yet checked

Security review: not tested

Test points add up to 75; the score is capped at 70 until a tool call passes. Not a quality or security rating. Methodology

Test points before the cap (75/100)

  • Install20/20
  • Handshake30/30
  • Maintenance0/25
  • Docs15/15
  • License10/10

Tested pypi langsmith-mcp-server 0.1.1 (MIT; repo langchain-ai/langsmith-mcp-server). Installed pinned with scripts off (uv venv, Python 3.13) and launched: initialize and tools/list over stdio succeeded with no credentials and listed 15 tools (no tool annotations). Real calls: 0 of 3 real function calls passed. Calls need a LangSmith API key. Core tools not tested: 70 cap.
Tool-call test (2026-11-13 UTC, direct MCP stdio client, no LLM): fail. 0 of 3 real function calls passed. Calls need a LangSmith API key. Evidence: local test run, Nov 13, 2026 UTC.

Transport (tested 2026-11-13 UTC): stdio

Token cost (estimate): approx. 4799 tokens, cl100k estimate (15 tools, measured 2026-11-13 UTC, version 0.1.1). This is an estimate from our testing of the tool names, descriptions and input schemas, counted with cl100k_base.

Permissions: 7 read, 7 write (state-changing), 1 unclassified · Auth: API key · Rate limit: unknown · Pricing unit: unknown

Auth: API key. Rate limit: unknown. Pricing unit: unknown.

  • create_dataset — write (state-changing)
  • fetch_runs — write (state-changing)
  • get_billing_usage — read
  • get_prompt_by_name — read
  • get_thread_history — write (state-changing)
  • list_datasets — read
  • list_examples — read
  • list_experiments — read
  • list_projects — read
  • list_prompts — read
  • push_prompt — write (state-changing)
  • read_dataset — write (state-changing)
  • read_example — write (state-changing)
  • run_experiment — unclassified
  • update_examples — write (state-changing)

listing-flags calls_need_key or auth_kind

Score 70/100 · Tools 15 · Token cost approx. 4,799 tokens · Compare · Last tested 2026-11-13 UTC · Version tested 0.1.1 · Package langsmith-mcp-server · Repo langchain-ai/langsmith-mcp-server

Summary: LangSmith MCP Server completed an MCP handshake without credentials and exposed 15 tools; it is deprecated or archived, so prefer a maintained alternative.

Evidence summary

Handshake
pass
Tools listed
15
Context cost
approx. 4799 tokens, cl100k estimate (15 tools, measured 2026-11-13 UTC, version 0.1.1; an estimate) ranking · add up your servers' context cost
API key
Connects without a key; tool calls need one
Tested
2026-11-13 UTC

Test results

TestResultPointsEvidence
installpass20uv pip install langsmith-mcp-server (Python venv, Linux) succeeded in 1s.
handshakepass30MCP initialize + tools/list over stdio succeeded with no credentials; 15 tools listed in 7.9s.
maintenancefail0GitHub repo langchain-ai/langsmith-mcp-server is archived.
docspass15README over 3,000 chars; install instructions present; mcpServers config example present. Check of the package's own README (PyPI long_description (package README)).
licensepass10MIT (OSI-approved).

Local-model readiness

Token cost (estimate): approx. 4799 tokens, cl100k estimate for 15 tools · about 59% of an 8k context. This is an estimate from our testing (counted with cl100k_base).

Small-model friendly (schema heuristic, not a model result): unknown.

Not tested

Small-model task runs: Not yet tested.

What we tested and what we didn't

Tested on 2026-11-13 UTC

  • Install from the registry
  • We connected with no credentials and asked for its tool list: it listed 15 tools (Method details: MCP initialize then tools/list)
  • Maintenance signals (last push, archived/deprecated status)
  • Docs (README length, install and config example)
  • License (OSI-approved or not)
  • Token cost (estimate)

Not tested

  • Security: not audited (no code review, no permission or data-flow analysis)
  • Real tasks (no functional task runs yet)
  • Behaviour with real credentials or a real account
  • Performance, rate limits or load
  • Correctness of results beyond the listed tasks

Methodology: how we test. A high score means it installs and responds, not that giving it access to your data carries no risk.

Known security issues

Not yet checked.

Monthly roundup: MCP security issues, October 2026.

Tool list changes

Baseline recorded from the 2026-11-13 UTC test run; changes appear after the next test.

Snapshot hash (fingerprint of the tool list)

6cdbeb48346c: SHA-256 of the canonical tool names, descriptions and input schemas. If it changes, the tool list changed.

Feeds: RSS · JSON.

Raw test record (JSON)
{
  "name": "langsmith-mcp-server",
  "version": "0.1.1",
  "registry": "pypi",
  "status": "working",
  "tested": "2026-11-13",
  "tools_count": 15,
  "deprecated": false,
  "archived": true,
  "tests": {
    "install": [
      "pass",
      20,
      "`uv pip install langsmith-mcp-server` (Python venv, Linux) succeeded in 1s."
    ],
    "handshake": [
      "pass",
      30,
      "MCP `initialize` + `tools/list` over stdio succeeded with no credentials; 15 tools listed in 7.9s."
    ],
    "maintenance": [
      "fail",
      0,
      "GitHub repo langchain-ai/langsmith-mcp-server is archived."
    ],
    "docs": [
      "pass",
      15,
      "README over 3,000 chars; install instructions present; `mcpServers` config example present. Check of the package's own README (PyPI long_description (package README))."
    ],
    "license": [
      "pass",
      10,
      "MIT (OSI-approved)."
    ]
  }
}

Install

uvx langsmith-mcp-server

Pin the version we tested

uvx langsmith-mcp-server==0.1.1

pip install langsmith-mcp-server==0.1.1

tested 2026-11-13 UTC. Newer versions may differ and were not tested by us.

{
  "mcpServers": {
    "langsmith-mcp-server": {
      "command": "uvx",
      "args": [
        "langsmith-mcp-server==0.1.1"
      ]
    }
  }
}

Tools listed

list_prompts, get_prompt_by_name, push_prompt, get_thread_history, fetch_runs, list_projects, get_billing_usage, list_experiments, list_datasets, list_examples, read_dataset, read_example, create_dataset, update_examples, run_experiment

Quick config

{
  "mcpServers": {
    "langsmith-mcp-server": {
      "command": "uvx",
      "args": [
        "langsmith-mcp-server"
      ]
    }
  }
}

Setup

  1. Install. Run uv pip install langsmith-mcp-server (Python venv). This is the exact command we ran on 2026-11-13 UTC.
  2. Configure. Add the mcpServers entry below to your MCP client config.
  3. Verify. Restart the client and check that tools/list returns tools (15 in our test).
{
  "mcpServers": {
    "langsmith-mcp-server": {
      "command": "uvx",
      "args": [
        "langsmith-mcp-server"
      ]
    }
  }
}

Test history

Follow this server (RSS) · Monthly changelog

FAQ

Is LangSmith MCP Server deprecated?

Yes. langsmith-mcp-server has an archived GitHub repo (as of 2026-11-13 UTC).

What replaces LangSmith MCP Server?

None identified. We have no evidence of a maintained replacement yet.

Does LangSmith MCP Server need an API key?

It connects and lists 15 tools without a key, but its tool calls need one.

Embed the badge

TestedMCP: 70/100, tested 2026-11-13 UTC

Every tested server gets a free embeddable badge. It shows the current score and test date and expires after 90 days. Testing is free; payment never changes a score.

Claim this listing · Report an issue