LangSmith MCP Server
Capped at 70: no passing tool call yet; the tests add up to 75. Why scores are capped
Test points add up to 75; the score is capped at 70 until a tool call passes. Not a quality or security rating. Methodology
Test points before the cap (75/100)
- Install20/20
- Handshake30/30
- Maintenance0/25
- Docs15/15
- License10/10
Tested pypi langsmith-mcp-server 0.1.1 (MIT; repo langchain-ai/langsmith-mcp-server). Installed pinned with scripts off (uv venv, Python 3.13) and launched: initialize and tools/list over stdio succeeded with no credentials and listed 15 tools (no tool annotations). Real calls: 0 of 3 real function calls passed. Calls need a LangSmith API key. Core tools not tested: 70 cap.
Tool-call test (2026-11-13 UTC, direct MCP stdio client, no LLM): fail. 0 of 3 real function calls passed. Calls need a LangSmith API key. Evidence: local test run, Nov 13, 2026 UTC.
Transport (tested 2026-11-13 UTC): stdio
Token cost (estimate): approx. 4799 tokens, cl100k estimate (15 tools, measured 2026-11-13 UTC, version 0.1.1). This is an estimate from our testing of the tool names, descriptions and input schemas, counted with cl100k_base.
Permissions: 7 read, 7 write (state-changing), 1 unclassified · Auth: API key · Rate limit: unknown · Pricing unit: unknown
Auth: API key. Rate limit: unknown. Pricing unit: unknown.
create_dataset— write (state-changing)fetch_runs— write (state-changing)get_billing_usage— readget_prompt_by_name— readget_thread_history— write (state-changing)list_datasets— readlist_examples— readlist_experiments— readlist_projects— readlist_prompts— readpush_prompt— write (state-changing)read_dataset— write (state-changing)read_example— write (state-changing)run_experiment— unclassifiedupdate_examples— write (state-changing)
listing-flags calls_need_key or auth_kind
Score 70/100 · Tools 15 · Token cost approx. 4,799 tokens · Compare · Last tested 2026-11-13 UTC · Version tested 0.1.1 · Package langsmith-mcp-server · Repo langchain-ai/langsmith-mcp-server
Summary: LangSmith MCP Server completed an MCP handshake without credentials and exposed 15 tools; it is deprecated or archived, so prefer a maintained alternative.
Evidence summary
- Handshake
- pass
- Tools listed
- 15
- Context cost
- approx. 4799 tokens, cl100k estimate (15 tools, measured 2026-11-13 UTC, version 0.1.1; an estimate) ranking · add up your servers' context cost
- API key
- Connects without a key; tool calls need one
- Tested
- 2026-11-13 UTC
Test results
| Test | Result | Points | Evidence |
|---|---|---|---|
| install | pass | 20 | uv pip install langsmith-mcp-server (Python venv, Linux) succeeded in 1s. |
| handshake | pass | 30 | MCP initialize + tools/list over stdio succeeded with no credentials; 15 tools listed in 7.9s. |
| maintenance | fail | 0 | GitHub repo langchain-ai/langsmith-mcp-server is archived. |
| docs | pass | 15 | README over 3,000 chars; install instructions present; mcpServers config example present. Check of the package's own README (PyPI long_description (package README)). |
| license | pass | 10 | MIT (OSI-approved). |
Local-model readiness
Token cost (estimate): approx. 4799 tokens, cl100k estimate for 15 tools · about 59% of an 8k context. This is an estimate from our testing (counted with cl100k_base).
Small-model friendly (schema heuristic, not a model result): unknown.
Not tested
Small-model task runs: Not yet tested.
What we tested and what we didn't
Tested on 2026-11-13 UTC
- Install from the registry
- We connected with no credentials and asked for its tool list: it listed 15 tools (Method details: MCP
initializethentools/list) - Maintenance signals (last push, archived/deprecated status)
- Docs (README length, install and config example)
- License (OSI-approved or not)
- Token cost (estimate)
Not tested
- Security: not audited (no code review, no permission or data-flow analysis)
- Real tasks (no functional task runs yet)
- Behaviour with real credentials or a real account
- Performance, rate limits or load
- Correctness of results beyond the listed tasks
Methodology: how we test. A high score means it installs and responds, not that giving it access to your data carries no risk.
Known security issues
Not yet checked.
Monthly roundup: MCP security issues, October 2026.
Tool list changes
Baseline recorded from the 2026-11-13 UTC test run; changes appear after the next test.
Snapshot hash (fingerprint of the tool list)
6cdbeb48346c: SHA-256 of the canonical tool names, descriptions and input schemas. If it changes, the tool list changed.
Raw test record (JSON)
{
"name": "langsmith-mcp-server",
"version": "0.1.1",
"registry": "pypi",
"status": "working",
"tested": "2026-11-13",
"tools_count": 15,
"deprecated": false,
"archived": true,
"tests": {
"install": [
"pass",
20,
"`uv pip install langsmith-mcp-server` (Python venv, Linux) succeeded in 1s."
],
"handshake": [
"pass",
30,
"MCP `initialize` + `tools/list` over stdio succeeded with no credentials; 15 tools listed in 7.9s."
],
"maintenance": [
"fail",
0,
"GitHub repo langchain-ai/langsmith-mcp-server is archived."
],
"docs": [
"pass",
15,
"README over 3,000 chars; install instructions present; `mcpServers` config example present. Check of the package's own README (PyPI long_description (package README))."
],
"license": [
"pass",
10,
"MIT (OSI-approved)."
]
}
}Install
uvx langsmith-mcp-server
Pin the version we tested
uvx langsmith-mcp-server==0.1.1
pip install langsmith-mcp-server==0.1.1
tested 2026-11-13 UTC. Newer versions may differ and were not tested by us.
{
"mcpServers": {
"langsmith-mcp-server": {
"command": "uvx",
"args": [
"langsmith-mcp-server==0.1.1"
]
}
}
}Tools listed
list_prompts, get_prompt_by_name, push_prompt, get_thread_history, fetch_runs, list_projects, get_billing_usage, list_experiments, list_datasets, list_examples, read_dataset, read_example, create_dataset, update_examples, run_experiment
Quick config
{
"mcpServers": {
"langsmith-mcp-server": {
"command": "uvx",
"args": [
"langsmith-mcp-server"
]
}
}
}Setup
- Install. Run
uv pip install langsmith-mcp-server(Python venv). This is the exact command we ran on 2026-11-13 UTC. - Configure. Add the mcpServers entry below to your MCP client config.
- Verify. Restart the client and check that tools/list returns tools (15 in our test).
{
"mcpServers": {
"langsmith-mcp-server": {
"command": "uvx",
"args": [
"langsmith-mcp-server"
]
}
}
}Test history
- 2026-11-13 UTC: New
- 2026-11-13 UTC: Deprecated
FAQ
Is LangSmith MCP Server deprecated?
Yes. langsmith-mcp-server has an archived GitHub repo (as of 2026-11-13 UTC).
What replaces LangSmith MCP Server?
None identified. We have no evidence of a maintained replacement yet.
Does LangSmith MCP Server need an API key?
It connects and lists 15 tools without a key, but its tool calls need one.
Embed the badge
Every tested server gets a free embeddable badge. It shows the current score and test date and expires after 90 days. Testing is free; payment never changes a score.