How to Connect Scrapeless to Grok: MCP Connector Setup

TL;DR:

• Grok's CLI speaks MCP, so connecting Scrapeless is one TOML table: a url and an x-api-token header.
• grok mcp doctor answers whether it worked — ✓ server started (1.0s), ✓ handshake OK (protocol 2025-06-18), ✓ 25 tools discovered.
• Scope decides whether the server runs at all. A repo-local .grok/config.toml in an untrusted folder reports ✗ folder untrusted and is counted as 0 servers; the same table at user scope starts.
• The header is x-api-token, not Authorization: Bearer. A Bearer header fails the handshake: grok mcp doctor reports ✗ handshake failed with HTTP 401.
• grok mcp add --header "..." writes the right config and puts your key in shell history; writing the table yourself does not.
• 25 tools discovered proves the header arrived, not that the key is valid — Scrapeless lists all 25 tools for any key value. One real tools/call that returns page content is the only proof the credential works.
• Get a key on the Scrapeless free plan first.

An agent in a terminal is good at reading files and running commands, and blind to anything on the open web. MCP is how that gap closes: the client mints tool definitions from a server, the model picks one mid-turn, and "what does this page say right now" becomes a call rather than a copy-paste.

Grok's CLI carries a first-class MCP implementation, including a diagnostic subcommand that reports which stage of a connection failed rather than a single red or green. Setting the connector up is two TOML tables; reading that diagnostic is the part that saves time later.

What You Get

Twenty-five tools, enumerated from a live tools/list rather than copied from documentation:

scrape_markdown covers most of what an agent asks for — one call, one document. The browser_* group is a session the model drives across several turns, which is what anything behind a click or a login needs.

Prerequisites

• The Grok CLI. The version used throughout here is grok 0.2.118 (1e1687c1cf).
• A Scrapeless API key.
• Nothing to install for the server. It is hosted, so there is no package and no local process — the client connects to a URL over streamable HTTP, one of the two transports the Model Context Protocol specification defines.

Step 1: Add the Server

The CLI has a subcommand for it:

grok mcp add -t http -s user scrapeless https://api.scrapeless.com/mcp \
  --header "x-api-token: YOUR_SCRAPELESS_API_KEY"

Added HTTP MCP server 'scrapeless' with URL: https://api.scrapeless.com/mcp to user config
File modified: ~/.grok/config.toml

-t http selects the transport (the alternatives are stdio and the deprecated sse), and -s user writes to ~/.grok/config.toml rather than the repository.

What it writes is a pair of TOML tables:

[mcp_servers.scrapeless]
url = "https://api.scrapeless.com/mcp"
enabled = true

[mcp_servers.scrapeless.headers]
x-api-token = "YOUR_SCRAPELESS_API_KEY"

Knowing that shape matters, because the --header flag puts your key in shell history and in the process list while the command runs. Writing those six lines yourself avoids both, and it lets you add the timeouts the CLI does not set:

[mcp_servers.scrapeless]
url = "https://api.scrapeless.com/mcp"
startup_timeout_sec = 30
tool_timeout_sec = 120

[mcp_servers.scrapeless.headers]
x-api-token = "YOUR_SCRAPELESS_API_KEY"

The header name is the detail to get right. Scrapeless reads x-api-token; most MCP examples show Authorization: Bearer because that is what the HTTP authentication framework specifies for bearer credentials. A Bearer header here fails before the handshake completes — the initialize request comes back HTTP 401 Unauthorized: Missing x-api-token header, and the doctor counts the server as failing.

Step 2: Read the Diagnostic

This is the part worth learning. grok mcp doctor reports every stage separately:

MCP Doctor

  Config sources
    ~/.grok/config.toml                      1 server
    ~/.claude.json                           not found
    .mcp.json                                not found
    grok.com                                 skipped (not logged in)

  scrapeless (http: https://api.scrapeless.com/mcp)
    ✓ server started (1.0s)
    ✓ handshake OK (protocol 2025-06-18)
    ✓ 25 tools discovered

Four independent facts in that output. Which config files were read and how many servers each contributed. Whether the connection opened, and how long it took. Whether the MCP handshake completed, and at which protocol version. And how many tools came back from discovery.

Those last two are ordinary JSON-RPC 2.0 exchanges — an initialize request followed by a tools/list — which is why they can succeed or fail independently of each other.

A failure at any one of those points to a different cause, which is why the staged output beats a single red or green. There is also a --json mode when you want to assert on this in a script rather than read it.

grok mcp list is the quicker check once it is working:

  scrapeless: https://api.scrapeless.com/mcp

Step 3: Understand Scope, Or It Will Not Start

Grok reads MCP config from user scope and from a repo-local .grok/config.toml. The second one has a condition attached that produces a confusing first run.

The same server table, placed in a project folder:

  Config sources
    ~/.grok/config.toml                      not found
    /root/verify-grok-proj/.grok/config.toml 0 servers

  scrapeless (http: https://api.scrapeless.com/mcp)
    ✗ folder untrusted (repo-local (project-scoped) server not started for an untrusted folder)

Two things to notice. The server did not start — a project-scoped MCP server is not launched until the folder is trusted, because a config file in a checked-out repository can otherwise point your agent at any endpoint the author chose. And the config-source line reads 0 servers even though the file defines one, so counting sources is not enough to tell you the config was accepted.

Use user scope for a key that is yours. Use project scope to share a server with a team, and expect the folder-trust step per machine.

Step 4: Confirm the Capability, Not the Badge

25 tools discovered is the result of a tools/list, and that call is answered by the MCP server itself — it never reaches the upstream API. So discovery succeeds whether or not the credential behind it is any good.

That is not a theoretical distinction. A routing gateway fronting this same endpoint with a stale stored token discovered its full toolset and then returned an invalid-token error on the first real call, while the same endpoint with a working key returned HTTP 200.

The check that settles it is one tool call:

initialize   HTTP 200   server=scrapeless-mcp-server v0.2.0
tools/list   HTTP 200   25 tools
tools/call scrape_markdown  HTTP 200  8940 chars of page content

Page content in that result is the evidence. Everything before it is setup reporting on itself: with a wrong key the call still returns HTTP 200, without an isError flag, and its text starts with Failed to fetch data.

Note: driving that call from inside a Grok turn needs xAI authentication, which this walkthrough's environment did not have — grok -p "..." returns Not signed in. The connector, the handshake, the discovery and the tool call above are all verified; the final model-authored turn is the one step taken on trust here. Sign in with grok login or set XAI_API_KEY and the same tools are available to the model.

Step 5: Prompt It

Once the tools are discovered, the model chooses among them. Naming the tool removes a round of guessing:

Use the scrapeless scrape_markdown tool on
https://books.toscrape.com/catalogue/category/books/mystery_3/index.html
and give me the first five titles with their prices as a table.

Two habits help. Name the tool when the job is one fetch, and describe the sequence when it is not — the browser_* tools share a session, so "create a session, go to the URL, click the filter, then read the text" is a different instruction from four unrelated ones.

And ask for the output shape you want. scrape_markdown returns a document; whether you get a table or a paragraph is decided by the prompt, not the tool.

Setting this up now? The Scrapeless free plan covers enough calls to get through the handshake and the first few tool calls.

What Comes Back

scrape_markdown returns the page as Markdown in the content block:

Response:  "-   [Home](https://books.toscrape.com/index.html)
-   [Books](https://books.toscrape.com/catalogue/category/books_1/index.html)
...

Markdown rather than HTML is the right default for a model. The same page is 8,940 characters from scrape_markdown against 53,800 from scrape_html, so scrape_html spends roughly six times the context on markup nobody reads. Use scrape_html when your own code will parse the result, and scrape_markdown when the model is the consumer.

A Router Changes the Tool Count

If the client points at a gateway that fronts several MCP servers behind one URL, the discovered list is the router's own dispatch tools rather than the provider's. The same client, same command: 3 tools through a smart-routing gateway, 25 against https://api.scrapeless.com/mcp directly.

Both arrangements are legitimate. A router keeps one credential and one audit trail across many providers; a direct connection gives the model the real tool surface. grok mcp doctor's tool count tells you which one you are running, which is reason enough to read it after any config change.

For the same product driven from code instead of an agent, our Grok web scraping guide covers the model-plus-fetch pattern, and the MCP server launch post covers what the server exposes. The Scraping API page describes the actor family behind these tools, the docs carry the per-actor reference, and pricing lists what a call costs.

Conclusion

Two TOML tables and a header name is the whole connector. grok mcp doctor then tells you which of the four stages worked, and its 25 tools discovered line is the one to check after any change — because 3 means you are talking to a router and 0 servers from a file that has one means the folder is untrusted.

The two mistakes worth avoiding are both cheap. Use x-api-token rather than a Bearer header, since the Bearer version is rejected with a 401 during the handshake and grok mcp doctor flags it immediately. And treat discovery as setup reporting on itself: one tools/call returning real page content is what actually proves the credential works.

Ready to give Grok a fetch it can call? Start with the Scrapeless free plan and add the server.

FAQ

Q: Does Grok support MCP servers?

Yes. The CLI has a dedicated grok mcp subcommand with add, list, remove, enable, disable and doctor, and it supports the stdio, http and sse transports. Remote HTTP is the one to use for a hosted server like this, since it needs no local process.

Q: How do I add the Scrapeless MCP server to Grok?

grok mcp add -t http -s user scrapeless https://api.scrapeless.com/mcp --header "x-api-token: ...", or write the equivalent [mcp_servers.scrapeless] and [mcp_servers.scrapeless.headers] tables into ~/.grok/config.toml yourself. The hand-written route keeps the key out of shell history and lets you set startup_timeout_sec and tool_timeout_sec.

Q: Why is my project-scoped MCP server not starting?

Because the folder is untrusted. A repo-local .grok/config.toml is not launched until you trust the folder, and the diagnostic says so explicitly: ✗ folder untrusted (repo-local (project-scoped) server not started for an untrusted folder). The config-source line also counts it as 0 servers, which makes the file look empty when it is not. Moving the entry to user scope avoids the gate when the key is yours.

Q: Should the header be x-api-token or Authorization: Bearer?

x-api-token. A request without it returns 401 Unauthorized: Missing x-api-token header. A Bearer header is rejected the same way at the handshake, so grok mcp doctor shows ✗ handshake failed and Found 0 healthy, 1 failing — the doctor catches the one-word mistake before any tool call.

Q: How do I check which tools Grok can see?

grok mcp doctor prints the discovered count per server, and --json gives the same thing machine-readably. Against this endpoint it reports 25. If you see 3, the client is pointed at a routing gateway rather than the server, and those three are the router's dispatch tools.

Q: Is 25 tools discovered enough to know it works?

No. Discovery is a tools/list, which the MCP server answers locally without contacting the upstream API, so it succeeds even against a rejected credential. Make one tool call and look for real page content; a bad key returns text starting with Failed to fetch data, and Scrapeless does not set isError on it.

Q: Do I need to be signed in to xAI for the connector to work?

The connector itself does not need it: the handshake and tool discovery run without any xAI credential. Having the model actually call a tool does, because that is a Grok inference turn — without it, grok -p "..." returns Not signed in. Run grok login or set XAI_API_KEY.

Q: Can I limit which tools the model may call?

Yes, on the client side. The CLI exposes permission allow and deny rules plus --tools and --disallowed-tools for built-ins, so a setup that only needs page content can permit scrape_markdown and leave the browser-session tools unavailable. Narrow it to the job.