zendriver + Scrapeless: Driving a Remote Browser Over CDP

TL;DR:

• zendriver is a community fork of nodriver that drives Chrome over raw CDP, and like nodriver it is built to launch the browser itself.
• Browser.start() has a connect_existing path, but discovery runs through HTTPApi, which hardcodes plain HTTP with a host and a port — a secure WebSocket endpoint carrying a token cannot be expressed that way.
• The real blocker is the tab address. zendriver builds every tab from a host, a port and a target id, and a remote endpoint serves no such route.
• One CDP URL means one browser: a second connection to the same endpoint gets a different remote browser and sees none of the first one's targets.
• The seam is flat-session multiplexing — Target.attachToTarget(flatten=True) returns a sessionId, and stamping it onto outgoing frames makes zendriver's own Tab helpers reach the remote page.
• With the session attached, a real extraction returned a fully rendered page and 20 product cards with correct titles and prices.
• Start on the Scrapeless free plan and drive a browser you did not have to launch.

The Scrapeless Scraping Browser is a cloud browser you connect to rather than launch. It listens on a secure WebSocket CDP endpoint, which is the same protocol zendriver speaks — so on paper the two should meet in one line.

They do not. Playwright has connect_over_cdp. Puppeteer has connect. zendriver has no equivalent, and the request for one has been sitting in the tracker since April 2025 as an open issue asking for exactly this, unanswered. The official raw-CDP tutorial covers page.send() and event handlers and never mentions remote browsers at all.

This guide works out what actually stands in the way — which is not the connection — and then attaches a remote browser using nothing but the library's own primitives.

How zendriver Starts a Browser

zendriver is a fork of nodriver created to merge unmerged bugfixes and reopen the project to contributions. Architecturally the two are the same idea: drive Chrome directly over the Chrome DevTools Protocol with no WebDriver layer in between, which removes the automation surface that driver-based stacks expose. The fork adds standard asyncio.run() entry, Docker support, and cookie persistence.

The normal entry point launches a browser:

import asyncio

import zendriver as zd


async def main():
    browser = await zd.start(headless=True, no_sandbox=True)
    page = await browser.get("https://books.toscrape.com/")
    await page.wait_for("h3 a", timeout=20)
    await page.sleep(2)
    print("cards:", len(await page.select_all("article.product_pod")))
    await browser.stop()


asyncio.run(main())

cards: 20

no_sandbox=True is needed when the process runs as root, which is the normal case inside a container; drop it on a desktop. The sleep call earns its place for a reason covered further down, and that reason applies equally to the remote path.

That spawns Chrome as a child process on whatever machine runs the script. Every zendriver guide in the result set works this way: drive a browser the script launched itself, and shape it through browser_args — a Chromium --proxy-server switch being the usual example. None of them attaches to a browser it did not spawn.

Prerequisites

• Python 3.10 or later.
• A Scrapeless API key from the dashboard, exported as SCRAPELESS_KEY.
• A local Chrome or Chromium install, only if you want to run the local-launch block above. The remote path needs no browser binary.

Install

pip install "zendriver==0.15.5"

export SCRAPELESS_KEY="your_api_key_here"

What connect_existing Actually Does

Reading Browser.start() gives the impression that remote attachment is already supported. When both config.host and config.port are set, it flips a flag and skips launching a process entirely. The installed package will show you:

import inspect
import textwrap

from zendriver.core.browser import Browser

source = inspect.getsource(Browser.start).splitlines()
start = next(i for i, line in enumerate(source) if "connect_existing = False" in line)
print(textwrap.dedent("\n".join(source[start:start + 6])))

connect_existing = False
if self.config.host is not None and self.config.port is not None:
    connect_existing = True
else:
    self.config.host = "127.0.0.1"
    self.config.port = util.free_port()

That is genuine — it does connect to a browser it did not start. The limitation is what comes next. Discovery goes through HTTPApi, which takes a host-and-port pair and interpolates it into a fixed address template, then fetches /json/version with urllib to learn the browser's WebSocket URL. You can see the shape of that address without leaving Python:

from zendriver.core.browser import HTTPApi

api = HTTPApi(("example-host", 9222))
print("scheme:", api.api.split("://")[0])
print("room for a query string:", "?" in api.api)
print("room for a path:", api.api.count("/") > 2)

scheme: http
room for a query string: False
room for a path: False

Three things break at once for a cloud endpoint. The scheme is fixed to plain http. The address is a host and a port, so there is nowhere to put a path, a query string, or an authentication token. And a hosted endpoint generally does not expose the CDP HTTP discovery routes to the public internet — requesting /json/version and /json/list on the Scrapeless host returns HTTP 403 in both cases.

So connect_existing is for a Chrome you started yourself with --remote-debugging-port, on a host and port you control. It is not a remote-attach feature.

The Real Blocker Is the Tab URL

Suppose discovery were solved. The connection still would not be usable, and the reason sits one layer below where most people look.

zendriver constructs tab connections by string-building them from the same host and port, interpolated together with the target id into a /devtools/page/<target_id> path. It does this in two separate places in browser.py, which you can confirm from the installed package:

import inspect

from zendriver.core import browser

source = inspect.getsource(browser)
print("tab addresses built from host and port:", source.count("{self.config.host}:{self.config.port}"))
print("devtools path template present:", "/devtools/" in source)

tab addresses built from host and port: 2
devtools path template present: True

A local Chrome serves that route. A cloud browser does not — it exposes one endpoint, and the per-target DevTools paths are not part of its public surface. Four plausible shapes were tried against a live session: with the token, without it, under the /api/v2/browser prefix, and as a /page/ route. All four were rejected with HTTP 404.

This is why the problem cannot be solved by finding the right URL. There is no per-tab URL to find.

One CDP URL, One Browser

The next instinct is to open a second connection for the tab. That does not work either, and it fails in a way that is quiet enough to waste an afternoon.

import asyncio
import os
from urllib.parse import urlencode

import zendriver as zd
from zendriver import cdp


def cdp_url():
    return "wss://browser.scrapeless.com/api/v2/browser?" + urlencode(
        {"token": os.environ["SCRAPELESS_KEY"], "sessionTTL": 300, "proxyCountry": "US"}
    )


async def main():
    a = zd.Connection(cdp_url())
    b = zd.Connection(cdp_url())

    tid = await a.send(cdp.target.create_target("https://books.toscrape.com/"))
    a_ids = {str(t.target_id) for t in await a.send(cdp.target.get_targets())}
    b_ids = {str(t.target_id) for t in await b.send(cdp.target.get_targets())}

    print("A sees its own target:", str(tid) in a_ids)
    print("B sees it:", str(tid) in b_ids)
    print("shared target ids:", len(a_ids & b_ids))

    await a.aclose()
    await b.aclose()


asyncio.run(main())

A sees its own target: True
B sees it: False
shared target ids: 0

Each connection to the endpoint is its own browser. The two sessions share nothing — not the target you just created, not a single target id. Anything built on "one socket per tab" is quietly driving a different browser than it thinks.

Note what did work, though. zd.Connection(cdp_url()) connected and answered CDP commands. The transport was never the problem.

Ready to drive a browser you did not have to launch? Create a free Scrapeless account and point zendriver at it.

Attach a Flat Session

Everything has to travel over the one connection, which is what the CDP Target.attachToTarget method is for. With flatten set, it returns a sessionId, and any frame carrying that sessionId is routed to the attached target instead of the browser.

zendriver does not use it. Every outgoing frame is serialized by one property, Transaction.message:

import inspect

from zendriver.core.connection import Transaction

body = inspect.getsource(Transaction.message.fget)
print(body.strip().splitlines()[-1].strip())
print("sessionId present:", "sessionId" in body)

return json.dumps({"method": self.method, "params": self.params, "id": self.id})
sessionId present: False

There is no sessionId field, so commands always land on the browser. Adding one is the whole integration: attach the session, then stamp it onto frames on the way out.

import asyncio
import json
import os
from urllib.parse import urlencode

import zendriver as zd
from zendriver import cdp

CDP_URL = "wss://browser.scrapeless.com/api/v2/browser?" + urlencode(
    {"token": os.environ["SCRAPELESS_KEY"], "sessionTTL": 300, "proxyCountry": "US"}
)


class ScrapelessTab(zd.Tab):
    """A zendriver Tab bound to a remote browser over a single websocket."""

    def __init__(self, url):
        super().__init__(url, target=None)
        self._session_id = None

    async def open_page(self, url):
        tid = await self.send(cdp.target.create_target(url))
        infos = await self.send(cdp.target.get_targets())
        self._target = next(t for t in infos if str(t.target_id) == str(tid))
        self._session_id = await self.send(
            cdp.target.attach_to_target(tid, flatten=True)
        )

        websocket, session_id = self.websocket, self._session_id
        original_send = websocket.send

        async def send_with_session(message, *args, **kwargs):
            frame = json.loads(message)
            frame.setdefault("sessionId", str(session_id))
            return await original_send(json.dumps(frame), *args, **kwargs)

        websocket.send = send_with_session
        return self


async def main():
    tab = ScrapelessTab(CDP_URL)
    await tab.open_page("https://books.toscrape.com/")
    print("session attached:", bool(tab._session_id))
    await tab.aclose()


asyncio.run(main())

session attached: True

Subclassing Tab rather than Connection is deliberate: Tab already carries every page helper, and it extends Connection, so one object holds one socket, one listener, and one response map. Passing target=None to the constructor is fine because the real target is assigned once it exists. Replies arrive with the same id they were sent with, so zendriver's existing dispatch resolves them untouched.

Run a Real Extraction

With the session attached, the ordinary API works. Two timing details bite first, and neither is specific to remote browsers — both behave the same against a locally launched one.

get_content() sends its command immediately rather than polling. Call it too early and it returns the empty document skeleton, 39 characters, with no error at all — which reads like a broken connection rather than a page that has not rendered.

wait_for() does poll, but it returns as soon as its selector matches once. On a page whose markup is still arriving, that is a weaker guarantee than it looks: selecting straight after wait_for("h3 a") returned 9 cards on one local run and 4 on the next, against 20 once the DOM settled. Give the page a moment before you count anything.

    await tab.wait_for("h3 a", timeout=20)
    await tab.sleep(2)

    html = await tab.get_content()
    print("fully rendered page:", len(html) > 40000)
    print("catalogue marker present:", "All products" in html)

    cards = await tab.select_all("article.product_pod")
    print("product cards:", len(cards))

    for card in cards[:5]:
        link = await card.query_selector("h3 a")
        price = await card.query_selector("p.price_color")
        print(f"  {link.attrs.get('title')} — {price.text}")

fully rendered page: True
catalogue marker present: True
product cards: 20
  A Light in the Attic — £51.77
  Tipping the Velvet — £53.74
  Soumission — £50.10
  Sharp Objects — £47.82
  Sapiens: A Brief History of Humankind — £54.23

Selectors and element queries behave exactly as they do against a local browser, because from zendriver's side nothing changed except which socket the frames go down. proxyCountry takes a two-letter code when the request needs to egress from a particular country, and sessionTTL bounds how long the remote session stays open.

Conclusion

zendriver has no connect_over_cdp, and the reason is not that the WebSocket transport is missing — zd.Connection speaks to a remote endpoint on the first try. The obstacle is that tab connections are string-built from a host and a port, and a cloud browser has no per-tab route to point them at.

Flat sessions close that gap. One connection, one Target.attachToTarget call, and a sessionId stamped onto outgoing frames turns the library's own Tab into a handle on a remote page, with every selector helper intact.

When a setup like this misbehaves, two checks usually settle it. If results look empty rather than wrong, wait for a selector before reading content — get_content() does not wait. If tabs appear to open but nothing is ever found in them, confirm you are not opening a second connection, because that is a different browser.

For background on the protocol underneath, see what the Chrome DevTools Protocol is and the comparison of nodriver and Patchright as driver-level stealth tools. Plan details are on the Scrapeless pricing page, and session parameters are in the Scrapeless documentation.

FAQ

Q: Does zendriver have a connect_over_cdp method?

No. There is no equivalent to Playwright's connect_over_cdp or Puppeteer's connect. The connect_existing behaviour inside Browser.start() is close but targets a locally reachable host and port, not a remote WebSocket URL with a token.

Q: Why does setting host and port to a remote endpoint not work?

Because HTTPApi interpolates a host and a port into a fixed plain-HTTP address and fetches /json/version from it. A secure WebSocket URL with a query string cannot be represented as a host and a port, and hosted endpoints generally do not expose those discovery routes publicly.

Q: Can I open several tabs on one remote browser?

Yes, but they must share the connection. Call Target.attachToTarget once per target and stamp the matching sessionId onto that tab's frames. Opening another connection to the endpoint gives you a separate browser instead.

Q: Does the fork's anti-detection patching still apply to a remote browser?

Those patches act on how a browser is launched and configured, so they belong to the process zendriver starts. When you attach to a browser you did not launch, its configuration is whatever the provider set, and zendriver is acting purely as a protocol client.

Q: Why did get_content() return an almost empty document?

Because it does not wait. It issues its command immediately, so a page that has not finished rendering returns the empty document skeleton — 39 characters — with no error. Wait on a selector with wait_for() first.

Q: Do I need Chrome installed locally for the remote path?

No. Nothing is launched on your machine, so no browser binary is required. You only need one to run the local zd.start() example.

Q: What does sessionTTL control?

How long the remote browser session stays open, in seconds. Set it above the time your run needs; the session ends when the connection closes or the window expires, whichever comes first.