# Bots and filtering

Which automated traffic ProductLense drops before it is stored, why it never counts toward your plan, and what still gets through.

ProductLense filters automated traffic twice: the script does not run its sends in automated browsers, and the server drops requests whose user agent looks automated. Dropped requests still answer `204`, so a bot learns nothing, and they never count toward your monthly events.

## In the browser

The script sends nothing when:

- `navigator.webdriver` is true (Selenium, Playwright, Puppeteer and most headless browsers set it).
- The hostname is `localhost`, unless the script tag has `data-dev="true"`.
- `window.__pl_disable` is set before the script runs.

## On the server

An event is treated as a bot when the user agent is empty, when the user-agent parser recognises it as a bot, or when its first 512 characters contain any of these markers, ignoring case:

| Kind | Markers |
|---|---|
| Generic | `bot`, `crawl`, `spider`, `slurp`, `preview`, `fetch`, `monitor` |
| Headless and automation | `headless`, `phantomjs`, `puppeteer`, `playwright`, `selenium`, `webdriver` |
| HTTP clients | `curl/`, `wget/`, `python-requests`, `python-urllib`, `python/`, `go-http-client`, `java/`, `axios/`, `node-fetch`, `libwww`, `httpclient`, `http_request`, `scrapy`, `aiohttp` |
| App clients (see below) | `electron/`, `okhttp` |
| Audits and uptime | `lighthouse`, `pagespeed`, `gtmetrix`, `pingdom`, `uptimerobot`, `statuscake`, `site24x7` |
| SEO crawlers | `ahrefs`, `semrush`, `mj12`, `dotbot`, `petalbot`, `bytespider`, `dataforseo`, `screaming frog` |
| Link unfurlers | `facebookexternalhit`, `whatsapp`, `telegram`, `discord`, `skypeuripreview`, `embedly` |
| Search engines | `googlebot`, `bingbot`, `yandex`, `baiduspider`, `duckduckbot`, `applebot`, `ia_archiver` |

### App clients

Electron desktop apps and `okhttp` (the HTTP client under React Native and most Android apps) are used by real people and by scripts. An event from one of them is counted when it carries the screen width (`w`) that the ProductLense script reads in the browser, and dropped as a bot without it. A user agent that also matches another marker, such as `HeadlessChrome … Electron/28`, is always dropped. Counted app visits show as browser `Electron` (desktop) or `Android app` (mobile).

## Other checks on every request

- The site id must exist, and the page URL's hostname must be one of the site's hostnames. When a browser request carries an `Origin` or `Referer` header, its hostname must be on the list as well.
- The body must be valid JSON of at most 8 KB with at most 50 events, and every event must pass the field rules. A request with an invalid event is rejected as a whole with `400`.

## What still gets through

User-agent rules cannot catch a bot that runs a real browser and hides `navigator.webdriver`. Such traffic usually shows as one-page sessions from a single country and device in a short window. ProductLense has no IP blocklist, because it never stores IP addresses.

## Related

- [Exclude pages](/docs/track/exclude-pages)
- [Errors](/docs/api/errors)