# Firehose

Synthient streams every data feed in real time as **newline-delimited JSON (NDJSON)** over a long-lived HTTP connection. Each line of the response is one complete JSON event. Connections stay open for up to **30 minutes** before the server cleanly closes them; clients should reconnect immediately on disconnect. The server also emits a blank-line heartbeat every 15 seconds so intermediaries don't drop idle connections.

The stream identifiers covered on this page are:

`proxies`, `anonymizers`, `torrents`.

> **Note**: Honeypot captures (`honeypot_http`, `honeypot_https`, `honeypot_dns`, `honeypot_adb`) are documented separately under [Helios](https://docs.synthient.com/enterprise/helios), the Synthient honeypot platform.

## Authentication

All endpoints are served from `https://api.synthient.com` under `/api/v4` and require your API key in the `x-api-key` header. Each stream requires its own scope (`PROXY_FIREHOSE`, `ANONYMIZERS_STREAM`, `TORRENTS_STREAM`).

```bash title="cURL"
curl -N https://api.synthient.com/api/v4/feeds/proxies/stream \
  -H "x-api-key: $SYNTHIENT_API_KEY"
```

```bash title="CLI"
synthient stream proxies --duration 5s
```

> **Note**: Use `curl -N` (or your client's equivalent unbuffered/streaming mode) so that lines are delivered to your handler as soon as they arrive. A single API key may only hold a small number of concurrent streams per feed; excess connections are rejected with `429`.

---

## Common response codes

A successful stream returns `200` with an NDJSON body, one JSON object per line, held open for up to 30 minutes. `403` means the key lacks the per-stream feed scope. `429` means too many concurrent streams are already open. See [Errors](https://docs.synthient.com/errors) for the full reference and retry guidance.

---

## Stream proxies (GET /api/v4/feeds/proxies/stream)

Real-time stream of proxy IP observations.

- `ip` `string` — IPv4 or IPv6 address of the proxy.
- `provider` `string` — Provider name, such as `BRIGHTDATA` or `IPIDEA`.
- `type` `string` — Proxy type, such as `RESIDENTIAL_PROXY`, `DATACENTER_PROXY`, or `MOBILE_PROXY`.
- `timestamp` `integer` — Unix timestamp in **seconds** when the proxy was observed.
- `country_code` `string` — ISO 3166-1 alpha-2 country of the proxy IP.
- `asn` `integer` — Autonomous System Number of the proxy IP.

**Request** `GET /api/v4/feeds/proxies/stream`

```bash title="cURL"
curl -N https://api.synthient.com/api/v4/feeds/proxies/stream \
  -H "x-api-key: $SYNTHIENT_API_KEY"
```

```bash title="CLI"
synthient stream proxies
```

**Sample event**

```json title="Event"
{
  "ip": "38.238.45.9",
  "provider": "DATAIMPULSE",
  "type": "RESIDENTIAL_PROXY",
  "timestamp": 1762605697,
  "country_code": "US",
  "asn": 174
}
```

```json title="Datacenter"
{
  "ip": "45.137.21.88",
  "provider": "OXYLABS",
  "type": "DATACENTER_PROXY",
  "timestamp": 1762605712,
  "country_code": "DE",
  "asn": 200651
}
```

```json title="Mobile"
{
  "ip": "49.207.182.14",
  "provider": "IPROYAL",
  "type": "MOBILE_PROXY",
  "timestamp": 1762605733,
  "country_code": "IN",
  "asn": 24560
}
```

> **Note**: Events are delivered as **newline-delimited JSON**: each event arrives as a single line. The samples above are pretty-printed for readability.

---

## Stream anonymizers (GET /api/v4/feeds/anonymizers/stream)

Real-time stream of anonymizer observations: VPNs, Tor exits, and relay-class detections. Anonymizer events emit IP **ranges** rather than individual addresses.

- `range_start` `string` — First IP in the anonymizer range.
- `range_end` `string` — Last IP in the anonymizer range.
- `provider` `string` — Provider name, such as `NORDVPN` or `MULLVAD`.
- `type` `string` — Anonymizer type, such as `COMMERCIAL_VPN`, `TOR_NODE`, or `PRIVATE_RELAY`.
- `timestamp` `integer` — Unix timestamp in **seconds** when the range was observed.

**Request** `GET /api/v4/feeds/anonymizers/stream`

```bash title="cURL"
curl -N https://api.synthient.com/api/v4/feeds/anonymizers/stream \
  -H "x-api-key: $SYNTHIENT_API_KEY"
```

```bash title="CLI"
synthient stream anonymizers
```

**Sample event**

```json title="VPN"
{
  "range_start": "2.56.252.0",
  "range_end": "2.56.252.255",
  "provider": "NORDVPN",
  "type": "COMMERCIAL_VPN",
  "timestamp": 1762605697
}
```

```json title="Tor"
{
  "range_start": "185.220.101.32",
  "range_end": "185.220.101.32",
  "provider": "TOR",
  "type": "TOR_NODE",
  "timestamp": 1762605714
}
```

```json title="Relay"
{
  "range_start": "172.224.224.0",
  "range_end": "172.224.231.255",
  "provider": "APPLE",
  "type": "PRIVATE_RELAY",
  "timestamp": 1762605731
}
```

---

## Stream torrents (GET /api/v4/feeds/torrents/stream)

Real-time stream of torrent peer sightings from the DHT network and trackers, with info hash, metadata, and observed peers.

- `info_hash` `string` — 40-character hex SHA-1 info hash of the torrent.
- `name` `string` — Torrent name from metadata.
- `magnet_uri` `string` — Magnet URI for the torrent.
- `total_size` `integer` — Total size of files in bytes.
- `piece_length` `integer` — Piece length in bytes.
- `file_count` `integer` — Number of files in the torrent.
- `files` `array<object>` — Per-file `path` and `length` in bytes.
- `peers` `array<object>` — Observed peers: `ip`, `port`, `source` of `DHT`, `PEX`, or `tracker`, and `encrypted`.
- `timestamp` `integer` — Unix timestamp in **seconds** when the torrent was observed.

**Request** `GET /api/v4/feeds/torrents/stream`

```bash title="cURL"
curl -N https://api.synthient.com/api/v4/feeds/torrents/stream \
  -H "x-api-key: $SYNTHIENT_API_KEY"
```

```bash title="CLI"
synthient stream torrents
```

**Sample event**

```json title="Single-file"
{
  "info_hash": "abcdef0123456789abcdef0123456789abcdef01",
  "name": "ubuntu-26.04-desktop-amd64.iso",
  "magnet_uri": "magnet:?xt=urn:btih:abcdef0123456789abcdef0123456789abcdef01&dn=ubuntu-26.04-desktop-amd64.iso",
  "total_size": 4831838208,
  "piece_length": 2097152,
  "file_count": 1,
  "files": [
    { "path": "ubuntu-26.04-desktop-amd64.iso", "length": 4831838208 }
  ],
  "peers": [
    { "ip": "203.0.113.42", "port": 51413, "source": "DHT", "encrypted": true },
    { "ip": "198.51.100.17", "port": 6881, "source": "PEX", "encrypted": false }
  ],
  "timestamp": 1762605697
}
```

```json title="Multi-file"
{
  "info_hash": "1f2e3d4c5b6a7988170615243342510fafedcb91",
  "name": "Big.Buck.Bunny.2008.1080p.BluRay",
  "magnet_uri": "magnet:?xt=urn:btih:1f2e3d4c5b6a7988170615243342510fafedcb91",
  "total_size": 8642097152,
  "piece_length": 4194304,
  "file_count": 3,
  "files": [
    { "path": "Big.Buck.Bunny.2008.1080p.BluRay.mkv", "length": 8638901248 },
    { "path": "subs/eng.srt", "length": 84512 },
    { "path": "README.txt", "length": 3111392 }
  ],
  "peers": [
    { "ip": "45.137.21.88", "port": 6881, "source": "tracker", "encrypted": false },
    { "ip": "2a01:4f8:c17:b8f::1", "port": 51413, "source": "DHT", "encrypted": true },
    { "ip": "82.165.224.31", "port": 6890, "source": "PEX", "encrypted": true }
  ],
  "timestamp": 1762605731
}
```

---

## Consuming streams

A minimal NDJSON consumer reads the response body line by line and parses each line as JSON. The server cleanly closes connections every \~30 minutes. Treat that as the normal case and reconnect immediately. On a _failed_ reconnect, use **exponential backoff with jitter**: start at 1s, double up to a 60s cap, randomize ±25% so concurrent workers don't synchronize. See [Rate Limits](https://docs.synthient.com/rate-limits) for the full retry recipe.

**NDJSON Consumer**

```python title="Python"
import json, os, random, time, requests

URL = "https://api.synthient.com/api/v4/feeds/proxies/stream"
HEADERS = {"x-api-key": os.environ["SYNTHIENT_API_KEY"]}

attempt = 0
while True:
    try:
        with requests.get(URL, headers=HEADERS, stream=True, timeout=None) as r:
            r.raise_for_status()
            attempt = 0  # successful connect, reset backoff
            for line in r.iter_lines(decode_unicode=True):
                if not line:
                    continue
                event = json.loads(line)
                handle(event)
    except requests.RequestException:
        delay = min(60, 2 ** attempt) * random.uniform(0.75, 1.25)
        time.sleep(delay)
        attempt += 1
```

```go title="Go"
package main

import (
    "bufio"
    "encoding/json"
    "math"
    "math/rand"
    "net/http"
    "os"
    "time"
)

func main() {
    attempt := 0
    for {
        req, _ := http.NewRequest("GET", "https://api.synthient.com/api/v4/feeds/proxies/stream", nil)
        req.Header.Set("x-api-key", os.Getenv("SYNTHIENT_API_KEY"))

        resp, err := http.DefaultClient.Do(req)
        if err == nil && resp.StatusCode == 200 {
            attempt = 0 // successful connect, reset backoff
            scanner := bufio.NewScanner(resp.Body)
            scanner.Buffer(make([]byte, 1<<20), 1<<24)
            for scanner.Scan() {
                var event map[string]any
                if err := json.Unmarshal(scanner.Bytes(), &event); err == nil {
                    handle(event)
                }
            }
            resp.Body.Close()
        } else {
            attempt++
        }
        base := math.Min(60, math.Pow(2, float64(attempt)))
        jitter := 0.75 + rand.Float64()*0.5
        time.Sleep(time.Duration(base*jitter*float64(time.Second)))
    }
}
```