# Feeds

Stream each feed's real-time events and download its hourly and daily Parquet exports.

Each feed is a `readonly` property of `Client` with type `Feed<T>`, where `T` is the feed's event class. Every feed has a real-time [stream](#streams) and hourly Parquet [exports](#exports) that roll up into daily exports. Streaming and exports need separate scopes on the API key. The event classes are listed under [Event types](https://docs.synthient.com/sdks/php/types#event-types).

| Property      | Event type         | Stream scope            | Export scope          |
| ------------- | ------------------ | ----------------------- | --------------------- |
| `proxies`     | `ProxyEvent`       | `PROXY_FIREHOSE`        | `PROXY_FEEDS`         |
| `anonymizers` | `AnonymizerEvent`  | `ANONYMIZERS_STREAM`    | `ANONYMIZERS_FEED`    |
| `torrents`    | `TorrentEvent`     | `TORRENTS_STREAM`       | `TORRENTS_FEED`       |
| `ja4t`        | `Ja4tEvent`        | `JA4T_STREAM`           | `JA4T_FEED`           |
| `heliosHttp`  | `HeliosHttpEvent`  | `HONEYPOT_HTTP_STREAM`  | `HONEYPOT_HTTP_FEED`  |
| `heliosHttps` | `HeliosHttpsEvent` | `HONEYPOT_HTTPS_STREAM` | `HONEYPOT_HTTPS_FEED` |
| `heliosDns`   | `HeliosDnsEvent`   | `HONEYPOT_DNS_STREAM`   | `HONEYPOT_DNS_FEED`   |
| `heliosAdb`   | `HeliosAdbEvent`   | `HONEYPOT_ADB_STREAM`   | `HONEYPOT_ADB_FEED`   |

What each feed carries:

- `proxies` carries residential, mobile, and datacenter proxies.
- `anonymizers` carries VPN and anonymizer IP ranges.
- `torrents` carries torrents and the peers sharing them.
- `ja4t` carries JA4T TCP fingerprints.
- The four Helios feeds carry honeypot captures: HTTP requests, TLS handshakes, tunnel destinations, and Android Debug Bridge commands. Despite its name, `heliosDns` carries tunnel destinations, not DNS queries.

`Feed` is a `final readonly` class whose constructor is internal, so use the instances on `Client` instead of creating your own.

See [Firehose](https://docs.synthient.com/enterprise/firehose), [Feeds](https://docs.synthient.com/enterprise/feeds), and [Helios](https://docs.synthient.com/enterprise/helios) for the data behind each feed.

## Streams

### stream

Connects to the feed's real-time stream and returns an `EventStream<T>`.

```php
public function stream(): EventStream
```

- **Errors:** The connection is made by this call, so errors such as a missing scope (403) or too many open streams (429) are thrown here, before iteration. Any status other than 200 throws an `ApiException`.
- **Connection:** Connecting is retried like other account and feed requests: on 500, 502, 503, and 504 responses, on transport failures other than TLS errors, and on 429 responses that carry `Retry-After`.

### EventStream

An open connection to a real-time feed.

- **Use:** Iterate it once with `foreach` to receive events as they arrive.
- **Connection:** The server keeps a stream open for up to 30 minutes and then closes it, which ends iteration without an error, so reconnect in a loop.
- **Reconnecting:** Events sent between connections are missed.
- **Cleanup:** Call `close()` when you stop early, ideally in a `finally` block, so the connection is not held until garbage collection.

`getIterator()` yields events of type `T` as they arrive.

```php
public function getIterator(): Generator
```

- **Errors:** A read that fails throws `ConnectionException`, a read that gets no data within `timeout` throws `TimeoutException`, and a malformed event throws `SynthientException`. Each ends the stream. A connection that closes between events ends iteration without an error, as the server's routine close does. Iterating a second time throws `\LogicException`.
- **Cleanup:** The connection is closed when iteration finishes or fails.

`close()` closes the connection.

```php
public function close(): void
```

- **Results:** An iteration in progress then ends without an error, after the events already received.

* `EventStream::MAX_EVENT_SIZE` `int` — `16777216` (16 MiB), the largest event the stream accepts. A larger event throws `SynthientException`.

Reconnect in a loop to keep consuming a feed:

```php
while (true) {
    $events = $client->proxies->stream();
    try {
        foreach ($events as $event) {
            echo $event->ip, ' ', $event->provider, ' ', $event->type, "\n";
        }
    } finally {
        $events->close();
    }
}
```

The loop reconnects immediately after a clean close. Opening a connection already uses the SDK's bounded retries, including `Retry-After` handling. Any error that remains after those retries, or occurs while reading the stream, stops this consumer. A `429` without `Retry-After` can indicate an exhausted monthly request budget or too many open streams; resolve that condition before reconnecting.

Every feed streams the same way. Only the event class changes:

**Streams**

```php title="Proxies"
$events = $client->proxies->stream();
try {
    foreach ($events as $event) {
        echo $event->ip, ' ', $event->provider, ' ', $event->type, ' ', $event->countryCode, ' ', $event->asn, "\n";
    }
} finally {
    $events->close();
}
```

```php title="Anonymizers"
$events = $client->anonymizers->stream();
try {
    foreach ($events as $event) {
        echo $event->rangeStart, ' ', $event->rangeEnd, ' ', $event->provider, ' ', $event->type, "\n";
    }
} finally {
    $events->close();
}
```

```php title="Torrents"
$events = $client->torrents->stream();
try {
    foreach ($events as $event) {
        echo $event->infoHash, ' ', $event->name, ' ', count($event->peers ?? []), "\n";
    }
} finally {
    $events->close();
}
```

```php title="JA4T"
$events = $client->ja4t->stream();
try {
    foreach ($events as $event) {
        echo $event->ip, ' ', $event->ja4t, ' ', $event->provider, "\n";
    }
} finally {
    $events->close();
}
```

```php title="Helios HTTP"
$events = $client->heliosHttp->stream();
try {
    foreach ($events as $event) {
        echo $event->domain, ' ', $event->details?->method, ' ', $event->details?->uri, ' ', $event->meta->provider, "\n";
    }
} finally {
    $events->close();
}
```

```php title="Helios HTTPS"
$events = $client->heliosHttps->stream();
try {
    foreach ($events as $event) {
        if ($event->details === null) {
            continue;
        }
        echo $event->domain, ' ', $event->details->sni, ' ', $event->details->handshakeVersion, "\n";
    }
} finally {
    $events->close();
}
```

```php title="Helios DNS"
$events = $client->heliosDns->stream();
try {
    foreach ($events as $event) {
        echo $event->domain, ' ', $event->port, ' ', $event->meta->provider, "\n";
    }
} finally {
    $events->close();
}
```

```php title="Helios ADB"
$events = $client->heliosAdb->stream();
try {
    foreach ($events as $event) {
        echo $event->session, ' ', $event->sequentialId, ' ', $event->command, "\n";
    }
} finally {
    $events->close();
}
```

`HeliosHttpsEvent::$details` is `null` when the sensor could not parse the handshake, so check it before use.

## Exports

Each feed has hourly Parquet exports that roll up into daily exports.

- **Hourly exports:** An hourly export can be fetched until its day's rollup, shortly after 00:30 UTC the next day. After that, requests for the hour throw a 400 `ApiException`.
- **Daily exports:** While a daily export is still rolling up, requests for it throw a 404 `ApiException` with `retryAfter` set.

### Export IDs

Every export method takes a `string|DateTimeInterface` naming the export. A string is one of:

- `"latest"` for the feed's freshest export: the newest hourly export during the day, or the daily export once that day's hours are rolled up
- `"YYYY-MM-DD"` for a daily export
- `"YYYY-MM-DD/H"` for an hourly one, where a zero-padded hour is also accepted

`Export::$id` values can be passed as they are. A `DateTimeInterface` names the daily export of its UTC date. Any other string, such as an impossible date or an hour above 23, throws `\InvalidArgumentException` before a request is sent.

There are no helper functions or constants for export IDs. Build an hourly ID with `format('Y-m-d/G')` on a time in UTC:

```php
$utc = new DateTimeZone('UTC');
$yesterday = new DateTimeImmutable('yesterday', $utc);
$lastHour = (new DateTimeImmutable('-1 hour', $utc))->format('Y-m-d/G');

$daily = $client->proxies->exportMeta($yesterday);
$hourly = $client->proxies->exportMeta($lastHour);
echo $daily->id, ' ', $daily->rows, ' ', $hourly->id, ' ', $hourly->rows, "\n";
```

### listExports

Returns one page of the feed's exports, newest first.

```php
public function listExports(?int $limit = null, ?string $cursor = null): ExportPage
```

- `limit` `?int` — The page size. `null` uses the API default of 100, and the API clamps values above 500. A value under 1 throws `\InvalidArgumentException`.
- `cursor` `?string` — Continues a listing from a previous page's `nextCursor`. `null` or an empty string starts at the newest export.

```php
$page = $client->proxies->listExports(limit: 24);
foreach ($page->exports as $export) {
    echo $export->id, ' ', $export->kind, ' ', $export->rowCount, "\n";
}
echo 'next cursor: ', $page->nextCursor, "\n";
```

- **Paging:** Use `allExports()` to iterate over every page.

### allExports

Returns a `Generator<int, Export>` over the feed's exports, newest first.

```php
public function allExports(?int $limit = null, ?string $cursor = null): Generator
```

```php
foreach ($client->proxies->allExports() as $export) {
    echo $export->id, ' ', $export->kind, ' ', $export->sizeBytes, "\n";
}
```

- **Paging:** It fetches pages as iteration proceeds: the first request happens on the first iteration, and breaking out fetches nothing more.
- **Input:** `limit` and `cursor` work as in `listExports()`, and an invalid `limit` throws when `allExports()` is called.
- **Errors:** If the API returns a cursor it already returned, iteration throws `SynthientException`.

### exportMeta

Returns the details of one export: its checksum, size, row count, and Parquet schema.

```php
public function exportMeta(string|DateTimeInterface $export): ExportMeta
```

```php
$meta = $client->proxies->exportMeta('latest');
echo $meta->id, ' ', $meta->rows, ' ', $meta->size, ' ', $meta->checksum, "\n";
foreach ($meta->schema->fields as $column) {
    echo $column->name, ' ', $column->type, "\n";
}
```

### downloadUrl

Returns the presigned URL of an export's Parquet file.

```php
public function downloadUrl(string|DateTimeInterface $export): string
```

```php
$url = $client->proxies->downloadUrl('latest');
echo parse_url($url, PHP_URL_HOST), "\n";
```

- **Lifetime:** The URL is valid for 24 hours.
- **Security:** The URL grants access without an API key, so treat it as a secret.
- **Redirects:** The API answers with a redirect, which the SDK resolves and checks: the URL must be `http` or `https` with a host and no credentials, and an `https` API never redirects to `http`. Otherwise it throws `SynthientException`.

### download

Opens an export's Parquet file for reading and returns a PSR-7 `StreamInterface`.

```php
public function download(string|DateTimeInterface $export): StreamInterface
```

- **Cleanup:** The caller must close it.
- **Security:** The storage host that serves the file never receives the API key or the HTTP client's default headers, cookies, and auth, and errors never include the presigned URL.
- **Redirects:** Up to 10 further redirects from storage are followed and checked like `downloadUrl()`.
- **Errors:** A storage response other than 200 or a redirect throws an `ApiException` that carries only `statusCode`. Downloads from storage are not retried, and a failed read throws `ConnectionException` or `TimeoutException`.

The SDK does not verify the file. Compare its SHA-256 with `Export::$checksum` or `ExportMeta::$checksum`:

```php
$yesterday = new DateTimeImmutable('yesterday', new DateTimeZone('UTC'));
$meta = $client->proxies->exportMeta($yesterday);
$body = $client->proxies->download($yesterday);

$file = fopen('proxies.parquet', 'wb') ?: throw new RuntimeException('cannot open proxies.parquet');
$hash = hash_init('sha256');
try {
    while (!$body->eof()) {
        $chunk = $body->read(1 << 20);
        if (fwrite($file, $chunk) !== strlen($chunk)) {
            throw new RuntimeException('cannot write proxies.parquet');
        }
        hash_update($hash, $chunk);
    }
} finally {
    $body->close();
    fclose($file);
}

if (!hash_equals($meta->checksum, hash_final($hash))) {
    throw new RuntimeException('checksum mismatch');
}
```