Feeds

Stream each feed's real-time events and download its hourly and daily Parquet exports.

Each feed is a readonly property of Client with type Feed<T>, where T is the feed's event class. Every feed has a real-time stream and hourly Parquet exports that roll up into daily exports. Streaming and exports need separate scopes on the API key. The event classes are listed under Event types.

What each feed carries:

• proxies carries residential, mobile, and datacenter proxies.
• anonymizers carries VPN and anonymizer IP ranges.
• torrents carries torrents and the peers sharing them.
• ja4t carries JA4T TCP fingerprints.
• The four Helios feeds carry honeypot captures: HTTP requests, TLS handshakes, tunnel destinations, and Android Debug Bridge commands. Despite its name, heliosDns carries tunnel destinations, not DNS queries.

Feed is a final readonly class whose constructor is internal, so use the instances on Client instead of creating your own.

See Firehose, Feeds, and Helios for the data behind each feed.

Streams

stream

Connects to the feed's real-time stream and returns an EventStream<T>.

public function stream(): EventStream

• Errors: The connection is made by this call, so errors such as a missing scope (403) or too many open streams (429) are thrown here, before iteration. Any status other than 200 throws an ApiException.
• Connection: Connecting is retried like other account and feed requests: on 500, 502, 503, and 504 responses, on transport failures other than TLS errors, and on 429 responses that carry Retry-After.

EventStream

An open connection to a real-time feed.

• Use: Iterate it once with foreach to receive events as they arrive.
• Connection: The server keeps a stream open for up to 30 minutes and then closes it, which ends iteration without an error, so reconnect in a loop.
• Reconnecting: Events sent between connections are missed.
• Cleanup: Call close() when you stop early, ideally in a finally block, so the connection is not held until garbage collection.

getIterator() yields events of type T as they arrive.

public function getIterator(): Generator

• Errors: A read that fails throws ConnectionException, a read that gets no data within timeout throws TimeoutException, and a malformed event throws SynthientException. Each ends the stream. A connection that closes between events ends iteration without an error, as the server's routine close does. Iterating a second time throws \LogicException.
• Cleanup: The connection is closed when iteration finishes or fails.

close() closes the connection.

public function close(): void

• Results: An iteration in progress then ends without an error, after the events already received.

Reconnect in a loop to keep consuming a feed:

while (true) {
    $events = $client->proxies->stream();
    try {
        foreach ($events as $event) {
            echo $event->ip, ' ', $event->provider, ' ', $event->type, "\n";
        }
    } finally {
        $events->close();
    }
}

The loop reconnects immediately after a clean close. Opening a connection already uses the SDK's bounded retries, including Retry-After handling. Any error that remains after those retries, or occurs while reading the stream, stops this consumer. A 429 without Retry-After can indicate an exhausted monthly request budget or too many open streams; resolve that condition before reconnecting.

Every feed streams the same way. Only the event class changes:

$events = $client->proxies->stream();
try {
    foreach ($events as $event) {
        echo $event->ip, ' ', $event->provider, ' ', $event->type, ' ', $event->countryCode, ' ', $event->asn, "\n";
    }
} finally {
    $events->close();
}

HeliosHttpsEvent::$details is null when the sensor could not parse the handshake, so check it before use.

Exports

Each feed has hourly Parquet exports that roll up into daily exports.

• Hourly exports: An hourly export can be fetched until its day's rollup, shortly after 00:30 UTC the next day. After that, requests for the hour throw a 400 ApiException.
• Daily exports: While a daily export is still rolling up, requests for it throw a 404 ApiException with retryAfter set.

Export IDs

Every export method takes a string|DateTimeInterface naming the export. A string is one of:

• "latest" for the feed's freshest export: the newest hourly export during the day, or the daily export once that day's hours are rolled up
• "YYYY-MM-DD" for a daily export
• "YYYY-MM-DD/H" for an hourly one, where a zero-padded hour is also accepted

Export::$id values can be passed as they are. A DateTimeInterface names the daily export of its UTC date. Any other string, such as an impossible date or an hour above 23, throws \InvalidArgumentException before a request is sent.

There are no helper functions or constants for export IDs. Build an hourly ID with format('Y-m-d/G') on a time in UTC:

$utc = new DateTimeZone('UTC');
$yesterday = new DateTimeImmutable('yesterday', $utc);
$lastHour = (new DateTimeImmutable('-1 hour', $utc))->format('Y-m-d/G');

$daily = $client->proxies->exportMeta($yesterday);
$hourly = $client->proxies->exportMeta($lastHour);
echo $daily->id, ' ', $daily->rows, ' ', $hourly->id, ' ', $hourly->rows, "\n";

listExports

Returns one page of the feed's exports, newest first.

public function listExports(?int $limit = null, ?string $cursor = null): ExportPage

$page = $client->proxies->listExports(limit: 24);
foreach ($page->exports as $export) {
    echo $export->id, ' ', $export->kind, ' ', $export->rowCount, "\n";
}
echo 'next cursor: ', $page->nextCursor, "\n";

• Paging: Use allExports() to iterate over every page.

allExports

Returns a Generator<int, Export> over the feed's exports, newest first.

public function allExports(?int $limit = null, ?string $cursor = null): Generator

foreach ($client->proxies->allExports() as $export) {
    echo $export->id, ' ', $export->kind, ' ', $export->sizeBytes, "\n";
}

• Paging: It fetches pages as iteration proceeds: the first request happens on the first iteration, and breaking out fetches nothing more.
• Input: limit and cursor work as in listExports(), and an invalid limit throws when allExports() is called.
• Errors: If the API returns a cursor it already returned, iteration throws SynthientException.

exportMeta

Returns the details of one export: its checksum, size, row count, and Parquet schema.

public function exportMeta(string|DateTimeInterface $export): ExportMeta

$meta = $client->proxies->exportMeta('latest');
echo $meta->id, ' ', $meta->rows, ' ', $meta->size, ' ', $meta->checksum, "\n";
foreach ($meta->schema->fields as $column) {
    echo $column->name, ' ', $column->type, "\n";
}

downloadUrl

Returns the presigned URL of an export's Parquet file.

public function downloadUrl(string|DateTimeInterface $export): string

$url = $client->proxies->downloadUrl('latest');
echo parse_url($url, PHP_URL_HOST), "\n";

• Lifetime: The URL is valid for 24 hours.
• Security: The URL grants access without an API key, so treat it as a secret.
• Redirects: The API answers with a redirect, which the SDK resolves and checks: the URL must be http or https with a host and no credentials, and an https API never redirects to http. Otherwise it throws SynthientException.

download

Opens an export's Parquet file for reading and returns a PSR-7 StreamInterface.

public function download(string|DateTimeInterface $export): StreamInterface

• Cleanup: The caller must close it.
• Security: The storage host that serves the file never receives the API key or the HTTP client's default headers, cookies, and auth, and errors never include the presigned URL.
• Redirects: Up to 10 further redirects from storage are followed and checked like downloadUrl().
• Errors: A storage response other than 200 or a redirect throws an ApiException that carries only statusCode. Downloads from storage are not retried, and a failed read throws ConnectionException or TimeoutException.

The SDK does not verify the file. Compare its SHA-256 with Export::$checksum or ExportMeta::$checksum:

$yesterday = new DateTimeImmutable('yesterday', new DateTimeZone('UTC'));
$meta = $client->proxies->exportMeta($yesterday);
$body = $client->proxies->download($yesterday);

$file = fopen('proxies.parquet', 'wb') ?: throw new RuntimeException('cannot open proxies.parquet');
$hash = hash_init('sha256');
try {
    while (!$body->eof()) {
        $chunk = $body->read(1 << 20);
        if (fwrite($file, $chunk) !== strlen($chunk)) {
            throw new RuntimeException('cannot write proxies.parquet');
        }
        hash_update($hash, $chunk);
    }
} finally {
    $body->close();
    fclose($file);
}

if (!hash_equals($meta->checksum, hash_final($hash))) {
    throw new RuntimeException('checksum mismatch');
}