# Feeds Stream each feed's real-time events and download its hourly and daily Parquet exports. Each feed is a property of `SynthientClient` with type `Feed`, where `TEvent` is the feed's event type. Every feed has a real-time [stream](#streams) and hourly Parquet [exports](#exports) that roll up into daily exports. Streaming and exports need separate scopes on the API key. The event types are listed under [Event types](https://docs.synthient.com/sdks/csharp/types#event-types). ```csharp public sealed class Feed ``` | Property | Event type | Stream scope | Export scope | | ------------- | ------------------ | ----------------------- | --------------------- | | `Proxies` | `ProxyEvent` | `PROXY_FIREHOSE` | `PROXY_FEEDS` | | `Anonymizers` | `AnonymizerEvent` | `ANONYMIZERS_STREAM` | `ANONYMIZERS_FEED` | | `Torrents` | `TorrentEvent` | `TORRENTS_STREAM` | `TORRENTS_FEED` | | `Ja4t` | `Ja4tEvent` | `JA4T_STREAM` | `JA4T_FEED` | | `HeliosHttp` | `HeliosHttpEvent` | `HONEYPOT_HTTP_STREAM` | `HONEYPOT_HTTP_FEED` | | `HeliosHttps` | `HeliosHttpsEvent` | `HONEYPOT_HTTPS_STREAM` | `HONEYPOT_HTTPS_FEED` | | `HeliosDns` | `HeliosDnsEvent` | `HONEYPOT_DNS_STREAM` | `HONEYPOT_DNS_FEED` | | `HeliosAdb` | `HeliosAdbEvent` | `HONEYPOT_ADB_STREAM` | `HONEYPOT_ADB_FEED` | What each feed carries: - `Proxies` carries residential, mobile, and datacenter proxy sightings. - `Anonymizers` carries VPN and anonymizer IP ranges. - `Torrents` carries torrents seen on the BitTorrent DHT and trackers. - `Ja4t` carries JA4T TCP fingerprints. - The four Helios feeds carry honeypot captures: HTTP requests, TLS ClientHellos, tunnel destinations, and Android Debug Bridge commands. Despite its name, `HeliosDns` does not carry DNS queries. See [Firehose](https://docs.synthient.com/enterprise/firehose), [Feeds](https://docs.synthient.com/enterprise/feeds), and [Helios](https://docs.synthient.com/enterprise/helios) for the data behind each feed. ## Streams ### StreamAsync Streams the feed's events in real time. ```csharp public IAsyncEnumerable StreamAsync(CancellationToken cancellationToken = default) ``` - **Connection:** Each enumeration opens a new connection, which the server keeps open for up to 30 minutes and then closes, ending the enumeration without an error. Opening the connection is retried like other feed requests. Leaving the loop closes the connection. - **Cancellation:** The token can also be passed with `WithCancellation`, and cancelling it throws `OperationCanceledException`. - **Errors:** A non-200 response throws an `ApiException`. A 429 without `RetryAfter` means the key's monthly request budget is spent or it already has too many open streams. An event the SDK cannot decode throws a `SynthientException`. - **Timeout:** If nothing arrives for longer than `SynthientClientOptions.Timeout`, not even the blank keepalive lines the API sends, the enumeration throws an `ApiTimeoutException`. A slow consumer does not trigger the timeout. - **Reconnecting:** Events sent between connections are not replayed. The API has no resume cursor, so enumerate again to reconnect: ```csharp while (!cancellationToken.IsCancellationRequested) { await foreach (var proxy in client.Proxies.StreamAsync(cancellationToken)) { Console.WriteLine($"{proxy.IP} {proxy.Provider} {proxy.Type}"); } } ``` The loop reconnects immediately after a clean close. Opening a connection already uses the SDK's bounded retries, including `Retry-After` handling. Any error that remains after those retries, or occurs while reading the stream, stops this consumer. A `429` without `Retry-After` can indicate an exhausted monthly request budget or too many open streams; resolve that condition before reconnecting. Cancelling the token throws `OperationCanceledException` during iteration. Every feed streams the same way. Only the event type changes: **Streams** ```csharp title="Proxies" await foreach (var proxy in client.Proxies.StreamAsync()) { Console.WriteLine($"{proxy.IP} {proxy.Provider} {proxy.Type} {proxy.CountryCode} {proxy.Asn}"); } ``` ```csharp title="Anonymizers" await foreach (var range in client.Anonymizers.StreamAsync()) { Console.WriteLine($"{range.RangeStart} {range.RangeEnd} {range.Provider} {range.Type}"); } ``` ```csharp title="Torrents" await foreach (var torrent in client.Torrents.StreamAsync()) { Console.WriteLine($"{torrent.InfoHash} {torrent.Name} {torrent.Peers?.Count}"); } ``` ```csharp title="JA4T" await foreach (var sighting in client.Ja4t.StreamAsync()) { Console.WriteLine($"{sighting.IP} {sighting.Ja4t} {sighting.Provider}"); } ``` ```csharp title="Helios HTTP" await foreach (var request in client.HeliosHttp.StreamAsync()) { Console.WriteLine($"{request.Domain} {request.Details?.Method} {request.Details?.Uri} {request.Meta.Provider}"); } ``` ```csharp title="Helios HTTPS" await foreach (var handshake in client.HeliosHttps.StreamAsync()) { if (handshake.Details is null) { continue; } Console.WriteLine($"{handshake.Domain} {handshake.Details.Sni} {handshake.Details.HandshakeVersion}"); } ``` ```csharp title="Helios DNS" await foreach (var destination in client.HeliosDns.StreamAsync()) { Console.WriteLine($"{destination.Domain} {destination.Port} {destination.Meta.Provider}"); } ``` ```csharp title="Helios ADB" await foreach (var command in client.HeliosAdb.StreamAsync()) { Console.WriteLine($"{command.Session} {command.SequentialId} {command.Command}"); } ``` `HeliosHttpsEvent.Details` is `null` when the sensor could not parse the handshake, so check it before use. ## Exports Each feed has hourly Parquet exports that roll up into daily exports. - **Hourly exports:** An hourly export is available until its day's daily export is rolled up, shortly after 00:30 UTC the next day. After that, requests for the hour fail with a 400 `ApiException`. - **Daily exports:** While a daily export is being rolled up, requests for it fail with a 404 `ApiException` that has `RetryAfter` set. ### Export IDs An export ID is one of: - `ExportId.Latest` - a UTC day as `"YYYY-MM-DD"` - a UTC hour as `"YYYY-MM-DD/H"`, where a zero-padded hour is also accepted `Export.Id` values can be passed as they are. On the first page of a listing, the newest hourly export has the ID `latest`, a moving alias for the freshest export. To keep a stable ID, build it from the export's `Date` and `Hour` with `ExportId.Hourly(export.Date, export.Hour!.Value)`, such as `2026-10-14/21`. An ID is checked when it is used, and an invalid one throws an `ArgumentException` before any request is sent. - `ExportId.Latest` `const string` — `"latest"`, the feed's freshest export: the newest hourly export during the day, or the daily export once that day's hours are rolled up. `ExportId.Daily` returns the ID of the daily export for a UTC day. `ExportId.Hourly` returns the ID of the hourly export for an hour of a UTC day. ```csharp public static string Daily(DateOnly day) public static string Hourly(DateOnly day, int hour) ``` ```csharp var now = DateTime.UtcNow; var yesterday = ExportId.Daily(DateOnly.FromDateTime(now.AddDays(-1))); var lastHour = now.AddHours(-1); var hourly = ExportId.Hourly(DateOnly.FromDateTime(lastHour), lastHour.Hour); Console.WriteLine($"{yesterday} {hourly}"); ``` - **Input:** Both take a `DateOnly`, so convert a time to UTC before taking its date. `ExportId.Hourly` throws an `ArgumentOutOfRangeException` if `hour` is outside 0 to 23. ### ListExportsAsync Returns one page of the feed's exports, newest first. ```csharp public Task ListExportsAsync(int? limit = null, string? cursor = null, CancellationToken cancellationToken = default) ``` ```csharp var page = await client.Proxies.ListExportsAsync(limit: 24); foreach (var export in page.Exports) { Console.WriteLine($"{export.Id} {export.Kind} {export.RowCount}"); } Console.WriteLine($"next cursor: {page.NextCursor}"); ``` - **Cost:** It costs no credits. - **Input:** `limit` is the page size, from 1 to 500, and `null` uses the API default of 100. `cursor` continues a listing from a previous page's `NextCursor`. A `limit` below 1 throws an `ArgumentOutOfRangeException` without a request. - **Paging:** Use `ListAllExportsAsync` to read every page. ### ListAllExportsAsync Returns every export of the feed, newest first, fetching pages lazily as enumeration proceeds. ```csharp public IAsyncEnumerable ListAllExportsAsync(int? limit = null, string? cursor = null, CancellationToken cancellationToken = default) ``` ```csharp await foreach (var export in client.Proxies.ListAllExportsAsync()) { Console.WriteLine($"{export.Id} {export.Kind} {export.SizeBytes}"); } ``` - **Paging:** Leaving the loop early fetches no more pages. - **Errors:** A failure on any page ends the enumeration with its exception, and a page that repeats a cursor the API already returned throws a `SynthientException`. - **Input:** `limit` and `cursor` work as in `ListExportsAsync`. An invalid `limit` throws when the method is called rather than when enumeration starts. ### GetExportMetaAsync Returns the checksum, size, row count, and Parquet schema of an export. ```csharp public Task GetExportMetaAsync(string id, CancellationToken cancellationToken = default) ``` ```csharp var meta = await client.Proxies.GetExportMetaAsync(ExportId.Latest); Console.WriteLine($"{meta.Id} {meta.Rows} {meta.Size} {meta.Checksum}"); foreach (var column in meta.Schema.Fields) { Console.WriteLine($"{column.Name} {column.Type}"); } ``` - **Cost:** It costs no credits. ### GetDownloadUrlAsync Returns a presigned URL for an export's Parquet file. ```csharp public Task GetDownloadUrlAsync(string id, CancellationToken cancellationToken = default) ``` ```csharp var url = await client.Proxies.GetDownloadUrlAsync(ExportId.Latest); Console.WriteLine(url.Host); ``` - **Lifetime:** The URL is valid for 24 hours. - **Security:** The URL grants access without an API key, so treat it as a secret. - **Use:** It is useful for tools that read Parquet over HTTPS directly. ### DownloadAsync Opens an export's Parquet file for reading. ```csharp public Task DownloadAsync(string id, CancellationToken cancellationToken = default) ``` - **Cleanup:** Dispose the returned stream when done. - **Security:** The API key is sent only to the Synthient API, never to the storage host that serves the file, and exceptions never include the presigned URL. - **Errors:** An error status from either throws an `ApiException`, and one from the storage host carries only the status code. Downloads from the storage host are not retried. - **Cancellation:** `cancellationToken` covers opening the file, so pass a token to each read to bound the rest. - **Timeout:** A read that stalls for longer than `SynthientClientOptions.Timeout` throws an `ApiTimeoutException`. The SDK does not verify the file. Compare its SHA-256 with `Export.Checksum` or `ExportMeta.Checksum`. Read the checksum and download the file by the same stable ID, because `ExportId.Latest` can move to a newer export between the two calls: ```csharp using System.Security.Cryptography; using Synthient; using var client = new SynthientClient(); var yesterday = ExportId.Daily(DateOnly.FromDateTime(DateTime.UtcNow.AddDays(-1))); await SaveExportAsync(client.Proxies, yesterday, "proxies.parquet"); static async Task SaveExportAsync(Feed feed, string id, string path) { var meta = await feed.GetExportMetaAsync(id); using var sha256 = SHA256.Create(); await using (var body = await feed.DownloadAsync(id)) await using (var file = new CryptoStream(File.Create(path), sha256, CryptoStreamMode.Write)) { await body.CopyToAsync(file); } if (!string.Equals(Convert.ToHexString(sha256.Hash!), meta.Checksum, StringComparison.OrdinalIgnoreCase)) { throw new InvalidDataException("checksum mismatch"); } } ```