# Feeds

Stream each feed's real-time events and download its hourly and daily Parquet exports.

Each feed is returned by a method of `SynthientClient` as a `Feed<T>`, where `T` is the feed's event type. Every feed has a real-time [stream](#streams) and hourly Parquet [exports](#exports) that roll up into daily exports. Streaming and exports need separate scopes on the API key. The event types are listed under [Event types](https://docs.synthient.com/sdks/java/types#event-types).

| Method          | Event type         | Stream scope            | Export scope          |
| --------------- | ------------------ | ----------------------- | --------------------- |
| `proxies()`     | `ProxyEvent`       | `PROXY_FIREHOSE`        | `PROXY_FEEDS`         |
| `anonymizers()` | `AnonymizerEvent`  | `ANONYMIZERS_STREAM`    | `ANONYMIZERS_FEED`    |
| `torrents()`    | `TorrentEvent`     | `TORRENTS_STREAM`       | `TORRENTS_FEED`       |
| `ja4t()`        | `Ja4tEvent`        | `JA4T_STREAM`           | `JA4T_FEED`           |
| `heliosHttp()`  | `HeliosHttpEvent`  | `HONEYPOT_HTTP_STREAM`  | `HONEYPOT_HTTP_FEED`  |
| `heliosHttps()` | `HeliosHttpsEvent` | `HONEYPOT_HTTPS_STREAM` | `HONEYPOT_HTTPS_FEED` |
| `heliosDns()`   | `HeliosDnsEvent`   | `HONEYPOT_DNS_STREAM`   | `HONEYPOT_DNS_FEED`   |
| `heliosAdb()`   | `HeliosAdbEvent`   | `HONEYPOT_ADB_STREAM`   | `HONEYPOT_ADB_FEED`   |

What each feed carries:

- `proxies()` carries proxy sightings.
- `anonymizers()` carries the address ranges of VPN and anonymizing services.
- `torrents()` carries torrents and their peers.
- `ja4t()` carries TCP fingerprints seen from proxies.
- The four Helios feeds carry what honeypots received through proxies: HTTP requests, TLS handshakes, tunnel destinations, and Android Debug Bridge commands. Despite its name, `heliosDns()` carries tunnel destinations, not DNS query names.

See [Firehose](https://docs.synthient.com/enterprise/firehose), [Feeds](https://docs.synthient.com/enterprise/feeds), and [Helios](https://docs.synthient.com/enterprise/helios) for the data behind each feed.

## Streams

### stream

Connects to the feed's real-time stream and returns it as an open `EventStream<T>`.

```java
public EventStream<T> stream()
```

- **Connection:** The connection is retried like any other feed request (see [Retries and timeouts](https://docs.synthient.com/sdks/java/errors#retries-and-timeouts)), but events are never replayed once the stream is open.
- **Errors:** A non-success response throws an `ApiException`. A 429 without `retryAfter()` means the key's monthly request budget is spent or it already has too many open streams.

### EventStream

An `EventStream<T>` is an `Iterable<T>` and a `Closeable`, so read it with a for-each loop inside try-with-resources.

```java
public Iterator<T> iterator()
public void close()
```

- **Iteration:** Reading blocks until the server sends the next event. `iterator()` may be called once, and a second call throws `IllegalStateException`.
- **Cleanup:** `close()` disconnects. It is safe to call more than once and from any thread, and calling it from another thread ends the iteration.
- **Connection:** The server keeps a stream open for up to 30 minutes and then closes it, which ends the iteration normally, without an exception.
- **Errors:** A lost connection throws `ApiConnectionException`, and an event that cannot be decoded throws `SynthientException`. Either one ends the stream.
- **Timeout:** A stream that receives nothing for longer than the client's `timeout` throws `ApiTimeoutException`. The server sends a heartbeat line about every 15 seconds, so a `timeout` under 15 seconds breaks healthy streams.
- **Reconnecting:** The API has no resume cursor, so events sent between connections are missed.

Call `stream()` again to reconnect:

```java
while (true) {
    try (EventStream<ProxyEvent> events = client.proxies().stream()) {
        for (ProxyEvent event : events) {
            System.out.println(event.ip() + " " + event.provider() + " " + event.type());
        }
    }
}
```

The loop reconnects immediately after a clean close. Opening a connection already uses the SDK's bounded retries, including `Retry-After` handling. Any error that remains after those retries, or occurs while reading the stream, stops this consumer. A `429` without `Retry-After` can indicate an exhausted monthly request budget or too many open streams; resolve that condition before reconnecting.

Every feed streams the same way. Only the event type changes:

**Streams**

```java title="Proxies"
try (EventStream<ProxyEvent> events = client.proxies().stream()) {
    for (ProxyEvent event : events) {
        System.out.println(event.ip() + " " + event.provider() + " " + event.type() + " " + event.asn());
    }
}
```

```java title="Anonymizers"
try (EventStream<AnonymizerEvent> events = client.anonymizers().stream()) {
    for (AnonymizerEvent event : events) {
        System.out.println(event.rangeStart() + " " + event.rangeEnd() + " " + event.provider());
    }
}
```

```java title="Torrents"
try (EventStream<TorrentEvent> events = client.torrents().stream()) {
    for (TorrentEvent event : events) {
        List<TorrentPeer> peers = event.peers();
        System.out.println(event.infoHash() + " " + event.name() + " " + (peers == null ? 0 : peers.size()));
    }
}
```

```java title="JA4T"
try (EventStream<Ja4tEvent> events = client.ja4t().stream()) {
    for (Ja4tEvent event : events) {
        System.out.println(event.ip() + " " + event.ja4t() + " " + event.provider());
    }
}
```

```java title="Helios HTTP"
try (EventStream<HeliosHttpEvent> events = client.heliosHttp().stream()) {
    for (HeliosHttpEvent event : events) {
        HttpRequestDetails details = event.details();
        if (details == null) {
            continue;
        }
        System.out.println(event.domain() + " " + details.method() + " " + details.uri());
    }
}
```

```java title="Helios HTTPS"
try (EventStream<HeliosHttpsEvent> events = client.heliosHttps().stream()) {
    for (HeliosHttpsEvent event : events) {
        TlsClientHello details = event.details();
        if (details == null) {
            continue;
        }
        System.out.println(event.domain() + " " + details.sni() + " " + details.handshakeVersion());
    }
}
```

```java title="Helios DNS"
try (EventStream<HeliosDnsEvent> events = client.heliosDns().stream()) {
    for (HeliosDnsEvent event : events) {
        System.out.println(
                event.domain() + " " + event.port() + " " + event.meta().provider());
    }
}
```

```java title="Helios ADB"
try (EventStream<HeliosAdbEvent> events = client.heliosAdb().stream()) {
    for (HeliosAdbEvent event : events) {
        System.out.println(event.session() + " " + event.sequentialId() + " " + event.command());
    }
}
```

## Exports

Each feed has hourly Parquet exports that roll up into daily exports.

- **Hourly exports:** An hourly export is available until its day is rolled up into the daily export, shortly after 00:30 UTC the next day. After that, requests for the hour return a 400 `ApiException`.
- **Daily exports:** While a daily export is being rolled up, requests for it return a 404 `ApiException` with `retryAfter()` set.

### Export IDs

An export ID is one of:

- `ExportId.LATEST`
- a UTC day such as `"2026-05-03"`
- an hour of a UTC day such as `"2026-05-03/7"`, where a zero-padded hour is also accepted

`Export.id()` values can be passed as they are. Any other string, including a day that does not exist, throws `IllegalArgumentException` before a request is sent.

- `ExportId.LATEST` `String` — `"latest"`, the newest export: the latest hourly export during the day, then the daily rollup.

`ExportId.daily` returns the ID of the daily export of a UTC day. `ExportId.hourly` returns the ID of the hourly export of an hour of a UTC day.

```java
public static String daily(LocalDate date)
public static String hourly(LocalDate date, int hour)
```

```java
ZonedDateTime lastHour = ZonedDateTime.now(ZoneOffset.UTC).minusHours(1);
String hourly = ExportId.hourly(lastHour.toLocalDate(), lastHour.getHour());
String daily = ExportId.daily(LocalDate.now(ZoneOffset.UTC).minusDays(1));
System.out.println(hourly + " " + daily);
```

- **Input:** `ExportId.hourly` throws `IllegalArgumentException` if the hour is not between 0 and 23.

### listExports

Returns one page of the feed's exports, newest first.

```java
public ExportPage listExports()
public ExportPage listExports(int limit, @Nullable String cursor)
```

```java
ExportPage page = client.proxies().listExports(24, null);
for (Export export : page.exports()) {
    System.out.println(export.id() + " " + export.kind() + " " + export.rowCount());
}
System.out.println("next cursor: " + page.nextCursor());
```

- **Limit:** `limit` is the page size; the API allows up to 500 and clamps larger values. Without arguments, it returns the first page with the API's default page size of 100.
- **Input:** A limit below 1 throws `IllegalArgumentException`.
- **Paging:** `cursor` continues from a previous page's `nextCursor()`, or is `null` for the first page. Use `allExports` to iterate over every page.

### allExports

Returns every export from `cursor` on, newest first, fetching pages as iteration reaches them.

```java
public Iterable<Export> allExports()
public Iterable<Export> allExports(int limit, @Nullable String cursor)
```

```java
for (Export export : client.proxies().allExports()) {
    System.out.println(export.id() + " " + export.kind() + " " + export.sizeBytes());
}
```

- **Input:** `limit` and `cursor` work as in `listExports`.
- **Paging:** Each call to `iterator()` starts again from `cursor`, or from the first page when it is `null`, and stopping early fetches nothing more.
- **Errors:** A failed page throws from `hasNext()` and can be retried by calling it again. A cursor the API already returned throws `SynthientException` rather than looping.

### exportMeta

Returns the checksum, size, row count, and Parquet columns of an export.

```java
public ExportMeta exportMeta(String id)
```

```java
ExportMeta meta = client.proxies().exportMeta(ExportId.LATEST);
System.out.println(meta.id() + " " + meta.rows() + " " + meta.size() + " " + meta.checksum());
for (ParquetColumn column : meta.schema().fields()) {
    System.out.println(column.name() + " " + column.type());
}
```

### downloadUrl

Returns the presigned URL of an export's Parquet file.

```java
public URI downloadUrl(String id)
```

```java
URI url = client.proxies().downloadUrl(ExportId.LATEST);
System.out.println(url.getHost());
```

- **Lifetime:** The URL is valid for 24 hours.
- **Security:** The URL grants access to the file without an API key, so treat it as a secret.
- **Use:** It is useful for tools that read Parquet over HTTPS directly.
- **Errors:** If the API does not redirect to a usable location, or redirects from `https` to `http`, it throws `SynthientException`.

### download

Returns the contents of an export's Parquet file.

```java
public InputStream download(String id)
```

- **Cleanup:** The caller must close the returned stream.
- **Security:** The API key is sent only to the Synthient API, never to the storage host that serves the file, and errors never include the presigned URL.
- **Errors:** An error status from the storage host throws an `ApiException` without the response body. The download from the storage host is not retried and is not reported to the response hook.
- **Timeout:** The client's `timeout` applies while reading. If the transfer stalls or drops, reading throws an `IOException`, such as `java.net.SocketTimeoutException`, rather than an SDK exception.

The SDK does not verify the file. Compare its SHA-256 with `Export.checksum()` or `ExportMeta.checksum()`. Use the same dated ID for the metadata and the download, because `ExportId.LATEST` can move to a newer export in between:

```java
static void saveExport(Feed<?> feed, String id, Path path) throws IOException, NoSuchAlgorithmException {
    ExportMeta meta = feed.exportMeta(id);
    MessageDigest sha256 = MessageDigest.getInstance("SHA-256");
    try (InputStream body = new DigestInputStream(feed.download(id), sha256);
            OutputStream out = Files.newOutputStream(path)) {
        body.transferTo(out);
    }
    if (!HexFormat.of().formatHex(sha256.digest()).equals(meta.checksum())) {
        throw new IOException("checksum mismatch");
    }
}
```

```java
LocalDate yesterday = LocalDate.now(ZoneOffset.UTC).minusDays(1);
saveExport(client.proxies(), ExportId.daily(yesterday), Path.of("proxies.parquet"));
```