Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions .changeset/precompiles-get-logs-in-range.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
---
'@sei-js/precompiles': minor
---

Add `getLogsInRange`, `streamLogsInRange`, `blockRanges` and `MAX_GET_LOGS_BLOCK_RANGE` for reading logs across a block range.

`eth_getLogs` is capped per request, so reading more history than one request allows means walking it in chunks, and every project that needs logs writes that loop again. This walk only sends requests a Sei node can answer. Spans are counted inclusively the way the node counts them (2000 blocks passes, 2001 is refused). A span too heavy to answer is halved and asked again, whether the node refuses it for matching more than `max_log_no_block` logs (sei-chain v6.7 and later), the response passes viem's size limit (before v6.7, when bounded requests are served whole), or the span times out. A node whose refusal names a smaller `max_blocks_for_log` is walked at that, and busy or rate limited refusals are retried with backoff. Every request carries an explicit `toBlock`, because nodes before v6.7 silently cut an open-ended request off at the log cap.

`streamLogsInRange` yields each chunk with its logs, so a backfill can store as it goes and resume from the last `toBlock`. `getLogsInRange` collects the walk into one array and awaits an optional `onChunk` for each chunk. Both take viem's `getLogs` filter (`address`, `event` with `args`, `events`, `strict`), accept a whole contract ABI as `events`, and take any viem `Client`, including one that carries an account. Without a `toBlock` they read to the head, since Sei finalises a block as it is produced. `blockRanges` gives the fixed-width plan without making requests.

No dependency or peer range changes: this uses the `viem` peer already declared.
87 changes: 87 additions & 0 deletions packages/precompiles/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,3 +69,90 @@ Import the canonical Sei mainnet and testnet definitions from the package root o
import { sei, seiTestnet } from '@sei-js/precompiles';
// or: import { sei, seiTestnet } from '@sei-js/precompiles/viem';
```

## Reading logs across a block range

`eth_getLogs` is capped per request, so reading more history than one request
allows means walking it in chunks. `getLogsInRange` walks a range and returns
every log in it, and `streamLogsInRange` yields each chunk as it lands. Both
take a viem client and the same filter viem's `getLogs` takes (`address`,
`event` with `args`, `events`, `strict`):

```ts
import { createPublicClient, http, parseAbiItem } from 'viem';
import { getLogsInRange, sei } from '@sei-js/precompiles/viem';

const client = createPublicClient({ chain: sei, transport: http() });
const head = await client.getBlockNumber();

const logs = await getLogsInRange(client, {
address: '0x…',
event: parseAbiItem('event Transfer(address indexed from, address indexed to, uint256 value)'),
args: { to: '0x…' },
fromBlock: head - 20_000n,
toBlock: head
});
```

For a long backfill, use the stream and store each chunk before asking for the
next. Memory then holds one chunk at a time, and a walk that fails part way
through resumes from the block after the last `toBlock` you stored. `events`
takes a whole contract ABI as well, keeping only its events:

```ts
import { streamLogsInRange } from '@sei-js/precompiles/viem';

for await (const chunk of streamLogsInRange(client, { address: '0x…', events: abi, fromBlock: cursor + 1n })) {
await save(chunk.logs, chunk.toBlock);
}
```

What the walk handles so you don't have to:

- **The inclusive boundary.** `MAX_GET_LOGS_BLOCK_RANGE` is `2000n`, the
[`max_blocks_for_log` default](https://github.com/sei-protocol/sei-chain/blob/v6.7.0-rc2/evmrpc/config/config.go#L337-L339) both public endpoints enforce. The node counts a
span as `toBlock - fromBlock + 1`, so 2000 blocks passes and 2001 is refused
with `block range too large (2001), maximum allowed is 2000 blocks`. If a node
allows fewer, the walk drops to the maximum its refusal names.
- **Heavy spans.** From sei-chain v6.7 a node refuses a request matching more
than `max_log_no_block` logs ([10,000 by default](https://github.com/sei-protocol/sei-chain/blob/v6.7.0-rc2/evmrpc/config/config.go#L337-L339)) with `query matches
too many logs`. Before v6.7 it serves a bounded request whole, and on a busy range that
can pass viem's 10 MiB `maxResponseBodySize` instead. Either way, and when the
node or the client times out on a span, the walk halves the span and asks
again. It then holds below the span that failed and only tries it again after
a run of successes, so a steadily dense range isn't refused on every other
request. A single block still too heavy can't be split, so that's thrown and
the fix is a narrower filter.
- **Busy nodes.** Sei's `server too busy`, `server I/O saturated` and `system
overloaded` refusals and its large query rate limit are retried with backoff
(`retryCount`, `retryDelay`). viem's transport doesn't retry these itself,
because they arrive as JSON-RPC `-32000`. The rate limit [only applies to
spans over 100 blocks](https://github.com/sei-protocol/sei-chain/blob/v6.7.0-rc2/evmrpc/filter.go#L50-L51), so when it outlasts the retries the walk steps
down to 100 rather than giving up.
- **Open-ended requests.** Every request carries an explicit `toBlock`. Nodes
before sei-chain v6.7 silently cut a request missing either bound off at the
log cap, which reads exactly like a quiet range.

Without a `toBlock`, the walk reads to the head. Sei finalises a block as it is
produced, so there's no reorg window to wait out and no confirmation depth to
subtract. Pass an explicit `toBlock` to lag head deliberately.

Public endpoints prune history, and how much they keep is up to each provider.
A `fromBlock` older than a node keeps is refused with
`requested fromBlock … is before earliest available block …`, so a backfill
over months of history needs an archive node. The walk reads `eth_getLogs`,
which leaves out the logs Sei synthesises for CosmWasm and pointer activity;
`sei_getLogs` includes those.

`blockRanges` gives the same fixed-width plan without making any requests, for
estimating a backfill or handing ranges to a bounded worker pool where each
worker calls `getLogsInRange` with an explicit `toBlock`. Keep a pool against a
public endpoint to a few workers, since by default a node allows
[30 requests a second over 100 blocks](https://github.com/sei-protocol/sei-chain/blob/v6.7.0-rc2/evmrpc/filter.go#L50-L51), shared by every client it serves:

```ts
import { blockRanges } from '@sei-js/precompiles/viem';

const chunks = [...blockRanges(1_000_000n, 1_006_000n)];
// [{ fromBlock: 1000000n, toBlock: 1001999n }, … ]
```
1 change: 1 addition & 0 deletions packages/precompiles/src/index.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
export * from './ethers';
export * from './precompiles';
export * from './viem/chain';
export * from './viem/logs';
Loading
Loading