Add gas bench fixture capture tooling by rodiazet · Pull Request #23 · argotorg/solc-bench

rodiazet · 2026-09-15T22:20:28Z

It introduces a way to capture most popular txs from the chain to a contract address.
Based on this data it generates fixtures in the form EEST which can be run by any EL client (including evmone) and measure the gas used.
Using this tooling I generated the fixtures for all the contacts in benchmarks_data which have deployed tag.

It’s a first step to have gas benchmarks which are implemented in following PR. The following PR implements a bytecode swapping in these fixtures and run them on evmone and measure gas usage. The fixtures in the EEST assure that the swapped bytecode does exactly the same what the original does.

This PR adds:

Tolling for capturing the most popular transaction for a contract.
Creating EEST fixtures based on captured transactions.
Add results of capturing transactions for all benchmarks tagged “deployed”

r0qs · 2026-09-16T11:28:30Z

+    one example tx hash each, and a name from Etherscan's `functionName`."""
+    address = address.lower()
+    response = requests.get(
+        "https://api.etherscan.io/v2/api",


Can't we use sourcify instead? :)

does sourcify have transactions? :)

No. That’s why we need etherescan or something similar. Archive node also is not enough. We can potentially scan shitload of block for these transactions but it’s just wasting of resources.-

Ah, right :/

Still, do we need Etherscan at all? We could just take an RPC endpoint instead, which the tool already requires for tracing anyway. No?

But we need to scan a lot of blocks to do it and parsing them and check the transaction. All I did for generating the fixtures many times I did with free etherscan API plan.

we could have used our archive node. but anyways. it's fine. something for a follow-up.

We need also to remember the we would have to somehow undecode selectors to functions names.

r0qs · 2026-09-16T11:37:51Z

Also, can you add to the README how to use it?

clonker

it seems some of the fixtures have transactions that revert (requiredStatus is 0x0). perhaps we should bias towards the happy path. not that reverts aren't important to sample, too, but my feeling is that a reverting transaction is usually not doing much at all.

aave-v3-pool and aave-v4-hub-spoke are not listed in benchmarks.toml. intentional?

also discovery_limit is set to 500 in the targets.toml but the cli defaults to 1000, doesn't it?

beyond all that: i know we haven't been doing it yet but i'd really like to have some testing here.

clonker · 2026-09-16T07:39:23Z

 ["oeth"]
 source = "https://etherscan.io/address/0xd86756dbb01e75a11aadacb75c8495759ed92033"
 version = "0xd86756dbb01e75a11aadacb75c8495759ed92033"
+discovery_address = "0x856c4efb76c1d1ae02e20ceb03a2a6a08b0b8dc3"


the discovery addresses are redundant with the data in the target.tomls, right? should we perhaps just have one of the two?

no exactly. because we wanna save in the results for the address which was used to discover this transaction. For next capturing we want to use for example different one and leave the old one results too.

so why do we record anything in benchmarks.toml?

Just to make it easier to find gas benchmarks for the gas run. Other way we would have to search them in the gas folder. Which means load the targets.toml. It can be also just mapping folder name too. I can change it this way and remove this change in bechmarks.toml

But wait this is other entry. discovery_address is needed because when we have empty fixtures we need to know how to discover the txs for the contracts.

Maybe to make it more readable I can move the data in the targets.toml which are related to how the fixture was generated to some kind of a metadata section. WDYT?

I'd be for removing this field from benchmarks.toml and instead list it in targets.toml like

[[target]] address = "0xe1e61cc36ebdbe30d51d04f38d7930663dcfe940" standard_json = "aave-v4-hub.json" contract_name = "HubInstance" [target.discovery] address = "0x973a023a77420ba610f06b3858ad991df6d85a08" end_block = 25996062 limit = 500

If you remove it from bechmarks.toml it’s impossible to generate gas fixtures based on it and that’s the goal. gas directory content is fully generated using discovery address (+ end_block and limit taken from the console input). Finding these discovery addresses is not super easy and straightforward process. Sometimes it’s difficult to find proper, being used, proxy address which makes delegate calls to address. The process of adding a new fixtures starts from finding the address, dicovery_address a fetching to-be-bechmarked contract sources.

As a next step I wanted to add automatic script which captures all the fixtures for the all contract deployed tagged. Maybe we can add this discovery_address in bechmarks.toml there. I agree it’s a little in the air now.

I have this script and used it to regenerate the fixtures. I can add it to this PR too. Just need an hour to clean it up. I believe it’s not super important but if you want it just let me know.

clonker · 2026-09-16T07:45:04Z

+    """The bare name from Etherscan's `functionName` field (e.g.
+    "supply(address,...)" -> "supply"), or None if it's not decoded."""
+    name = function_name.split("(", 1)[0].strip()
+    return name.lower() if re.fullmatch(r"[A-Za-z_][A-Za-z0-9_]*", name) else None


why the regex here? shouldnt be the split enough?

Yeah. Initially I was thinking that it can be anything in case of undecoded function name, but it appeared that the undecoded function names are just empty string.

clonker · 2026-09-16T07:48:14Z

+    path.write_text(tomlkit.dumps(doc))
+
+
+def _fixture_touches_address(fixture_path: Path, address: str) -> bool:


so this means that a transaction goes via some address? is that right? i'm not sure i understand the method by its name and/or docstring

Yes. So if we capture a transaction by the proxy for example we can end up in the situation where the tx does not touch the address we wanna bechmark code of. In this case the fixture is dropped.

so can we maybe rename the method to _prestate_has_account or so and simplify the docstring?

Yeah. It’s better. The name was taken from the EL implementation.

clonker · 2026-09-16T11:34:27Z

+    one example tx hash each, and a name from Etherscan's `functionName`."""
+    address = address.lower()
+    response = requests.get(
+        "https://api.etherscan.io/v2/api",


does sourcify have transactions? :)

clonker · 2026-09-16T11:38:19Z

+    try:
+        fixture_replay(output_path, evmone_statetest_bin)
+    except ReplayMismatch as e:
+        print(f"WARNING: {e}", file=sys.stderr)


should we perhaps raise here?

Yeah. I was thinking of it and finally left it. But in case of building the fixture we should raise. But on the other hand if not raise we could continue and re-run on failures after a fix.

clonker · 2026-09-16T11:39:01Z

+def _keccak256(data: bytes) -> str:
+    result = subprocess.run(
+        ["cast", "keccak", "0x" + data.hex()],
+        capture_output=True,
+        text=True,
+        check=True,
+    )
+    return result.stdout.strip()


this could be done with pure python right?

Maybe. I’m no a python expert. :) Will check and fix.

Maybe we can use https://github.com/Legrandin/pycryptodome or https://github.com/ethereum/eth-hash

i'd prefer a python implementation over calling cast here via a subprocess

clonker · 2026-09-16T11:40:01Z

is this used at all?

In the following PR

then it should go into the following pr

rodiazet · 2026-09-16T11:56:48Z

Also, can you add to the README how to use it?

I can but this is not the command which is going to be used often. The more important command is added by the following pr.

r0qs · 2026-09-16T13:37:06Z

+            build_fixture_for_tx(examples[selector], rpc_url, evmone_statetest_bin, fixture_path, test_name=name)
+        except Exception as e:
+            print(f"{examples[selector]}: failed to build, skipping ({e})", file=sys.stderr)
+            if force and fixture_path.exists():


This will only delete a possible broken fixture if force is given. Maybe we should always delete the generated fixture if the build process fails.

Yes. It’s leftover from the debugging.

msooseth · 2026-09-16T14:04:58Z

Let's add a nice short description to the README.md about this? Let's focus on the "short" :D

msooseth

My 2 cents :)

blishko · 2026-09-22T15:44:01Z

What's the actual change here?

It introduces a way to capture most popular txs from the chain to a contract address.
Based on this data it generates fixtures in the form EEST which can be run by any EL client (including evmone) and measure the gas used.
Using this tooling I generated the fixtures for all the contacts in benchmarks_data which have deployed tag.

It’s a first step to have gas benchmarks which are implemented in following PR. The following PR implements a bytecode swapping in these fixtures and run them on evmone and measure gas usage. The fixtures in the EEST assure that the swapped bytecode does exactly the same what the original does.

I meant the change in this specific file that I referenced. makerdao-dss-vat.json

So the discovery_address is needed to discover transaction to a contract implementation which is behind a proxy for example. The interesting implementation contract does not have any tx to itself because they all go through the proxy. So we need go capture txs to the proxy and replay them. On the other hand In the following PR we want to swap the implementation contract bytecode not the proxy. that’s why we need to have the second address in the targets.toml to know which bytecode we have to swap.

And regarding the file you are referring. :)
So this contract was properly built but failed on first tx, because the newer solidity makes the overflow check by default. I need to wrap some code fragment into unchecked block to make it running properly.

But this probably means that it should go to the following PR because it’s a problem in it not here yet.

clonker · 2026-09-28T08:51:07Z

        run: |
          nix shell . --inputs-from . nixpkgs#python3Packages.pytest \
-            --command pytest -v tests/test_smoke.py
+            --command pytest -v tests/test_smoke.py tests/test_verify_fixtures.py


why doesn't this run tests/test_capture_contract_smoke.py? i think it would be better to just run the whole directory.

this should suffice: run: nix develop --command pytest -v -rs

Forgot to add this

clonker · 2026-09-28T08:52:31Z

 reuse the clone. Bumping `version` errors out — delete the stale clone and
 re-run.

+### Gas-bench fixtures (real mainnet transactions)


i think this should've gone into the 2nd pr but i really think we should move this forward so let's just keep it here

Add tooling for capturing gas-bench fixtures from mainnet transactions

…d entries Add aave-v4-hub-spoke benchmark and correct deployed addresses

Arguments passed to the capture-contract `--end-block 25996062 --limit 500 --max-selectors 7`

dismissing the CR, I think it all has been addressed and otherwise we can fix stuff up later

rodiazet force-pushed the gas-bench-capture-tooling branch from 77136c4 to 6a3c943 Compare September 15, 2026 22:21

rodiazet added this pull request to stack #25 September 15, 2026 22:23

rodiazet changed the title ~~Gas bench capture tooling.~~ Add gas bench fixture capture tooling Sep 15, 2026

rodiazet force-pushed the gas-bench-capture-tooling branch 3 times, most recently from 5a92a02 to 2eaf9fa Compare September 16, 2026 10:36

rodiazet requested review from clonker and r0qs September 16, 2026 11:09

r0qs reviewed Sep 16, 2026

View reviewed changes

clonker reviewed Sep 16, 2026

View reviewed changes

r0qs reviewed Sep 16, 2026

View reviewed changes

Comment thread src/solc_bench/add_benchmark/fixture_builder.py Outdated

msooseth previously requested changes Sep 16, 2026

View reviewed changes

Comment thread src/solc_bench/add_benchmark/fixture_builder.py Outdated

Comment thread src/solc_bench/add_benchmark/fixture_builder.py Outdated

Comment thread src/solc_bench/add_benchmark/fixture_builder.py Outdated

rodiazet force-pushed the gas-bench-capture-tooling branch 7 times, most recently from 22ac186 to 9404129 Compare September 18, 2026 08:20

rodiazet requested review from clonker, msooseth and r0qs September 18, 2026 08:21

rodiazet force-pushed the gas-bench-capture-tooling branch from 9404129 to 73c8d06 Compare September 21, 2026 12:55

rodiazet marked this pull request as ready for review September 21, 2026 14:49

blishko reviewed Sep 22, 2026

View reviewed changes

rodiazet force-pushed the gas-bench-capture-tooling branch from 73c8d06 to 753e01f Compare September 23, 2026 10:48

rodiazet force-pushed the gas-bench-capture-tooling branch 3 times, most recently from 363df9c to 42842a3 Compare September 24, 2026 10:31

clonker reviewed Sep 28, 2026

View reviewed changes

rodiazet added 4 commits September 28, 2026 17:31

Add tooling for capturing gas-bench fixtures from mainnet transactions

8e02650

Add tooling for capturing gas-bench fixtures from mainnet transactions

Record discovery_address in benchmarks.toml for proxy-fronted deploye…

366d827

…d entries Add aave-v4-hub-spoke benchmark and correct deployed addresses

Capture most popular tx for all test cases with deployed tag

1eb73a8

Arguments passed to the capture-contract `--end-block 25996062 --limit 500 --max-selectors 7`

Add capture-contract smoke test

cd6d730

rodiazet force-pushed the gas-bench-capture-tooling branch from 42842a3 to cd6d730 Compare September 28, 2026 15:59

clonker added 2 commits September 30, 2026 13:39

Re-extract etherfi-liquidity-pool input for the new implementation

bc2d26b

Document capture-contract and tidy up review leftovers

03104d5

clonker approved these changes Sep 30, 2026

View reviewed changes

clonker merged commit 61fbe36 into main Sep 30, 2026
1 check passed

		path.write_text(tomlkit.dumps(doc))


		def _fixture_touches_address(fixture_path: Path, address: str) -> bool:

Conversation

rodiazet commented Sep 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

r0qs commented Sep 16, 2026

Uh oh!

clonker left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rodiazet Sep 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rodiazet Sep 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rodiazet Sep 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

r0qs Sep 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

rodiazet commented Sep 15, 2026 •

edited

Loading

rodiazet Sep 16, 2026 •

edited

Loading

rodiazet Sep 28, 2026 •

edited

Loading

rodiazet Sep 28, 2026 •

edited

Loading

r0qs Sep 16, 2026 •

edited

Loading