Skip to content

smite-scenarios: sync target chain view via RPC after mining - #211

Open
NishantBansal2003 wants to merge 2 commits into
lnfuzz:masterfrom
NishantBansal2003:target-cli
Open

smite-scenarios: sync target chain view via RPC after mining#211
NishantBansal2003 wants to merge 2 commits into
lnfuzz:masterfrom
NishantBansal2003:target-cli

Conversation

@NishantBansal2003

Copy link
Copy Markdown
Contributor

ref: #195

Targets (CLN, LDK) previously learned about new blocks through bitcoind's blocknotify, which fires asynchronously. When multiple blocks are mined, this can queue up many sync calls, adding unnecessary load. Instead, explicitly sync the target once after each MineBlocks operation.
Introduce a TargetRpc trait with a chain_sync() method, called by the executor immediately after MineBlocks. CLN and LDK sync explicitly via RPC and signals, respectively, while LND and Eclair use ZMQ and remain no-ops.
Dropping -blocknotify also removes the burst of asynchronous SIGUSR1s during initial block generation, so LDK no longer needs the pre_exec SIG_IGN

The reason I went with an RPC trait is that, in the future, we'll need access to the target's CLI/RPC to add payment hash and preimage information to the target so that it can send an update_fulfill_htlc message.
Also, for now, I only added the CLI as a trait implementation for each target to keep the changes contained and concise. In the future, we can definitely add direct RPC calls over the socket as well to save overhead.

@erickcestari erickcestari left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea! It's much better that it only calls the sync signal once per MineBlock instead of calling it for every block mined.

Comment thread smite-scenarios/src/targets/ldk.rs Outdated
Comment on lines +70 to +74
assert!(
out.status.success(),
"pkill -USR1 ldk-node-wrapper failed: {}",
String::from_utf8_lossy(&out.stderr)
);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we have an assert here? Imagine the executor crashes the target while executing the input. It would probably fail to sync and panic with a pkill error instead of failing with a Violation::Crashed error.

@NishantBansal2003 NishantBansal2003 Aug 20, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, great point, I think pkill or lightning-cli should always succeed in any case, and they should only error out when the target has already crashed. So, I think we could check whether the target has crashed before asserting this. If it has crashed, we can simply log the failure here, otherwise, we should assert it. I don't think there is any other case where RPC commands should be unresponsive, so I think it's better to be stricter. WDYT?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I agree! I also think logging is a good option.

cmd.env("LD_PRELOAD", handler);
}

// Ignore SIGUSR1 for the window between exec and the wrapper blocking

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some comments at workloads/ldk/src/main.rs also need to be updated after the remove of SIG_IGN.

@NishantBansal2003

Copy link
Copy Markdown
Contributor Author

I updated the CLN lightning-cli based calls to use direct RPC calls instead. This gives us more flexibility in detecting when someone is listening on the Unix RPC socket, allowing us to handle that case explicitly (as discussed in #211 (comment)) while hard-asserting on any other errors. The approach is largely inspired by https://github.com/rust-bitcoin/corepc/blob/8d82c88aebf8268bfc9732aaa15e64a5bba1f9a0/jsonrpc/src/simple_uds.rs and the cln_rpc rust crate.

I verified this by crashing the target before syncing the chain. CLN and LDK are now able to distinguish between the target having already crashed and other errors, logging the former and hard-asserting on the latter.

I haven't done the same for LND and Eclair since they currently don't use any CLI calls. I'll address those in follow-up PRs, as we need to make RPC calls for them as well

Comment thread smite-scenarios/src/targets/cln.rs Outdated

@erickcestari erickcestari left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! I think we can optimize this a bit by avoiding spawning the pkill process, consequently reducing the memory overhead and the amount of memory dirtied by process creation.

Comment on lines +66 to +84
fn chain_sync(&mut self) {
let out = Command::new("pkill")
.arg("-USR1")
.arg("-f")
.arg("^ldk-node-wrapper")
.output()
.expect("pkill -USR1 ldk-node-wrapper should not fail");

match out.status.code() {
Some(0) => {}
// pkill exits 1 when nothing matches, so LDK is gone, check_alive
// will report the crash at the end.
Some(1) => log::warn!("ldk-node-wrapper is not running, skipping chain sync"),
_ => panic!(
"pkill -USR1 ldk-node-wrapper failed: {}",
String::from_utf8_lossy(&out.stderr)
),
}
}

@erickcestari erickcestari Aug 31, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could use the ldk ManagedProcess to get the PID and use the syscall kill directly instead of creating a pkill process to do this for us.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done and Rebased!

@NishantBansal2003
NishantBansal2003 force-pushed the target-cli branch 2 times, most recently from 5931f69 to 9975e59 Compare September 2, 2026 08:59
Targets previously learned about new blocks through bitcoind's
blocknotify, which fires asynchronously. When multiple blocks
are mined, this can queue up many sync calls, adding unnecessary
load. Instead, explicitly sync the target once after each
MineBlocks operation.

Introduce a TargetRpc trait with a chain_sync() method, called by
the executor immediately after MineBlocks. CLN and LDK sync
explicitly via RPC and signals, respectively, while LND and Eclair
use ZMQ and remain no-ops.

Dropping -blocknotify also removes the burst of asynchronous SIGUSR1s
during initial block generation, so LDK no longer needs the pre_exec SIG_IGN.

Signed-off-by: Nishant Bansal <nishant.bansal.282003@gmail.com>
Comment thread smite-scenarios/src/targets/cln.rs Outdated
Comment on lines +102 to +108
assert!(
response.get("error").is_none(),
"lightningd rejected {method}: {}",
response["error"]
);

Ok(response["result"].take())

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we define an RpcResponse struct like we do for all other RPCs? It would make it clearer what we expect a response to contain.

Comment thread smite-scenarios/src/targets/cln.rs Outdated
#[derive(Debug, Clone)]
pub struct ClnRpc {
/// Path to the CLN node's unix RPC socket.
pub rpc_socket: PathBuf,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
pub rpc_socket: PathBuf,
rpc_socket: PathBuf,

Comment thread smite-scenarios/src/targets/cln.rs Outdated
}

impl ClnRpc {
// Bound RPC socket I/O so a stalled lightningd cannot block indefinitely.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// Bound RPC socket I/O so a stalled lightningd cannot block indefinitely.
/// Bound RPC socket I/O so a stalled lightningd cannot block indefinitely.

Comment thread smite-scenarios/src/targets/cln.rs Outdated
Comment on lines +97 to +101
let mut response: serde_json::Value = serde_json::Deserializer::from_reader(&mut sock)
.into_iter()
.next()
.unwrap_or_else(|| panic!("lightningd closed the socket without answering {method}"))
.unwrap_or_else(|e| panic!("failed to read {method} response from lightningd: {e}"));

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This code is confusing -- what is being unwrapped twice? Also, the second panic triggers on timeouts too, which seems like it should also be "warn-only" since a crash/hang will be detected later.

Perhaps we should match on the response for clarity and to let us distinguish the timeout error types from other types.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I updated the error handling to cover possible cases where the target is hung or crashed, whether before the call, during the write, or during the read, and also made it a bit clearer. I also verified some crash cases by crashing CLN before the run, after the write, and after the read, and the handling covers all of them

Wasn't fully sure how to test the hang cases, but I think TimedOut or UnexpectedEof should cover those cases as well

Comment thread workloads/ldk/src/main.rs Outdated
Comment on lines +35 to +40
/// defaults to terminating the process. That distinction is the whole point: an
/// *ignored* signal is discarded the moment it is delivered, while a *blocked*
/// one stays pending until `sigwait()` consumes it, regardless of its
/// disposition. So this call is what makes target's SIGUSR1 observable, and
/// why nothing here calls `sigaction`: the wait loop below is the only consumer
/// these signals need.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need to explain the difference between signal-blocking and SIG_IGN anymore.

Suggested change
/// defaults to terminating the process. That distinction is the whole point: an
/// *ignored* signal is discarded the moment it is delivered, while a *blocked*
/// one stays pending until `sigwait()` consumes it, regardless of its
/// disposition. So this call is what makes target's SIGUSR1 observable, and
/// why nothing here calls `sigaction`: the wait loop below is the only consumer
/// these signals need.
/// defaults to terminating the process. A blocked SIGUSR1
/// stays pending until `sigwait()` consumes it, regardless of its
/// disposition. So this call is what makes target's SIGUSR1 observable, and
/// why nothing here calls `sigaction`: the wait loop below is the only consumer
/// these signals need.

Comment on lines +71 to +74
panic!(
"failed to send SIGUSR1 to ldk-node-wrapper (pid {}): {e}",
self.pid
);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we panic for LDK but warn for CLN in the same situation?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because bad RPC calls already panic inside self.run for CLN, while cases where CLN has already crashed or hung just return Err so the caller can log them. In the future, I think we'll mostly remove this pattern, and will return Err directly from chain_sync or other RPC calls and instead let ir.rs log the RPC error itself, since we may also run commands like addinvoice and want their responses to proceed further

For LDK, the signal-based approach is essentially fire-and-forget. Even if LDK has crashed or hung, its PID remains until the parent reaps it, so the signal is delivered normally (though it won't be responsive) and the call returns Ok. So, any other error is unexpected, so we panic on it

impl TargetRpc for LndRpc {
/// LND receives new blocks directly from bitcoind over ZMQ, so no manual
/// chain synchronization is required.
fn chain_sync(&mut self) {}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: could it make sense to assert the block height here, since after calling this function, I think the caller expects the chain to be synced? But then we would need to pass the expected block height somehow.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I mean that is the defensive way, but the current sync model is essentially fire-and-forget. To actually get the updated block height for each implementation, we would need to wait for some time and then make another RPC call, and I'm not sure how long we should wait or how many times we should retry. In the worst case, we might make several RPC calls, which could hurt the fuzzing itself

So I believe a cheaper way to check whether the blocks have been synced is to run a full funding flow and see if we receive channel_ready from the peer

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants