Skip to content

Feature/multi keychain take 3 - #524

Open
thunderbiscuit wants to merge 7 commits into
pre-multi-keychainfrom
feature/multi-keychain-take-3
Open

Feature/multi keychain take 3#524
thunderbiscuit wants to merge 7 commits into
pre-multi-keychainfrom
feature/multi-keychain-take-3

Conversation

@thunderbiscuit

@thunderbiscuit thunderbiscuit commented Aug 4, 2026

Copy link
Copy Markdown
Member

Take 3 on the multi-keychain PR feature. I'm opening this as a new PR in order to not lose all the work done in #318, since it's a fairly new/reimagined way to work it (2 PRs together, etc.). See #523 for part 1 of this.

Note that this is made against the pre-multi-keychain branch.

Quite experimental at this point, but I wanted to share where I'm at so others can build upon it.

TODO

  • I have removed the APIs related to multipath descriptors as they were focused on 2-path descriptors, but we should now be able to handle all multipath. I think this part needed a bit more thinking and I wanted to punt on it for now to focus on the core of the feature. Multipath is still supported, you simply have to "unpack" them before giving them to the KeyRing.
  • Figure out how to deal with change in the Wallet::create_psbt API. To keep things simple here I require users always set the keychain explicitly, but it's the sort of thing that users would probably love to be able to set once and forget. The example of wallets that use 2 keychains and one of those is the change keychain is the most obvious one.
  • One of the issues left is the Wallet::balance method not being able to tell what's a change keychain and what's not, and so the concept of "trusted" is a bit lost in the current API (all defaults to untrusted until the UTXO is mined). But this is I believe fixed by refactor(wallet): derive balance trust from output ancestry #431 and would land hopefully before this, and so I choose to let this one sit as is for now.

Changelog notice

TODO

Before submitting

@codecov

codecov Bot commented Aug 4, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 79.74684% with 48 lines in your changes missing coverage. Please review.
✅ Project coverage is 82.20%. Comparing base (9ad7073) to head (eaa0946).

Files with missing lines Patch % Lines
src/wallet/persisted.rs 58.82% 14 Missing ⚠️
src/keyring/error.rs 23.07% 10 Missing ⚠️
src/wallet/params.rs 50.00% 8 Missing ⚠️
src/types.rs 68.42% 6 Missing ⚠️
src/wallet/error.rs 0.00% 6 Missing ⚠️
src/wallet/changeset.rs 94.44% 4 Missing ⚠️
Additional details and impacted files
@@                  Coverage Diff                   @@
##           pre-multi-keychain     #524      +/-   ##
======================================================
+ Coverage               81.79%   82.20%   +0.40%     
======================================================
  Files                      25       27       +2     
  Lines                    6362     6371       +9     
  Branches                  289      281       -8     
======================================================
+ Hits                     5204     5237      +33     
+ Misses                   1058     1038      -20     
+ Partials                  100       96       -4     
Flag Coverage Δ
rust 82.20% <79.74%> (+0.40%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@thunderbiscuit

Copy link
Copy Markdown
Member Author

A few ideas for interesting helper features to add to the KeyRing:

  1. A KeyRing::standard method that takes two descriptors and slots them right into KeychainKind::External and KeychainKind::Internal. This is basically allowing anyone who builds very run-of-the-mill wallets to do it quickly and simply, using the current approach of KeychainKind as their K keychain identifier.
  2. A more elegant API for dealing with multipath descriptors. This has been a request for a while and we currently only have so-so methods to work with them, mostly because we only could work with 2 keychains at a time. With the multi-keychain paradigm, really any number of keychains that the multpath descriptors expand to can be added to the wallet. We just need a way to deal with this elegantly (and minimize footguns).

@thunderbiscuit
thunderbiscuit force-pushed the feature/multi-keychain-take-3 branch from b2f34b5 to dd4d5d0 Compare August 5, 2026 15:30
@thunderbiscuit

Copy link
Copy Markdown
Member Author

TODO: Figure out how to deal with change in the Wallet::create_psbt API. To keep things simple here I require users always set the keychain explicitly, but it's the sort of thing that users would probably love to be able to set once and forget. The example of wallets that use 2 keychains and one of those is the change keychain is the most obvious of those, where clearly the user intends for one of those keychains to be used for change for all transactions.

My current naive idea to deal with this would be to add a field on the wallet change: K where users can define a keychain that the Wallet::create_psbt could always use in case no change keychain is set in the psbt creation.

@ValuedMammal happy to hear your thoughts on this.

One thought on this comment: by default the wallet would currently use UTXOs from all keychains to build transactions (maybe that needs work too; kind of like locking UTXOs, you might want to lock keychains or at least prevent the wallet mixing up all your keychains for privacy reasons), but in any case the base workflow would use all keychains. If each of those had defined a sister change keychain, then the wallet is back at square 1, not really knowing which one to use for change. Here again, explicitly setting the keychain to use for change is preferable IMO.

@thunderbiscuit
thunderbiscuit force-pushed the feature/multi-keychain-take-3 branch from c9f4922 to 09c9c5d Compare August 5, 2026 18:14

@thunderbiscuit thunderbiscuit left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Leaving here for review for tomorrow.

Comment thread src/keyring/error.rs
///
/// Two keychains sharing a descriptor cannot be told apart when attributing discovered
/// outputs, so the indexer rejects it.
DescriptorAlreadyAssigned(Box<Descriptor<DescriptorPublicKey>>),

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to confirm where exactly this check will succeed/fail given corner cases, because I know we've discussed this in the past and I need a refresh on it. Posting here for self-review.

@110CodingP 110CodingP Aug 6, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Won't we encounter this, if we try to add a descriptor to the keyring but it already exists under a different keychain?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Won't we encounter this, if we try to add a descriptor to the keyring but it already exists under a different keychain?

Yes and no. The equality check is not perfect if I remember correctly, and we ended up deciding that a hash of the first script derived by the descriptor was a more robust (if not perfect) way to check that two descriptors were basically "the same".

The wallet creation does that check using the descriptor_id() method (what I'm referring to above), so I'll use that instead of the equality used previously.

@thunderbiscuit
thunderbiscuit force-pushed the feature/multi-keychain-take-3 branch from 09c9c5d to eaa0946 Compare August 5, 2026 19:52
Comment thread src/wallet/mod.rs
let tx = &psbt.unsigned_tx;
if psbt.inputs.len() < tx.input.len() {
return Err(IndexOutOfBoundsError::new(
psbt.inputs.len(),

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tx.input.len() ?

Comment thread src/wallet/mod.rs
let index = &mut self.tx_graph.index;
let stage = &mut self.stage;

let ((index, spk), index_changeset) = index
.reveal_next_spk(keychain)
.reveal_next_spk(keychain.clone())
.expect("keychain must exist");

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should return an error instead?

Comment thread src/wallet/mod.rs
let psbt_input = &psbt
.inputs
.get(n)
.ok_or(IndexOutOfBoundsError::new(n, psbt.inputs.len()))?;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have already ensured this won't happen?

Comment thread src/wallet/mod.rs

fee_amount += fee_rate * tx.weight();
/// The derivation index of this wallet. It will return `None` if it has not derived any

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to handle the case when keychain doesn't exist here too.

@110CodingP

110CodingP commented Aug 8, 2026

Copy link
Copy Markdown
Collaborator

My current naive idea to deal with this would be to add a field on the wallet change: K where users can define a keychain that the Wallet::create_psbt could always use in case no change keychain is set in the psbt creation.

If convenience is the only reason for this change can we avoid persisting this? Can we have change_keychains: Vec<K>? The user could supply the value whenever we create/load the wallet. We could the choose the first one if we cannot decide which one to use? If the no change-keychain is specified and also no change_keychain is provided during tx-building we could throw up an error.

@CHEF-SAVY

Copy link
Copy Markdown

Run cargo audit locally to see the full details of the 4 securities issues.

@thunderbiscuit

thunderbiscuit commented Aug 11, 2026

Copy link
Copy Markdown
Member Author

I'll follow up on the comments here in this thread! @ValuedMammal

The privacy focused approach is to choose the change keychain based on the output type of the outputs of the transaction to be created, not the keychain of the utxos pulled in as inputs.

You're correct I got this backwards there for a second.

the cleanest way to define the change keychain is at wallet creation time, because it's tied directly to a wallet descriptor. Supplying the change keychains another way means you'll have to handle a possibly non-existing keychain.

Yeah I see your point there. It conflicts with @110CodingP's idea above ("If convenience is the only reason for this change can we avoid persisting this?"), but if we really value this idea of "set it and forget it" then I think adding it to persistence could be worth the trouble.


On having multiple change keychains

I don't personally like the idea of defining more than 1 change keychain wallet-wide at the Wallet level (and even then! But I don't like change keychains at all to be honest so I understand I come at this a bit biased). Overall, I think it implies the wallet can do more than what I think it can reasonably do (picking correctly between multiple change is just a hard problem that's better handled outside of the wallet IMO, see next section). Defining them app-layer within your keychain identifiers K is of course perfect for this, and can be used powerfully.

I think a reasonable feature would be to be able to define a single global change keychain, which would come into play when going down the order of preference for change keychain selection in the psbt creation process as you defined in the linked post. This would also facilitate and keep simple the "basic case" of 2-keychain wallets as we have them now in Wallet.


Order of preference for change

So for PSBT creation the order of precedence is

  1. User provided change-script/descriptor via the params
  2. Auto derive from a change keychain the Wallet/KeyRing knows about
  3. A third option could be a single globally configurable K value, but the real benefit of this comes from persisting it, so you can truly set it and forget it

Thanks for laying this out. I would do exactly this, but remove step 2. Or maybe I can be convinced, or it could come as a new feature at some point?


On choosing the change based on payment scripts

For me, this lives better at the application layer than the bdk_wallet layer. The basic privacy workflow is this:

  1. Figure out what type of script I'm paying to
  2. Pick a change script of the same type to ensure alll outputs are of the same script type

Attempting to do this on the bdk_wallet side is probably not impossible, but feels messy to me and very hard to do well, because the wallet would need to know so much about the user/app layer, i.e. carry a lot of metadata on all keychains and when to use which, which is exactly what the K approach is powerful for: letting the user hold keychain metadata with the descriptors, and leverage this for transaction building.

Even for the base case, the wallet would need to be able to parse the script type being paid to (what if it's a few different ones?), then see if it has keychains that are marked as change and have the same script type (again what if you pay to a tapscript address and all your 7 keychains are tapscript keychains?), and then pick one of those (which one? random? me don't like random). Again not impossible by any means, but this feels perfect for business-logic added on the application layer (at least to me at the moment).

@thunderbiscuit

Copy link
Copy Markdown
Member Author

This was a long post but here is what I suggest I focus on for the next round of this:

  1. Make sure the current 2-keychains External/Internal flow is clean and simple to use, with similar features to what we have now.
  2. Keep complete flexibility for what I expect is more advanced + custom users of the library (those who define many keychains).

@110CodingP

110CodingP commented Aug 13, 2026

Copy link
Copy Markdown
Collaborator

Do we want to allow the change_keychain or internal marker to be modified after wallet creation? Then this would also require a new field on the ChangeSet !

Comment thread src/wallet/changeset.rs
/// change descriptor should be defined at wallet creation time and respected for the life of the
/// wallet, meaning that if a change descriptor is originally defined, it must also be present in
/// the aggregate change set.
/// A keychain may be introduced by a later change set — a wallet can start tracking a new one at

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we planning to allow addition of descriptors in bdk_wallet 4.0 itself?

Comment thread src/wallet/changeset.rs
pub change_descriptor: Option<Descriptor<DescriptorPublicKey>>,
#[derive(Debug, Clone, PartialEq, Deserialize, Serialize)]
pub struct ChangeSet<K: Ord> {
/// The descriptor tracked by each keychain.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mammal brought this up on #318 : Since we need to follow the ChangeSet's field-deprecation policy we would need to keep the change_descriptor and descriptor fields around for v4.

@110CodingP

110CodingP commented Aug 14, 2026

Copy link
Copy Markdown
Collaborator

Also ig we could just let the user specify the change_keychain in this PR and have another PR (fixing the way user specifies the change keychain) when we have an agreement over the final API?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

3 participants