feat: wot core service#642
Conversation
🦋 Changeset detectedLatest commit: 47b2fc6 The changes in this PR will be included in the next version bump. Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
coverage: 66.831% (-0.2%) from 66.995% — saniddhyaDubey:feat/wot#2-service into cameri:main |
| relayUrls.map((url) => fetchEvents(url, filter, timeoutMs)) | ||
| ) | ||
|
|
||
| const seen = new Set<string>() |
There was a problem hiding this comment.
this dedup in memory is unbounded, so for a popular user following many other usrs this can cause Nostream to get OOM-killed
In these scenarios, we are better off using a bloom filter, and use streams to process the information so the amount of memory used is manageable
There was a problem hiding this comment.
Looking on a more robust stream based approach to keep memory bounded as well.
| */ | ||
| public constructor(private readonly fetcher: RelayFetcher = fetchFromRelays) {} | ||
|
|
||
| public async buildGraph(settings: WoTSettings): Promise<void> { |
There was a problem hiding this comment.
the amount of memory used by this function will also be quite significant, would it be a better choice to try and build the graph using PostgreSQL and use queries instead?
There was a problem hiding this comment.
One option to bound the in-memory footprint without switching to PostgreSQL: cap the followerCount Map at a configurable max size (e.g. 100k entries). Once a pubkey's count reaches minimumFollowers we stop incrementing, and once the map hits the size cap we stop adding new keys entirely. At 100k entries that's ~15 MB worst case, predictable and manageable. The tradeoff is that pubkeys encountered after the cap is hit get silently excluded, but for most relay operators that's acceptable since the most-followed pubkeys in the network will naturally appear first (only considering followerCount Map because Bloom filter + streaming is being implemented to make dedup memory bounded).
That said, the PostgreSQL approach is architecturally cleaner, event_tags already has kind-3 data normalized and indexed, so a GROUP BY + COUNT query runs entirely in the DB and returns only the final trusted pubkey list to Node. Zero accumulation on the Node side and exact dedup via DISTINCT.
The one tradeoff with PostgreSQL: it can only trust pubkeys whose kind-3 events have actually been relayed through this nostream instance, whereas the external relay fetch pulls from damus.io, nostr.band etc. regardless of what's been published here.
Happy to go either direction!
There was a problem hiding this comment.
That said, the PostgreSQL approach is architecturally cleaner, event_tags already has kind-3 data normalized and indexed, so a GROUP BY + COUNT query runs entirely in the DB and returns only the final trusted pubkey list to Node. Zero accumulation on the Node side and exact dedup via DISTINCT.
We should go with PostgreSQL (or Redis for caching) so that we don't have potential OOM issues, introduce artificial caps (100k limit), or surprises (silently excluding a pubkey). Imagine a new pubkey who is just followed and falls outside the cap: this user will remain outside, and the operator will think the relay isn't working.
The one tradeoff with PostgreSQL: it can only trust pubkeys whose kind-3 events have actually been relayed through this nostream instance, whereas the external relay fetch pulls from damus.io, nostr.band etc. regardless of what's been published here.
If the kind-3 events from seed relays are inserted into our database this trade-off goes away. By the way, we have a static mirroring working implementation. Can it be re-used for WoT? It already connects to other relays and fetches events.
There was a problem hiding this comment.
Pull request overview
Introduces a new WotService implementing the core Web of Trust (WoT) graph-building logic, along with settings/type updates and unit tests to support and validate the new behavior.
Changes:
- Adds
WotServiceto build a 2-hop trust graph by fetching kind-3 follow lists from configured seed relays. - Extends WoT settings to include
seedRelays, updates default settings, and adjusts settings unit tests. - Adds a comprehensive unit test suite for
WotService.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
src/services/wot-service.ts |
Adds the WoT graph building logic with relay fetching, batching, and trust-map swapping. |
src/@types/settings.ts |
Extends WoTSettings with seedRelays. |
src/@types/services.ts |
Introduces the IWotService interface. |
resources/default-settings.yaml |
Adds default wot.seedRelays values. |
test/unit/services/wot-service.spec.ts |
Adds unit tests covering WoT service behavior and edge cases. |
test/unit/utils/settings.spec.ts |
Updates settings tests to assert seedRelays defaulting and override behavior. |
.knip.json |
Ignores the new service file for unused-code checks. |
.changeset/solid-boats-try.md |
Adds a changeset entry for the new feature. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| minimumFollowers: 1 | ||
| # Hours between full trust graph rebuilds. | ||
| refreshIntervalHours: 24 | ||
| # Relay URLs to fetch follow lists from when building the trust graph. |
| /** | ||
| * Relay URLs to fetch follow lists from when building the trust graph. | ||
| * Falls back to the relay's own URL if empty. | ||
| */ | ||
| seedRelays: string[] |
| }) | ||
| }) | ||
|
|
||
| // testting reset functionality: |
Description
Introduces WotService - the core Web of Trust graph logic. Builds a 2-hop trust graph rooted at the relay owner's pubkey by fetching kind-3 follow lists from configured seed relays.
Also patches the wot: config block (#623) with the missing seedRelays field, and updates the settings unit tests accordingly.
Related Issue
Closes (#626).
How Has This Been Tested?
14 new unit tests in
test/unit/services/wot-service.spec.tscover:seedPubkeyalways trusted regardless of follower countminimumFollowersthreshold : pubkeys above pass, pubkeys below are rejectedbuildGraph()calls are no-opsbuildingflag cleared correctly even when the build throwsptags and pubkeys shorter than 64 chars ignoredreset()clears all state and allows a fresh buildScreenshots (if appropriate):
Types of changes
Checklist: