fix(hashlogcompare): drop the torn tail of a sealed hash log file

[bdchatham]

Summary

On atlantic-2, the reserve reader of canary-hashlog-comparator stalled for about 30 minutes after a pod restart. It logged no error and counted no source error, and only a restart of the comparator recovered it. The cause was a sealed hash log file whose last line a crash had torn. Reader.Poll waited for that line to finish, so it never called nextFile. This PR makes the reader drop that line and report it (PLT-1398).

Change

• On a sealed file, Poll moves the cursor past any bytes after the last newline and returns errTornRow. The error names the file and the byte count. The next poll reads zero bytes at end of file and follows the existing nextFile path.
• run.go maps errTornRow to a new reason, torn_row. It goes through the existing WARN log and source_errors_total increment. initPair exports the new series at zero.
• The Reader doc states the sei-chain guarantee that this rests on. ReadHashLogFile discards a torn last line, and sealHashLog removes an orphan with no complete row. Together, they mean that a sealed file always has a complete header and at least one row. Only its last line can be incomplete.
• Alert effect: one torn row does not fire HashLogSourceErrors, which needs 5 minutes of errors. If the restarted node does not write that height again, HashLogHeightGaps fires.

Verification

• Two new tests: TestReader_DropsTornTailOfSealedFile and TestPairRunner_TornReserveRowIsSourceError. Both fail without the change, with no error and no progress.
• The root module passes gofmt, go vet, go test (as make test runs it), and golangci-lint v2.12.1 with no filter on the package. staticcheck is clean on the package.
• Next: bump the comparator image in platform, then watch source_height on the atlantic-2 pair.

Refs: PLT-1398

🤖 Generated with Claude Code

[cursor]

PR Summary

Medium Risk
Changes hash-log tailing behavior for sealed files (could drop a height row) and adds a new Prometheus error reason; limited to the comparator package with tests.

Overview
Fixes a hashlog comparator stall when a sealed hash log file ends with a crash-torn incomplete line: the reader used to wait forever for that line instead of advancing to the next file.

Reader now treats trailing bytes after the last newline on a sealed file as a torn row—skips them, advances the cursor, and returns errTornRow (while still returning any complete rows parsed above the tear). The next poll can proceed via nextFile as usual. Open/unsealed files still wait for partial lines.

Observability: run maps errTornRow to a new torn_row source_errors_total reason (WARN log + counter); initPair pre-exports that label series. Tests cover reader recovery and pair-runner metrics.

^{Reviewed by Cursor Bugbot for commit 385a85a. Bugbot is set up for automated code reviews on this repo. Configure here.}

[seidroid]

On a sealed hash log file, the reader now skips an incomplete last line and reports it as a torn_row source error, so it moves on to the next file instead of waiting forever. This fixes the PLT-1398 stall, and I found nothing blocking: the sidecar returns the whole requested range with no size cap, so the reader only drops bytes that are really torn. codex's reading found nothing and I agree, so it gave me no findings to keep or drop; I could not run the tests because Go is not installed here.

Non-blocking

• The fix covers only a sealed file whose rows are torn. If a sealed file's header line has no newline, Poll still hits the r.header == nil branch and returns no error, and it does that on every poll forever. That is the same silent stall this PR fixes, and only the sei-chain sealing guarantee stops it from happening. Consider returning an error there when r.sealed is true, so a broken guarantee shows up as a source error rather than a hang.

_{seidroid review · decision approve · session a2943639c36548e38904a21f40200e01 · turn resp_claude_de25abf82c1c8a44a4e7390e1882afd1 · item 48fb903a1afd520a9a15a7a510780237}

Findings: 0 blocking | 1 non-blocking | 0 posted inline