Repository navigation
Add tested NSA-inspired novelty detection with eleven generation strategies - #747
kishordgupta wants to merge 4 commits into
Conversation
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 587fa62856
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Coverage Report for CI Build 35058654472Warning Build has drifted: This PR's base is out of sync with its target branch, so coverage data may include unrelated changes. Coverage increased (+0.7%) to 93.25%Details
Uncovered Changes
Coverage Regressions4 previously-covered lines in 4 files lost coverage.
Coverage Stats
💛 - Coveralls |
kishordgupta
changed the title
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f7a9d73040
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Carry detector and candidate quotas forward to finer hierarchy levels, add tiny-budget regressions, and synchronize with upstream development at 97fa2b4.
|
Updated the branch in 6a856c4 with the hierarchical-budget fix and a normal merge of current development at 97fa2b4. GitHub comparison now reports zero commits behind development. For the Coveralls drift notice: build 81781369 compared the August base build at ad39845 against a synthetic PR merge containing newer upstream commits. The four listed regression files (COPOD, ECOD, DeepSVDD tests, and utility.py) were changed upstream and are not edited by this PR, so that report includes changes outside the contribution. The refreshed CI run is https://github.com/yzhao062/pyod/actions/runs/35111771147; its coverage result is still pending. Local validation of the published source: 428 NSA tests plus 13 subtests passed; 359 integration/upstream-regression tests passed with one skip; NSA/helper statement-and-branch coverage remains 98%. PyOD 3.6.6 wheel/sdist builds, metadata checks, strict lint, generated-reference checks, and the real-data example passed. Sphinx builds successfully with existing unrelated documentation diagnostics. |
yzhao062
left a comment
yzhao062
left a comment
There was a problem hiding this comment.
Thank you for the substantial rewrite. Before an inclusion decision, please state the operating regime where NSA is the right choice and show representative evidence for it. The demonstration in the description reports ROC AUC of 0.27 to 0.65 against 0.97 for IForest on one dataset.
Please also drop the lazy package-level NSA export, since PyOD imports detectors from their modules. The pull request has grown to about 4,500 lines across 26 files; a slimmer core would be easier to review and maintain.
|
Thank you for the feedback. I ran a broader benchmark to identify the regime where NSA may be useful. The clearest result is for Dual NSA under a normal-only cold start. Normal-only cold start means training a detector from a small set of confirmed normal observations, with no real outlier examples used in preprocessing or initial fitting. The benchmark covers 17 ODDS datasets and five seeds, with all 43 other PyOD tabular entries accounted for, including unsuccessful and inapplicable configurations. In the PCA(2) pipeline, Dual NSA's results are:
The 12-page paper provides the per-dataset comparisons and Total rows in Tables 2–5; Table 1 summarizes the settings. These are equally weighted dataset means. The comparison field stays fixed at 25 fully covered PyOD baselines plus Dual, and top position means This gives evidence for a competitive initial option under scarce normal data. KDE remains ahead in mean AUROC at the two smallest sizes, Dual's AP declines, and results vary across datasets. The sensor cold-start use case still needs chronological validation. The code, raw results, and full paper with the appendix are available in the same repository. Would this narrower operating regime and the broader evidence provide a convincing case for a smaller NSA contribution? If so, I can restructure the PR into a slimmer documented core, remove the lazy package-level export, and use the usual |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 277326a480
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| np.r_[np.zeros(dimension), -1.0], | ||
| A_ub=np.column_stack((normals, np.ones(len(normals)))), | ||
| b_ub=-offsets, bounds=[(None, None)] * dimension + [(0, None)], | ||
| method='highs') |
There was a problem hiding this comment.
In environments permitted by requirements.txt with SciPy 1.5.x, linprog does not yet support the highs method (it was added in SciPy 1.6), so every NSA(strategy='voronoi') fit fails with an unknown-solver error before constructing a cell. Use a solver available in SciPy 1.5 or raise the package's minimum SciPy requirement.
Useful? React with 👍 / 👎.
3 participants
This adds one
NSAnovelty detector for training on known normal/self samples, with eleven distinct generation strategies and thirteen configurations including the three binary matching rules. Each strategy has a substantive implementation and explicit NSA-inspired labeling.The strategies are binary, fixed, variable, coverage, grid, hierarchical, voronoi, deterministic, suppressed, dual and annealed. Primary citations and the precise differences from the papers are documented. Statistical coverage stopping uses frozen-population exact-binomial tests with alpha spending; budget exhaustion is never claimed as coverage success. Hierarchical allocation reserves capacity for finer levels even when only one detector or candidate draw is available. Geometry limits, bounded clustering and annealing schedules are explicit. No unsupported method acronyms or published-performance claims are exposed.
The estimator follows BaseDetector scoring and threshold conventions, is exported through the package and ADEngine registry, and has runnable real-data examples with an IForest baseline. All supplied rows remain self; contamination calibrates the score threshold. Any fit failure clears every learned marker, including those recognized by generic sklearn
check_is_fitted, so a failed refit cannot leave a stale fitted state.Local validation on the source synchronized with PyOD 3.6.6: 428 NSA tests and 13 subtests passed; 359 integration and upstream-regression tests passed, with 1 skipped; 98% combined statement/branch coverage across NSA and its helpers. Strict flake8, generated-reference consistency, wheel/sdist build and package metadata checks pass. Sphinx builds successfully with existing documentation diagnostics in unrelated modules and no NSA-specific diagnostics. The PCA2 real-data example preserves the same train/test split for all compared methods and reports IForest's stronger result; no ranking or automatic routing rule is added.
I commit to maintaining this contributed detector for at least two years, including compatibility fixes, tests and documentation.