Repository navigation
Add Deep SAD (Deep Semi-Supervised Anomaly Detection) detector - #716
JayeshSuryavanshi wants to merge 9 commits into
Conversation
Implement DeepSAD (Ruff et al., ICLR 2020), the semi-supervised extension of Deep SVDD, as a new PyOD BaseDetector. Unlabeled/normal samples use the unsupervised Deep SVDD objective (minimize squared distance to the hypersphere center c) while labeled anomalies use the inverse-distance term (eta * dist**-1), pushing them away from c. fit(X, y=None) treats y==1 as known anomalies and everything else as unlabeled; y=None recovers pure Deep SVDD behavior. Mirrors the structure and API of pyod/models/deep_svdd.py. - pyod/models/deep_sad.py: DeepSAD + InnerDeepSAD encoder - pyod/test/test_deep_sad.py: mirrors test_deepsvdd (semi-supervised fit) - examples/deep_sad_example.py: semi-supervised usage example - README.rst: model-table row + reference - docs/zreferences.bib: ruff2020deepsad citation Loss/architecture adapted from lukasruff/Deep-SAD-PyTorch (MIT) and DeepOD's dsad.py (BSD-3); written clean in PyOD style.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 11f0649173
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| optimizer = optimizer_dict[self.optimizer]( | ||
| self.model_.parameters(), weight_decay=self.l2_regularizer) |
There was a problem hiding this comment.
With the default l2_regularizer=0.1, this sends weight_decay=0.1 into the optimizer for every default fit. In the default bias-free encoder, that decay aggressively pulls weights toward zero and can collapse the learned representation and scores even when callers do not tune anything; use a reference-scale weight decay, or separate activity regularization from optimizer decay, instead of training DeepSAD with this default.
Useful? React with 👍 / 👎.
| if self.random_state is not None: | ||
| torch.manual_seed(self.random_state) |
There was a problem hiding this comment.
Seeding the global torch RNG in __init__ makes reproducibility depend on what happens between construction and fit: if two DeepSAD(random_state=...) instances or sklearn clones are created before either is fitted, fitting the first advances the global RNG before the second builds its network/DataLoader, so they can initialize differently despite the same seed. Apply the seed inside fit or use a per-fit generator so the documented random_state actually makes training repeatable.
Useful? React with 👍 / 👎.
Per the automated review on the PR: - Remove the hidden_neurons > n_features check. Deep SAD uses only an encoder (no autoencoder branch like Deep SVDD), so hidden layers may legitimately be wider than the input dimension; the check wrongly rejected valid setups on datasets with fewer features than the default hidden width. - Validate a user-supplied c: it must match the representation dimension (hidden_neurons[-1]) and be finite, instead of silently broadcasting a scalar or a mismatched vector. Add tests for both.
|
Thanks for the review. I addressed the two Deep-SAD-specific points in feb561a:
I intentionally left the other two suggestions (the |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: feb561aad8
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
…seed torch from a RandomState, drop unsupported optimizers
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 148f05fe69
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
…pply dropout after every hidden activation, require positive eta, add DeepSAD to the docs, skip the tests without torch, train the test fixtures once
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3b1c56c8cf
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
…n the training fold, weight the epoch loss by batch size, validate eps, handle a single-width network
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 61e36bee54
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| n_val = min(int(round(len(idx) * self.validation_size)), | ||
| len(idx) - 1) |
There was a problem hiding this comment.
When the labeled-anomaly class is small, this rounding assigns none of it to validation—for example, the default validation_size=0.1 gives n_val == 0 for five or fewer labeled anomalies. The validation fold can still contain unlabeled samples, so checkpoint selection then silently optimizes only the hypersphere-compaction term and may restore an epoch that lost the supervised anomaly separation. Preserve the anomaly component in the selection loss, such as by combining training-anomaly loss with validation-unlabeled loss, while still keeping lone anomalies in training.
Useful? React with 👍 / 👎.
| torch.manual_seed(torch_seed) | ||
| check_parameter(dropout_rate, 0, 1, param_name='dropout_rate', | ||
| include_left=True) | ||
| check_parameter(eta, 0, np.inf, param_name='eta') |
There was a problem hiding this comment.
When eta=np.nan, check_parameter accepts it because every ordered comparison with NaN is false. If any labeled anomaly is present, the inverse-distance term becomes NaN, optimizer updates can corrupt the weights, and no epoch can satisfy selection_loss < best_loss, leaving users with nonfinite scores and no restored checkpoint. Add an explicit finiteness check for eta, as is already done for eps.
Useful? React with 👍 / 👎.
| optimizer = optimizer_dict[self.optimizer]( | ||
| self.model_.parameters(), weight_decay=self.l2_regularizer) | ||
|
|
||
| for epoch in range(self.epochs): |
There was a problem hiding this comment.
When epochs is zero or negative, this loop is skipped without an error, best_model_dict remains None, and fit still returns a seemingly fitted detector whose scores and threshold come from the randomly initialized network. Validate that epochs is a positive integer so a configuration error cannot silently produce untrained experimental results.
Useful? React with 👍 / 👎.
| if y is not None: | ||
| y = np.asarray(y).ravel() | ||
| semi_targets[y == 1] = -1 |
There was a problem hiding this comment.
When y contains values other than the documented 0/1 encoding, they are silently treated as unlabeled rather than rejected. In particular, passing sklearn-style outlier labels (1 for inliers and -1 for outliers) marks every inlier as a known anomaly while leaving the actual outliers unlabeled, reversing the supervision and corrupting the fitted detector without an error. Validate that supplied targets are one-dimensional, length-matched, and limited to {0, 1} before this conversion.
Useful? React with 👍 / 👎.
yzhao062
left a comment
yzhao062
left a comment
There was a problem hiding this comment.
Thank you for Deep SAD, and sorry this review comes late. Three changes are needed before it can merge:
decision_functionneeds a fitted-state check. On an unfitted estimator it currently raisesAttributeError.- Seed at fit time instead of in the constructor. Constructor seeding changes global Torch state, so two estimators built before either is fitted interfere. Please add a reproducibility test for that case.
- Deep SAD is semi-supervised, so it should not be routed to users who have no labels. Please state how it should appear in the model registry and in ADEngine routing.
decision_function now raises NotFittedError on an unfitted estimator instead of an AttributeError. The torch seed moves from __init__ to the start of fit, so building other estimators before fitting no longer changes the result. Scores for an estimator built and fitted right away are unchanged.
…es not route it ADEngine passes no labels, so Deep SAD would run there as plain Deep SVDD. As an experimental entry it is listed in the knowledge base and by explain_detector, but rule routing, the routing context for LLM agents, and make_plan all skip it. Detector counts move to 62 (44 tabular) and the tabular skill reference is regenerated.
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: aa6da4e997
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| optimizer = optimizer_dict[self.optimizer]( | ||
| self.model_.parameters(), weight_decay=self.l2_regularizer) |
There was a problem hiding this comment.
When a dataset needs a non-default step size, there is no supported way to configure it: this call omits lr and DeepSAD.__init__ has no learning-rate parameter, so each advertised optimizer silently uses a different PyTorch default (for example, Adam uses 1e-3 while Adadelta uses 1.0). This also prevents reproducing the sibling implementation's learning_rate=1e-4, which DeepSVDD explicitly forwards in pyod/models/deep_svdd.py:396-398; add and pass a learning-rate parameter here so experiments can control the optimization scale.
Useful? React with 👍 / 👎.
2 participants
Add Deep SAD (Deep Semi-Supervised Anomaly Detection)
Closes #709.
This adds
DeepSAD, the semi-supervised extension of the existingDeepSVDD, from Ruff et al., Deep Semi-Supervised Anomaly Detection, ICLR 2020.Motivation
PyOD ships
DeepSVDDbut not its widely-cited semi-supervised sibling. In fraud/risk and many real-world anomaly settings a small number of confirmed anomalies is available, and Deep SAD is the canonical way to use those labels: it keeps the Deep SVDD hypersphere objective for unlabeled data while pulling labeled anomalies away from the center, which measurably improves detection over the purely-unsupervised model.What it does
A network maps inputs into a representation space where inliers are pulled toward a fixed hypersphere center
cand labeled anomalies are pushed away from it, via the Deep SAD objective:c;(dist)^(-1), so their loss grows as they approach the center.The distance to
cis the anomaly score at inference. The semi-supervised interface is a single optionalyinfit(X, y=None)wherey == 1marks a known anomaly and everything else is unlabeled;y=Nonerecovers unsupervised Deep SVDD, so it drops in whereverDeepSVDDis used.Implementation notes
pyod/models/deep_svdd.py(sameBaseDetectorcontract, standardization, center initialization from a first forward pass,decision_functionreturning the distance-to-center score).lukasruff/Deep-SAD-PyTorch, MIT) and DeepOD'sdsad.py(BSD-3); neither is copied verbatim and both are attributed in the module header.Included
pyod/models/deep_sad.py— the detectorpyod/test/test_deep_sad.py— 15 tests mirroring the DeepSVDD test suite (parameters, scores, labels, proba, confidence, rejection,fit_predict,clone), all passing locallyexamples/deep_sad_example.pyREADME.rstmodel table entry +docs/zreferences.bibcitationScope
This is the core detector plus docs/tests. I deliberately did not add DeepSAD to the
ad_engineknowledge base (algorithms.json,benchmarks.json, routing rules, per-model analysis) because those entries carry curated metadata and benchmark numbers I don't want to fabricate; the skill/KB consistency checks pass as-is. Happy to add a proper KB entry as a follow-up if you'd like, or if you can point me at the benchmark source.