Repository navigation
feat(ai): make the AI decision provider selectable and add Drex - #346
Open
chichengmessi wants to merge 2 commits into
Open
chichengmessi wants to merge 2 commits into
chichengmessi wants to merge 2 commits into
Conversation
7 of 8 tasks
AI_DECISION_PROVIDER picks the System One model the pre-trade filter calls first: jev (default, unchanged) or drex. Drex is called with NACE_API_KEY and NACE_DEFAULT_DECISION_MODEL (default drex-max-finance) through the same questions, validation, confidence threshold, decision converger and fallbacks. The audit trail records provider "drex" and its reasons as drex_entry_*; Jev requests, reasons and error text are unchanged. Signed-off-by: Cheng Chi (Bill) <cheng@nace.ai>
Drex requests now send X-Nace-Client and User-Agent as quantdinger/<APP_VERSION>, so Nace can attribute traffic per integration. Jev requests are unchanged. Signed-off-by: Cheng Chi (Bill) <cheng@nace.ai>
chichengmessi
force-pushed
the
drex/decision-filter
branch
from
November 16, 2026 06:12
780e6f6 to
a2b4a53
Compare
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



Summary
Adds
AI_DECISION_PROVIDERso the pre-trade AI decision filter can call Nace Drex instead of Jev. Drex serves the samePOST /v1/systemonecontract, so it goes through the existing questions, answer validation, confidence threshold, decision rules, billing, LLM fallback and fail-open. With the setting unset orjev, nothing changes.Pointing the
JEV_*settings at Drex already works, but the decision timeline then labels Drex decisions asjev.To switch: in System Settings → AI / LLM, set AI Decision Provider to "Nace Drex" and paste the Nace API key (or set
AI_DECISION_PROVIDER=drexandNACE_API_KEY). NoJEV_*setting changes.Related issue
No issue yet. This touches the entry filter, which CONTRIBUTING treats as execution-critical; I'm happy to open a scoped issue or discussion first if you prefer.
Changes
ai_decision_filter.py:AI_DECISION_PROVIDER=drexreadsNACE_API_KEYandNACE_DEFAULT_DECISION_MODEL(defaultdrex-max-finance) with the base URL fixed tohttps://console.nace.ai/v1; timeout and threshold reuseJEV_TIMEOUT_SECONDS/JEV_MIN_CONFIDENCE. Decisions are stored with providerdrex, reasonsdrex_entry_approved/drex_entry_rejected:*, and fallback reasonsdrex:.... Jev requests, reasons and messages are unchanged; the shared settings-refresh debug log no longer says Jev. Drex requests sendX-Nace-ClientandUser-Agentasquantdinger/<APP_VERSION>so Nace can attribute traffic per integration; Jev requests get neither._jev_config/_evaluate_jevare renamed to_system_one_config/_evaluate_system_one.JEV_ENABLEDapplies to Jev only: choosing Drex turns Drex on.settings.py:AI_DECISION_PROVIDER,NACE_API_KEYandNACE_DEFAULT_DECISION_MODELin System Settings → AI / LLM.env.example,.env.example,README.md,README_CN.md: the three keys and one paragraph.test_ai_decision_filter.py: Drex settings, a Drex rejection audited asdrex(with its attribution headers asserted), a Drex error falling back to the LLM, and no Nace headers on Jev requests.Frontend follow-up: several decision-filter strings say JEV whatever the provider. For example, a Drex fallback shows "JEV confidence was below the configured threshold", and the
drex_entry_*reasons have no translations, so they show as raw codes. The settings page places onlyJEV_*keys in the AI Decision Filter card, so the three new fields appear under the common AI settings. Usingrecord.providerin those strings and adding the three keys to that card would cover it.Test plan
pytest tests/test_ai_decision_filter.py tests/test_strategy_v2_order_gateway.py tests/test_settings_secret_masking.py tests/test_sync_env.py tests/test_bootstrap_env.py tests/test_professional_report_settings_coverage.py tests/test_ai_agent_contract_alignment.py tests/test_strategy_runtime_settings.py(73 passed)ruff check,scripts/check_docs.py,git diff --checkmain: identical Jev request (URL, headers, body) and stored result for a valid, a low-confidence and a malformed answer, with and withoutNACE_API_KEYsetStrategyV2OrderGateway.submitwith the filter on (no pending-order worker, no exchange): Drex rejected a BTCUSDT long against the trend (drex_entry_rejected:signal_conflict, 370 ms) and approved the short (357 ms); an invalid key fell back to the LLM (drex:401 ...); no keys gaveai_not_configuredas before; provider unset used Jev as beforeAIDecisionFilter.evaluatesentX-Nace-Client/User-Agent: quantdinger/<APP_VERSION>and returned a normal decisiondrex, settings-file refresh forced to fail: no log line, result or stored decision mentions Jev, for a Drex decision or a Drex 401 that fell back to the LLMAPI documentation (if routes/schemas changed)
export_openapi.pyoutput identical)Drex vs Jev
Nace's Financial Decisions Benchmark (48 held-out finance benchmarks, 3,361 decisions):
Benchmark chart and details: Drex for Finance announcement.
On this filter: 100 entries on 10 market states, each in a strong trend on 1h, 4h and 1d. Per state there are 4 clean entries and 6 entries that each break one of the filter's own blocking criteria. Each model was called through
AIDecisionFilterat the default 0.55 threshold;jev-latestanswered asjev-1.14.0.gpt-4o-mini)risk_checkorexecution_qualityconfidence is under 0.55 on 98 of 100 entries. The filter hands those to the LLM fallback, or allows the order when no LLM is configured.usage.cost, at 2,798 tokens per decision.Questions about Drex: Nace Discord.