perf: harden gateway, billing, and ops paths under high concurrency by DR-lin-eng · Pull Request #4337 · Wei-Shaw/sub2api

DR-lin-eng · 2026-07-15T05:50:55Z

Summary

bound OpenAI request and response buffering while improving channel-aware failover behavior
make usage billing durable under load with queued persistence, bounded fallback, and mutation-version cache guards
consolidate dashboard reads and add Redis-backed operational and runtime controls

Key changes

add durable usage billing queueing with synchronous fallback when critical work cannot be enqueued
prevent stale cache repopulation after billing and balance mutations
bound request bodies, SSE scanning, session state, OAuth refresh locks, and account runtime state
improve channel-aware scheduling, retry handling, and temporary account exclusion
add shared dashboard snapshots and merged usage aggregations
add Redis-only system logs, runtime retention overrides, user usage statistics, and explicit clear-all operations
add a global temporary-unschedulable switch with cross-layer cleanup
expose the new scheduling, logging, retention, and usage controls in the admin UI

Testing

go test ./...
golangci-lint v2.9.0 run --timeout=30m ./...
make test-frontend
pnpm --dir frontend run build

I have read the CLA Document and I hereby sign the CLA

github-actions · 2026-07-15T05:51:03Z

Thank you for your contribution! Before we can merge this PR, we need you all to sign our Contributor License Agreement (CLA).

To sign, please reply with the following comment:

I have read the CLA Document and I hereby sign the CLA

You only need to sign once — it will be valid for all your future contributions to this project.

I have read the CLA Document and I hereby sign the CLA

14 out of 15 committers have signed the CLA.
✅ (DR-lin-eng)[https://github.com/DR-lin-eng]
✅ (BenjaminAaron196)[https://github.com/BenjaminAaron196]
✅ (yardbirds0)[https://github.com/yardbirds0]
✅ (Eyre921)[https://github.com/Eyre921]
✅ (dftian478)[https://github.com/dftian478]
✅ (SatellaPoi)[https://github.com/SatellaPoi]
✅ (wucm667)[https://github.com/wucm667]
✅ (aomsir)[https://github.com/aomsir]
✅ (adamglin0)[https://github.com/adamglin0]
✅ (feitianbubu)[https://github.com/feitianbubu]
✅ (DylanChiang-Dev)[https://github.com/DylanChiang-Dev]
✅ (AixLau)[https://github.com/AixLau]
✅ (heathermhuang)[https://github.com/heathermhuang]
✅ (mark-ly-wang)[https://github.com/mark-ly-wang]
❌ @CRAZYShimakaze
_{You can retrigger this bot by commenting recheck in this Pull Request.}_{Posted by the CLA Assistant Lite bot.}

DR-lin-eng · 2026-07-15T05:52:20Z

Thank you for your contribution! Before we can merge this PR, we need you to sign our Contributor License Agreement (CLA).

To sign, please reply with the following comment:

I have read the CLA Document and I hereby sign the CLA

You only need to sign once — it will be valid for all your future contributions to this project.

I have read the CLA Document and I hereby sign the CLA

You can retrigger this bot by commenting recheck in this Pull Request. Posted by the CLA Assistant Lite bot.

I have read the CLA Document and I hereby sign the CLA

DR-lin-eng · 2026-07-15T05:53:01Z

Thank you for your contribution! Before we can merge this PR, we need you to sign our Contributor License Agreement (CLA).
To sign, please reply with the following comment:

I have read the CLA Document and I hereby sign the CLA

You only need to sign once — it will be valid for all your future contributions to this project.
I have read the CLA Document and I hereby sign the CLA
You can retrigger this bot by commenting recheck in this Pull Request. Posted by the CLA Assistant Lite bot.

I have read the CLA Document and I hereby sign the CLA

recheck

DR-lin-eng · 2026-07-15T05:53:34Z

Thank you for your contribution! Before we can merge this PR, we need you to sign our Contributor License Agreement (CLA).

To sign, please reply with the following comment:

I have read the CLA Document and I hereby sign the CLA

You only need to sign once — it will be valid for all your future contributions to this project.

I have read the CLA Document and I hereby sign the CLA

You can retrigger this bot by commenting recheck in this Pull Request. Posted by the CLA Assistant Lite bot.

recheck

DR-lin-eng · 2026-07-15T05:54:18Z

recheck

DR-lin-eng · 2026-07-15T05:57:00Z

I have read the CLA Document and I hereby sign the CLA

DR-lin-eng · 2026-07-15T05:57:11Z

recheck

…index drift Close open text parts before late reasoning/tool items, and only stop matching Anthropic content-block types so finalize cannot close tool_use with a spurious output_text.done. Co-authored-by: Cursor <cursoragent@cursor.com>

Opus 4.8/4.7、Sonnet 5、Fable 5 上 thinking.display 默认为 "omitted"：模型照常思考、照常计费，但返回的 thinking block 文本为空，客户端无从展示，终端用户会误以为上游不支持思考。新增系统级三态开关 RectifierSettings.ThinkingDisplayMode： - display_only（默认）：只为已带 thinking.type=adaptive 的请求补 display=summarized。思考本来就在发生并计费，补齐不增加任何 token，也不影响 messages 层 prompt 缓存。 - force：额外为完全未携带 thinking 的请求注入思考。这会真正开启思考，故同时抬高 max_tokens——max_tokens 是思考+正文的总上限，在低预算请求上强开思考会让正文被截断且上游不报错。 - off：完全不干预。顺带修复：客户端发 thinking.type="enabled" / budget_tokens 给这些模型必定 400，而既有的 RectifyThinkingBudget 400 重试救不了它——它只调整 budget_tokens 的取值，而这些模型是整个拒绝该字段，重试照样 400。现归一化为 adaptive。注入点有两处：主转发链路与 Anthropic API-key 直通链路（后者在 gateway_forward.go 提前 return，不经过整流链）。只做一处会让同一用户因调度到的账号不同而时有时无，比一直没有更像故障。 GetThinkingDisplayMode 带 nil 接收者守卫：GetRectifierSettings 直接解引用 settingRepo，而本方法在网关热路径上调用。

github-actions Bot added a commit that referenced this pull request Jul 15, 2026

@DR-lin-eng has signed the CLA in #4337

9c5bade

DR-lin-eng force-pushed the main branch from f875c16 to 6aa1c62 Compare July 15, 2026 10:23

DR-lin-eng mentioned this pull request Jul 15, 2026

更新到156后，内存暴涨，重启不能解决问题 #4365

Open

DR-lin-eng force-pushed the main branch 5 times, most recently from 604fb67 to dfa806f Compare July 17, 2026 10:34

DR-lin-eng and others added 15 commits July 18, 2026 11:39

Support independent image rate multipliers for token billing

f566a32

feat(frontend): 展示使用记录 Token 输出速度

a8e2c0d

fix account last-used sorting

36177bf

Harden reviewed upstream PR integrations

cc6df42

Fix post-sync lint regressions

15f4018

Harden security audit request ownership and queue accounting

983c584

Fix settings tab sizing and label truncation

f8a2963

Add hourly usage stats to admin account list

f1b26ac

fix: 完善上游 Sub2API 计费倍率探测与账号展示

3d4e15b

fix: harden upstream billing sorting and bulk updates

d67f022

Optimize upstream billing rate sorting

f94c126

Fix upstream billing metadata validation lint

6783a9b

Optimize thinking display normalization hot path

7604470

DR-lin-eng and others added 30 commits August 1, 2026 22:21

chore: record upstream sync through b74024c

fdf8d96

refactor(frontend): normalize upstream frontend changes

f15e679

chore: record reviewed PR #3 frontend normalization

f6dd92c

docs: align reviewed frontend development guidance

0d49927

Remove sponsor promotions from README files

341f77d

feat: sync upstream billing and profit controls through c043c24

f3d30db

chore: record upstream sync through c043c24

19c5448

test: align integration fixtures with billing sync contracts

14f7a57

chore: record upstream sync through 7e2e9ba

3974cd8

chore: sync VERSION to 0.1.172 [skip ci]

a26ca38

Add automatic public model discovery to model plaza

7383d9a

feat: sync upstream changes through 27e8f69

9baca6b

chore: record upstream sync through 27e8f69

19af0a9

Route reporting statistics queries through the reporting database

acaa503

Center and shrink-wrap the schedulable account column

72f5e47

Refine backend and frontend workflows

f160537

chore: sync VERSION to 0.1.173 [skip ci]

313b162

feat: sync upstream PR 5183

38d28e3

chore: record upstream sync through 825ca7b

aaf38d8

refactor(frontend): enforce feature ownership boundaries

fa87975

Merge remote-tracking branch 'origin/main'

94c58cd

完善客服系统雏形

f54254b

客服系统新增红点提示

ddb73be

Merge pull request #4: add support chat

2a35693

feat: harden and integrate support chat

5718c2c

Add feature-flagged Model Plaza links to home page

f74d79b

refactor(frontend): finish admin accounts architecture pilot

5fee55e

fix(prompt-audit): parse responses output text

c812281

feat: integrate reviewed upstream changes and runtime hardening

612d3ff

chore: record reviewed upstream PR Wei-Shaw#5261

cadde25

Navigation Menu

perf: harden gateway, billing, and ops paths under high concurrency - #4337

perf: harden gateway, billing, and ops paths under high concurrency#4337
DR-lin-eng wants to merge 280 commits into
Wei-Shaw:mainfrom
DR-lin-eng:main

DR-lin-eng commented Jul 15, 2026

Uh oh!

github-actions Bot commented Jul 15, 2026 •

edited

Loading

Uh oh!

DR-lin-eng commented Jul 15, 2026

Uh oh!

DR-lin-eng commented Jul 15, 2026

Uh oh!

DR-lin-eng commented Jul 15, 2026

Uh oh!

DR-lin-eng commented Jul 15, 2026

Uh oh!

DR-lin-eng commented Jul 15, 2026

Uh oh!

DR-lin-eng commented Jul 15, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

17 participants

Conversation

DR-lin-eng commented Jul 15, 2026

Summary

Key changes

Testing

Uh oh!

github-actions Bot commented Jul 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

DR-lin-eng commented Jul 15, 2026

Uh oh!

DR-lin-eng commented Jul 15, 2026

Uh oh!

DR-lin-eng commented Jul 15, 2026

Uh oh!

DR-lin-eng commented Jul 15, 2026

Uh oh!

DR-lin-eng commented Jul 15, 2026

Uh oh!

DR-lin-eng commented Jul 15, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

17 participants

github-actions Bot commented Jul 15, 2026 •

edited

Loading