Skip to content

Commit e3197cd

Browse files
Merge remote-tracking branch 'origin/staging' into feat/tests
# Conflicts: # apps/sim/app/workspace/[workspaceId]/layout.tsx # apps/sim/lib/workflows/executor/execution-core.ts # packages/db/migrations/meta/0400_snapshot.json # packages/db/migrations/meta/_journal.json
2 parents cd14f90 + 33dfef2 commit e3197cd

548 files changed

Lines changed: 84657 additions & 8940 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.agents/skills/add-integration/SKILL.md‎

Lines changed: 107 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -26,11 +26,79 @@ Before writing any code:
2626
1. Use Context7 to find official documentation: `mcp__context7__resolve-library-id`, then fetch with `mcp__context7__query-docs`
2727
2. Or use WebFetch to read API docs directly
2828
3. Identify:
29-
- Authentication method (OAuth, API Key, both)
29+
- Supported authentication grants, who owns the app, and which permissions each operation needs
3030
- Available operations (CRUD, search, etc.)
3131
- Required vs optional parameters
3232
- Response structures
3333

34+
### Choose the connection flow before building tools
35+
36+
Read the provider's current authentication documentation first. “OAuth” describes a protocol;
37+
it does not imply a browser redirect or a deployment-wide client ID and secret. Compare the
38+
supported paths and choose the simplest supported setup for the intended user:
39+
40+
| Provider method | Sim connection pattern | Verify in the provider docs |
41+
| --- | --- | --- |
42+
| Authorization code | Shared OAuth app and consent flow | Partner approval, redirect URIs, tenant consent, scopes, refresh-token rotation |
43+
| Customer-owned client credentials | Saved `service_account` credential with a client ID and secret | Internal-app eligibility, grant activation, scope syntax, token lifetime and revocation behavior |
44+
| API token or personal access token | Saved token service account when reusable connections are useful | Token permissions, identity verification, expiry and rotation |
45+
| Private key, certificate, or service-account JSON | Existing key-based service-account framework | Signing algorithm, audience, subject, tenant binding and key rotation |
46+
47+
Do not present customer-owned credentials as a way around a provider's approval requirements
48+
for a shared public integration. Explain the supported account/app type in the setup docs. Keep
49+
existing OAuth connections usable when other users depend on them, unless a migration or removal is explicitly authorized.
50+
51+
For client credentials, extend the existing descriptors and minter registry under
52+
`apps/sim/lib/credentials/client-credential-accounts/`; for token credentials, use
53+
`token-service-accounts/`. Reuse the connect modal, encrypted storage, authorized credential
54+
operations, and execution-time token resolver. Do not collect reusable secrets on every block,
55+
invent a credential route, or store a short-lived access token as if it were permanent.
56+
57+
Verify the complete lifecycle, including connect-time verification, concurrent executions on
58+
different workers, expiry, secret rotation, and permission changes. Some providers invalidate the
59+
previous token whenever another is minted. Those providers require shared coordination keyed by
60+
the provider application identity, including across duplicate saved credentials; a process-local
61+
or per-scope token cache is insufficient. A cache hit must never authenticate a wrong secret or
62+
silently grant broader permissions. Bound token responses and retries, prevent credential-bearing
63+
redirects, and never include provider response bodies or secrets in errors.
64+
65+
Use explicit permission choices when the supported operations have different access needs. Keep
66+
region and permission selections on reconnect unless the user changes them. Verify documented
67+
identity endpoints rather than guessing which person a client-credentials token represents.
68+
69+
### Pair saved credentials with block and resource selectors
70+
71+
`oauth-input` is the shared saved-credential picker, including for service accounts that never
72+
redirect to OAuth. Give its basic/advanced pair one `canonicalParamId: 'oauthCredential'`, with
73+
the correct `serviceId`, and wire tool OAuth metadata to that same service. Tokens and trusted
74+
API origins are hidden execution inputs; the model receives a credential ID, never a secret.
75+
Use `credentialKind: 'service-account'` when only service accounts are supported, including in
76+
tool OAuth metadata. Use `credentialKind: 'any'` on the picker when both browser OAuth and saved
77+
service accounts are supported; omitting it defaults the connect action to browser OAuth. Per-connection scope
78+
choices belong in the descriptor and encrypted credential, not an all-permissions block
79+
`requiredScopes` array that would reject read-only connections.
80+
81+
When a documented list endpoint makes a resource ID discoverable, pair the saved credential
82+
with a dynamic resource selector using the `add-selector` and `validate-selector` skills:
83+
84+
- Declare the credential subblock and any parent resource in `dependsOn`, and give the resource
85+
selector and its manual advanced input the same canonical parameter.
86+
- Register browser-safe selector metadata and a server attachment through the shared selector
87+
framework. Resolve the credential with the expected provider binding and use a fixed or
88+
credential-bound destination; selectors and execution must use the same auth/region policy.
89+
- Exercise switching credentials, parent resources, pagination, expired tokens, denied access,
90+
and the advanced environment-reference path. Check that stale choices cannot survive a change
91+
of account. Do not fetch provider data or mint tokens in the browser.
92+
- Keep a manual ID path when the provider cannot enumerate a resource. Do not invent an endpoint
93+
solely to provide a dropdown.
94+
95+
For an existing integration, inspect persisted workflow serialization as well as the visible
96+
form before removing old auth fields. Establish whether existing users need a migration or
97+
compatibility path; do not add permanent legacy branches speculatively when removal is authorized.
98+
When compatibility is needed, prove that old values survive serialization and execution; hiding
99+
a field is not proof. If the user authorizes a production usage check, query only the aggregate usage/auth-shape evidence
100+
needed and keep identities, secrets, and local evidence out of commits and PRs.
101+
34102
### Hard Rule: No Guessed Response Schemas
35103

36104
If the official docs do not clearly show the response JSON shape for an endpoint, you MUST stop and tell the user exactly which outputs are unknown.
@@ -274,18 +342,28 @@ export const TRIGGER_REGISTRY: TriggerRegistry = {
274342

275343
## Step 7: Configure Deployment Availability
276344

277-
Do this for every visible OAuth integration. API-key and unauthenticated integrations do not need
278-
an OAuth client capability.
345+
Do this for every integration that uses the shared credential picker. Only a connection that
346+
depends on deployment-wide OAuth client fields needs an OAuth client capability; a customer-owned
347+
service account must remain available without those fields.
279348

280349
The block's `oauth-input.serviceId` is the canonical link between the generated integration catalog,
281350
the OAuth service configuration, deployment availability, and the setup CLI.
282351

283-
1. Ensure the block has exactly one distinct OAuth `serviceId` and that it matches the canonical
284-
service entry in `apps/sim/lib/oauth/oauth.ts`.
285-
2. Confirm `resolveOAuthClientCapabilityId(serviceId)` resolves to the intended provider entry in
352+
1. Set `authMode: AuthMode.OAuth` for integrations using browser OAuth or customer-owned OAuth
353+
client credentials. Token-only service accounts can retain `AuthMode.ApiKey` with the shared
354+
picker, as Coda does; `oauth-input` alone does not determine the authentication protocol.
355+
Register a new token-only service ID and block type in `tokenCredentialIntegrationTypes` in
356+
`packages/deployment-config/src/integration-availability.ts` so availability and integration
357+
policy recognize the saved credential path.
358+
For OAuth integrations, this value lets the catalog discover the connection flow instead of
359+
routing "Add to Sim" to chat. Ensure the block has exactly one distinct OAuth `serviceId`
360+
matching the canonical service in `apps/sim/lib/oauth/oauth.ts`. The canonical service's
361+
`authType` selects browser OAuth or the service-account modal. Verify the resulting catalog
362+
and block connection actions for the chosen authentication method.
363+
2. For browser OAuth, confirm `resolveOAuthClientCapabilityId(serviceId)` resolves to the intended provider entry in
286364
`OAUTH_CLIENT_CAPABILITIES` in `packages/deployment-config/src/env-capabilities.ts`. Google and
287365
Microsoft service IDs deliberately share provider-level capabilities.
288-
3. For a new OAuth provider, add the required client fields to `OAUTH_CLIENT_CAPABILITIES`, add
366+
3. For a new browser OAuth provider, add the required client fields to `OAUTH_CLIENT_CAPABILITIES`, add
289367
every referenced field to the env schema in `apps/sim/lib/core/config/env.ts`, and add the
290368
matching `text` or `secret` entries to `OAUTH_CLIENT_SETUP_FIELDS` in
291369
`packages/sim-setup/src/capability-config.ts`. Do not create integration-specific setup logic or
@@ -298,9 +376,13 @@ the OAuth service configuration, deployment availability, and the setup CLI.
298376
- no `deploymentRequirement` when the service-account path works independently of OAuth client fields;
299377
- `'oauth-client'` when it requires the same deployment OAuth client fields;
300378
- `'preview-gated'` when availability is controlled by the service-account preview block.
379+
For a service-account-only default, set the canonical service's `authType: 'service_account'`
380+
and `serviceAccountProviderId`. Verify both block availability and the connect modal with no
381+
deployment OAuth credentials configured. An existing browser OAuth path may remain for legacy
382+
credentials without becoming a prerequisite for the new path.
301383

302-
Never add a permissive fallback for missing capability metadata. A visible OAuth integration without
303-
a resolvable capability must fail validation.
384+
Never add a permissive fallback for missing capability metadata. A browser OAuth connection without
385+
a resolvable capability must fail validation; an independent service account uses its own metadata.
304386

305387
## Step 8: Generate and Validate the Catalog
306388

@@ -389,7 +471,8 @@ If creating V2 versions (API-aligned outputs):
389471
- [ ] Set `integrationType` to the correct `IntegrationType` enum value
390472
- [ ] `{Service}BlockMeta.tags` lists every applicable `IntegrationTag` (tags live on the meta, not the block)
391473
- [ ] Defined operation dropdown with all operations
392-
- [ ] Added credential field with `requiredScopes: getScopesForService('{service}')`
474+
- [ ] Added the saved-credential picker with the supported `credentialKind`; browser OAuth scopes
475+
use `getScopesForService('{service}')`, while variable service-account permissions stay on the credential
393476
- [ ] Added conditional fields per operation
394477
- [ ] Every `short-input`, `long-input`, `code`, and selector subBlock has a `placeholder`
395478
- [ ] Set up dependsOn for cascading selectors
@@ -407,15 +490,25 @@ If creating V2 versions (API-aligned outputs):
407490
- [ ] `canvasPresentation.sentences` covers every operation; `bun run apps/sim/scripts/check-canvas-sentences.ts --block={service}` passes
408491
- [ ] `{Service}BlockMeta` also sets `url` (verified external homepage) and `skills` (grounded in `tools.access`, sourced from real use cases) — see add-block → BlockMeta
409492

410-
### OAuth Scopes (if OAuth service)
493+
### Authentication and saved credentials
494+
- [ ] Compared documented authorization code, client credentials, token, and key-based methods
495+
- [ ] Chosen app ownership and approval requirements match the intended user
496+
- [ ] Saved credential picker, tools, and resource selectors share the same provider/region binding
497+
- [ ] Connect verification, expiry, concurrent workers, secret rotation, and scope changes are sound
498+
- [ ] Existing usage and the migration/removal decision are established; any required compatibility is verified through serialization and execution
499+
- [ ] New connection and reconnect flows verified in the running UI
500+
501+
### Browser OAuth Scopes (if authorization-code flow is supported)
411502
- [ ] Defined scopes in `lib/oauth/oauth.ts` under `OAUTH_PROVIDERS`
412503
- [ ] Added scope descriptions in `SCOPE_DESCRIPTIONS` within `lib/oauth/utils.ts`
413504
- [ ] Used `getCanonicalScopesForProvider()` in `lib/auth/connectors/providers.ts` (never hardcode)
414-
- [ ] Used `getScopesForService()` in block `requiredScopes` (never hardcode)
505+
- [ ] Used `getScopesForService()` for the browser OAuth permissions the block needs (never hardcode)
506+
- [ ] A picker that also accepts service accounts does not require broader scopes than every supported
507+
connection needs; per-connection service-account permissions are validated by the descriptor/minter
415508

416-
### Deployment Availability (if OAuth service)
509+
### Deployment Availability (if using the saved-credential picker)
417510
- [ ] Block declares exactly one distinct `oauth-input.serviceId`
418-
- [ ] `resolveOAuthClientCapabilityId(serviceId)` resolves to the intended `OAUTH_CLIENT_CAPABILITIES` entry
511+
- [ ] Browser OAuth resolves to the intended `OAUTH_CLIENT_CAPABILITIES` entry; independent service accounts work without deployment OAuth fields
419512
- [ ] Every new OAuth capability field exists in `apps/sim/lib/core/config/env.ts`
420513
- [ ] Runtime OAuth fields live in `OAUTH_CLIENT_CAPABILITIES`; matching CLI input modes live in the exhaustively checked `OAUTH_CLIENT_SETUP_FIELDS`
421514
- [ ] If `serviceAccountProviderId` is configured, `SERVICE_ACCOUNT_METADATA_BY_OAUTH_SERVICE_ID` has the matching projection and deployment requirement

‎.agents/skills/ship/SKILL.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -77,6 +77,9 @@ When the user runs `/ship`:
7777
}
7878
# Runs every audit CI runs, concurrently, and replays the output of any that fail.
7979
# The audit list is derived in scripts/run-audits.ts — do not hand-list audits here.
80+
# Install CI's pinned actionlint version for the host OS/architecture and verify its
81+
# artifact against the official release checksums in a local mktemp directory.
82+
# Preserve CI's -shellcheck= -pyflakes= flags; lint all workflows and abort ship if it fails.
8083
bun run check:audits || { echo "❌ audit(s) failed — do not ship"; exit 1; }
8184
bun run type-check || { echo "❌ type-check failed — do not ship"; exit 1; }
8285
# CI's "Verify docs manifest is in sync" step is not a `check:*` script, so the runner above

‎.github/actions/cache/action.yml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ runs:
2020
steps:
2121
- name: Mount sticky disk
2222
if: inputs.provider == '' || inputs.provider == 'blacksmith'
23-
uses: useblacksmith/stickydisk@4c034ba57b706cf0e3b4b0ce098c2a3b1071580c # v1
23+
uses: useblacksmith/stickydisk@3f6be1451e3cad893b778bc93495c80452a5da26 # v1
2424
with:
2525
key: ${{ inputs.key }}
2626
path: ${{ inputs.path }}
@@ -29,7 +29,7 @@ runs:
2929
# which would freeze the cache at the first run's contents.
3030
- name: Restore and save cache
3131
if: inputs.provider != '' && inputs.provider != 'blacksmith'
32-
uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5
32+
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v5
3333
with:
3434
key: ${{ inputs.key }}-${{ github.run_id }}
3535
restore-keys: ${{ inputs.key }}-

‎.github/actions/image/action.yml‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -63,13 +63,13 @@ runs:
6363
# under 300 MB, sat at 249 GB. Sticky disks bill at ~$0.51/GB-month.
6464
- name: Set up Blacksmith builder
6565
if: inputs.provider == '' || inputs.provider == 'blacksmith'
66-
uses: useblacksmith/setup-docker-builder@a5256a73e30f09e37e3eceb8ca36043d17621d24 # v2
66+
uses: useblacksmith/setup-docker-builder@19215110ab936351210feebdfa5b440b4493e184 # v2
6767
with:
6868
cache-key: ${{ steps.cache-key.outputs.value }}
6969

7070
- name: Build and push (Blacksmith)
7171
if: inputs.provider == '' || inputs.provider == 'blacksmith'
72-
uses: useblacksmith/build-push-action@fb9e3e6a9299c78462bfadd0d93352c316adc9b8 # v2
72+
uses: useblacksmith/build-push-action@9b0579bbec7a6cad2f171596c57e7ac1e7658850 # v2
7373
with:
7474
context: ${{ inputs.context }}
7575
file: ${{ inputs.file }}
@@ -170,12 +170,12 @@ runs:
170170
171171
- name: Set up Docker Buildx
172172
if: inputs.provider != '' && inputs.provider != 'blacksmith'
173-
uses: docker/setup-buildx-action@bb05f3f5519dd87d3ba754cc423b652a5edd6d2c # v4
173+
uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4
174174

175175
# No cache-to: type=gha — it shares the 10 GB repo quota with the cache mounts.
176176
- name: Build and push (GitHub)
177177
if: inputs.provider != '' && inputs.provider != 'blacksmith'
178-
uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7
178+
uses: docker/build-push-action@c3c9e263c25d99ce0380d002d59b67737d91b0dc # v7
179179
with:
180180
context: ${{ inputs.context }}
181181
file: ${{ inputs.file }}

‎.github/actions/setup/action.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -39,7 +39,7 @@ runs:
3939

4040
- name: Setup Node
4141
if: inputs.node-version != ''
42-
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6
42+
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v6
4343
with:
4444
node-version: ${{ inputs.node-version }}
4545
registry-url: ${{ inputs.registry-url }}

‎.github/scripts/http-e2e.sh‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -122,13 +122,15 @@ case "$group" in
122122
# Self-hosted: hosted billing admits a run only through a Redis usage reservation.
123123
stop-after)
124124
export NEXT_PUBLIC_FORCE_HOSTED=false
125+
export CRON_SECRET=stop-after-e2e-local-cron-secret
125126
export INTERNAL_API_SECRET=stop-after-http-ci-local-secret-at-least-32-characters
126127
export DB_TX_TRIPWIRE=throw
127128
export NEXT_PUBLIC_CHAT_DISABLED=true
128129
start_app stop-after 3018 workflow record-http-status
129130
STOP_AFTER_E2E_BASE_URL="$NEXT_PUBLIC_APP_URL" \
130131
STOP_AFTER_E2E_DATABASE_URL="$DATABASE_URL" \
131132
STOP_AFTER_E2E_REPORT_PATH="$report_dir/stop-after-http-report.json" \
133+
STOP_AFTER_E2E_CRON_SECRET="$CRON_SECRET" \
132134
bun run test:workflow-stop-after:e2e
133135
;;
134136

@@ -137,7 +139,6 @@ case "$group" in
137139
desktop-inbox)
138140
export REDIS_URL=redis://127.0.0.1:6379
139141
export NEXT_PUBLIC_FORCE_HOSTED=false
140-
export MSHIP_DESKTOP_BACKGROUND_EXECUTOR=true
141142
export COPILOT_TOOL_PERMISSIONS_ENABLED=true
142143
export INTERNAL_API_SECRET=desktop-inbox-http-ci-local-secret-at-least-32-characters
143144
export DB_TX_TRIPWIRE=throw

0 commit comments

Comments
 (0)