Repository navigation
v0.9.33: scheduled tasks, bun upgrade - #9339
Open
waleedlatif1 wants to merge 15 commits into
Open
waleedlatif1 wants to merge 15 commits into
waleedlatif1 wants to merge 15 commits into
Conversation
…#9331) * feat(mothership): run unattended turns durably on the leased chat path A scheduled job's occurrence must outlive the Sim process driving it. The headless lifecycle the wake door uses takes no controller lease, so a pod death leaves its run unrecoverable. An unattended turn now admits like a typed send (user message, run and recovery intent in one commit under the chat lock), runs detached from any HTTP response, and records a new unattended_stream recovery kind. Takeover restores it non-interactive with the secret authority it was admitted with, instead of the interactive default, and the abandoned-run sweep, stream liveness and billing-admission recording treat both durable kinds alike. Durable start now also forwards a caller's billing-admission hook, so a job can key its per-run budget on the run's billing request. The integration test kills the turn's controller three times mid-run; it fails on a headless implementation, on a sweep blind to the new kind, and on a takeover that restores the interactive default. * feat(db): add scheduled job occurrence columns and archive legacy jobs Scheduled jobs return as Sim-owned schedules whose occurrences run as durable chat turns. A job's run record (job_execution_logs) gains the occurrence it runs (unique per schedule), its outcome and summary, and the billing request its charges settle under; a schedule gains a per-run spend cap. Indexes are built concurrently. Every job row written for the retired Go runner is archived, never deleted, so none can start once the tick claims jobs again. The archive is a script migration committing per keyed batch, so it finishes on any row count and resumes after a crash. * feat(mothership): run scheduled jobs from the schedule tick Behind MOTHERSHIP_JOBS_ENABLED (off by default, and to stay off in production until unattended runs hold production effects for approval), the schedule tick gains a job lane: - Claim: each due job's occurrence is recorded (queued) and the schedule advances past it in one transaction, skipping locked rows, so two ticks record it once and a crash never leaves a stale claim. Missed occurrences collapse into one; cadence is never under 15 minutes; a one-shot job (no cron) runs once and completes; max_runs completes. - Dispatch: the occurrence starts as an unattended durable turn in the job's chat under its occurrence id (UUIDv5 of schedule and instant), so every retry finds the turn it admitted. A busy chat skips the occurrence; a payer over its plan limit skips it; an owner who lost access turns the job off. Only secrets a person named for the job mount. - Reconcile: a queued occurrence whose dispatch was lost is dispatched again after two minutes; a running one whose turn ended is finalized; one past six hours is stopped. - Finalize: exactly once under a row lock. A reported outcome is kept; a turn that ended well without one is not_met, summarized by its last reply. Three failed or interrupted runs in a row turn the job off and email its people; a met until_complete job completes. - Budget: each billing callback records the run's cumulative settled cost on its record (keyed by the run's billing request), and a run at its cap pauses at its next step with a job_run_budget card. The cost is cumulative per billing request, which recovered legs reuse, so the cap survives worker hand-offs and Sim takeovers. * improvement(mothership): run the job lane unflagged behind one indexed probe * improvement(mothership): simplify the job lane and stop jobs through the shared Stop delivery * chore(db): format the scheduled job migration metadata * improvement(mothership): keep the durable-run predicates out of the chat protocol graph * fix(mothership): settle job runs before claiming, keep spend-cap links and failure counts durable * fix(mothership): stop orphaned job runs at their limit and skip any number of deleted occurrences * fix(mothership): end a running job record whose run was removed
* docs(library): update best-ai-agents-support-ticket-triage * Pi Babysit: address PR #9337 feedback --------- Co-authored-by: Sim Pi Agent <[email protected]>
* docs(library): update dify-alternatives * Pi Babysit: address PR #9336 feedback --------- Co-authored-by: Sim Pi Agent <[email protected]>
Co-authored-by: Sim Pi Agent <[email protected]>
* fix(workflow-tests): keep the error of a mock that throws
A mock that throws reaches the server as { runId, callId, error }. The reply
schema tried the output branch first, and its optional `output` matched,
stripping `error`: the block failed with "A mocked block must return an
object" instead of the thrown message, so a workflow that branches on its
error text behaved differently under test than live. The error branch is now
tried first.
* fix(workflow-tests): never read a mock error past the cap as an empty output
An error longer than the 10,000-character cap failed the error branch and
fell through to the output branch, which still read it as an empty output.
The output branch is now strict, so such a reply is refused instead.
* test(workflow-tests): drop the case that restates the reply schema
…9334) * docs(library): update ai-agent-examples-by-department-and-industry * Pi Babysit: address PR #9334 feedback --------- Co-authored-by: Sim Pi Agent <[email protected]>
Bun 1.4.4 ships the updated root certificate store (oven-sh/bun#46159), so Bun's native fetch stops trusting the distrusted emSign ECC Root CA - G3. Moves the packageManager pin, engines floors, CI setup-bun versions, the oven/bun base images, and the Pi sandbox Bun pin together.
|
The latest updates on your projects. Learn more about Vercel for GitHub. |
github-actions
Bot
added
the
requires-mothership-merge
|
Contributor
 [Medium impact] The main reach is scheduled job execution and its cost tracking. Do not merge until success reporting and the dispatch race are fixed, and the import and props requirements are satisfied. Findings
SummaryAdds scheduled jobs as durable unattended chat turns.
Two behavior fixes are needed: wire up success reporting and prevent racing dispatches from closing logs for turns that still run. The new folder and extracted component also need to follow the repository's import and props rules.
|
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
All reported issues were addressed across 84 files
Heads up: you’re close to your flex budget. Increase your flex budget so reviews don’t pause.
Turn on auto-fix | Re-trigger cubic
OpenAI retired gpt-4.1-nano and the API now returns 404, so move it from legacy to deprecated: hidden from model pickers, still a known id so pinned workflows validate.
…covery (#9341) An MCP server whose hostname no longer resolves made tool discovery answer 500 and log at error. McpDnsResolutionError is now a server-unavailable condition (discover 422, v2 503), and categorizeError maps it to 502 and egress/domain refusals to 403, matching test-connection and server lifecycle.
6 tasks done
…ery poll (#9343) Gmail answers history.list with 404 once the stored cursor ages out of its retained history and the client must full-sync. The fallback searched the inbox, then took the mailbox profile's historyId as the new cursor, but a quiet mailbox reports the expired id itself, so it was persisted again and every poll repeated the 404, search and profile calls with an ERROR log. The resync now never adopts the rejected cursor and drops it when nothing newer exists, so the webhook polls through search until matching mail supplies one. The expected 404 logs at info and other history errors at warn.
…9344) * fix(logs): keep provenance sidecars intact through log PII redaction Log-stage PII masking walks every string leaf of the execution state, so a match inside an encrypted provenance entry rewrote the ciphertext and the display read failed closed with the final output withheld. Restore the finalOutput/workflowInput sidecars, loop/parallel scope provenance, and sourceExecutionId from the pre-redaction state alongside the fields already restored. An undecryptable stored envelope now logs one attributed error per read instead of three. * test(provenance): expect provenance imports to decrypt without the helper's own failure log
… input types (#9342) * fix(workflows): accept json input type and return 400 for unsupported input types A Start/API trigger input field typed "json" (the name blocks use for objects) made manual and copilot runs fail with a 500, and broke MCP schema sync on deploy, because the shared run schema threw a plain Error on any type outside its list. Treat json as object and files as file[] everywhere the type is read, raise OrchestrationError('validation') for reserved names and unsupported types so callers answer 400 naming the field and allowed types, and canonicalize or reject input-format row types at write time. * fix(workflows): sample aliased input types as their canonical type in mock payloads * fix(mcp): answer an unpublishable workflow input as a validation failure
* fix(mothership): close the scheduled-job review gaps - dispatch one attempt at a time per occurrence, so a skip or block never closes a live run's record - brief each occurrence with the run number fixed at its claim, not the job's moving count - report an automatic turn-off only when this failure turned the job off, not one a person already turned off - mark a deadline stop after it is delivered, so a crash before then delivers it again - state the run's spend cap only where hosted billing enforces it - recover an invalid index left by an interrupted build in the job migration, as 0435 does - give PlanUsageLimitDisplay a props interface * fix(mothership): record why a run stops before delivering the stop, and redeliver one an earlier pass did not finish
…loy-check transaction early, cache integration tool schemas (#9347) * fix(db): give billing and auth their own connection pools postgres.js pipelines new queries onto busy connections when none is idle, so slow reads saturating the process pool made short usage-ledger transactions wait seconds for BEGIN until Postgres killed them. - Add `billing` (4) and `auth` (3) dbFor roles, with DATABASE_URL_BILLING and DATABASE_URL_AUTH overrides falling back to the process URL. - Billing pool: recordCumulativeUsage, recordUsage without a caller tx, the execution logger's usage reconciliation, both threshold settlement transactions, and recordJobRunSpend. Each moved transaction only queries through its tx handle; readers stay on the process pool. - Auth pool: getAuthPool() backs the Better Auth adapter, getAuthDatabase() outside a transaction, the OAuth adapter guard defaults, and the API-key hash lookup. - isPrimaryPoolClient() in @sim/db replaces the `executor === db` checks in the security, session, and SSO policy caches, so reads through the auth pool keep hitting them instead of costing two queries per cached session. - Integration test saturates the process pool and requires the billing write, an adapter session read, and an API-key lookup to finish anyway. * fix(workflows): compare deployment states after the snapshot transaction commits checkNeedsRedeployment ran the full workflow diff inside its REPEATABLE READ transaction, holding a pooled connection idle in transaction during CPU work. Reads (active version, normalized draft, variables, credential migrations) stay in the snapshot; the pure diff now runs after commit. * fix(mothership): build integration tool schemas once per process instead of per user every 5s The integration catalog cached each viewer's full schema set for 5s, so nearly every call rebuilt a user-tool schema for all ~5.3k exposed tools, sized the ~7.7MB result with JSON.stringify, structuredClone'd it, and re-tokenized every tool for BM25 - all synchronously. The schemas depend only on the registry and deployment config, so they are now built once per (surface, personal mode), frozen, and shared; the build yields between tools. Each request still applies the viewer's own block visibility and permission policy, and the plan-dependent email tagline is read per user with the same 5s TTL. Results are byte-identical. BM25 term counts are memoized per shared schema object. * refactor(db): tighten pool-isolation follow-ups Assert each frozen-schema mutation throws, rank BM25 by query terms instead of copying each tool's vocabulary, gate the reporting usage cache on isPrimaryPoolClient like the auth caches, and trim comments. * fix(api-key): read workspace key settings on the auth pool Give the memory harness integration mock a dbFor so the billing pool resolves. * test(db): close every pool the isolation suite opens
…t use (#9349) * fix(logs): stop loading execution data and snapshots that reads do not use - Status polls select execution_data through a CASE only for the statuses the poll reads it for (failed; completed when output is requested; always for selected outputs), from one shared status rule. - The execution snapshot view no longer loads execution_data while resolving its context; it is read only after authorization, so a refused caller never fetches it. - The paused execution list projects execution_snapshot->'triggerIds' instead of loading whole rows with the serialized executor snapshot. - One shared terminal-status predicate replaces the copies in the HITL manager and run-file reads. * test(logs): give the recording client a connection option so its options type-check * fix(logs): read a snapshot view's execution data with the timing and cost it pairs with
This branch was previously deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.