Changelog

What's Changed

Minor features

• Add UI-managed workflow webhooks with durable delivery by @david-zlai in #1351

Bug fixes

• Crucible: resolve Ray, Flink and eval secrets in the job; fix Ray log scope by @diya-zipline in #1384
• fix(eval): better handling of pure parquet by @cristian-zlai in #1392
• Improve metric chart rate interval by @sean-zlai in #1394
• Keep secret placeholders for the job when the hub env value is empty by @kumar-zlai in #1396

Full Changelog: v1.27.1...v1.27.2

What's Changed

Minor features

• [Crucible] Apply mode-scoped quotas to compute pods by @tchow-zlai in #1341
• [Crucible] Select shared compute through stable role labels by @tchow-zlai in #1378

Bug fixes

• Split Ray head and worker logs on the container label Hub already uses by @diya-zipline in #1375
• Always resolve job secrets in-job; drop deferSecretsToJob by @kumar-zlai in #1377
• Fill non-secret conf placeholders from the hub env in the batch submit path by @kumar-zlai in #1380
• Add Crucible compute-plane monitor tab by @piyush-zlai in #1382
• Normalize workflow ranges to the terminal partition grid by @sean-zlai in #1362
• Pin a scheduled run's upstream nodes to the conf version it was scheduled with by @diya-zipline in #1374
• Run Crucible training once and release Ray pods when the job ends by @diya-zipline in #1383
• Preserve kv.tablePrefix across backend selection by @cristian-zlai in #1391

Full Changelog: v1.27.0...v1.27.1

What's Changed

Minor features

• Trials: schema-backed experiment tracking, the API, and the UI by @diya-zipline in #1356
• Say why a failed experiment failed, and on which engine by @diya-zipline in #1359
• Add bulk workflow cancellation end to end by @sean-zlai in #1361
• Split Ray logs into head and worker views by @diya-zipline in #1370
• Give a trial's confs and features the identity a name alone lacks by @diya-zipline in #1360

Bug fixes

• ci: scan eval-aws, eval-azure and Spark 4 AWS canary images with grype by @piyush-zlai in #1354
• Add workflow duration display by @sean-zlai in #1355
• Add cancel workflow action to workflow detail view by @sean-zlai in #1358
• Update better-auth to 1.7.7 by @sean-zlai in #1363
• Add workflow cancellation inside nested workflow views (drilldown drawer) by @sean-zlai in #1364
• Disable orchestrator sensor retries for new workflows by @tchow-zlai in #1366
• feat(orchestration): forward Iceberg catalog config to Flink streaming jobs by @piyush-zlai in #1369
• Say "that namespace does not exist" instead of "forbidden" by @diya-zipline in #1371
• fix: grype scan failures — spark 3.5.8, frontend npm overrides, spark4 jackson 2.21.7 by @piyush-zlai in #1373
• fix(eval): concurrent isolation by @cristian-zlai in #1372

Full Changelog: v1.26.0...v1.27.0

What's Changed

Minor features

• Route Crucible workloads to team namespaces by @tchow-zlai in #1335
• Defer secret resolution to the job on the EMR Serverless route by @kumar-zlai in #1346
• Add trials and experiments schema with its DAO by @diya-zipline in #1322

Bug fixes

• Strip shaded JLine builtins from eval and Azure hub assemblies by @piyush-zlai in #1340
• Make the demo data something a model can actually learn from by @diya-zipline in #1339
• Surface Ray logs and dashboard links in Hub by @diya-zipline in #1317
• Make the local Hub stack usable out of the box by @diya-zipline in #1331

Full Changelog: v1.25.0...v1.26.0

What's Changed

Minor features

• Add conf archiving and localStorage filter persistence by @sean-zlai in #1330

Bug fixes

• ci: set ZIPLINE_ALLOW_DIRTY=1 for integration test jobs by @piyush-zlai in #1334
• Accept single query values for list params in Hub request parsing by @piyush-zlai in #1338
• Raise Hub request body limit and accept gzip request bodies by @piyush-zlai in #1337

Full Changelog: v1.24.1...v1.25.0

What's Changed

Minor features

• Give each model version its own output directory by @diya-zipline in #1328

Bug fixes

• Route Crucible pods to the submitting team's node pool by @kumar-zlai in #1325

Full Changelog: v1.24.0...v1.24.1

What's Changed

Bug fixes

• Tag job submission/completion metrics with workflow branch by @piyush-zlai in #1321
• fix(eval): s3a checks + contextual sources by @cristian-zlai in #1323

Full Changelog: v1.23.4...v1.24.0

What's Changed

Bug fixes

• ui(stats): set a category for complex features by @cristian-zlai in #1319
• Streamline timeline partition and workflow payloads by @sean-zlai in #1309

Full Changelog: v1.23.3...v1.23.4

What's Changed

Minor features

• UI: Show online joins from fetcher on monitor overview with detail drawers showing per-group batch data status by @sean-zlai in #1303
• Route TRAIN_MODEL directly through Crucible (Slice B) by @diya-zipline in #1297
• Harden direct Ray training submissions and trainer archives by @diya-zipline in #1307

Bug fixes

• Prevent feature canary runs from promoting stable by @tchow-zlai in #1300
• improv(crucible): different jar manifest spark3/4 by @cristian-zlai in #1311
• improv(eval): address staging query failure on quickstart by @cristian-zlai in #1306
• feat(aws): fetcher with logging by @cristian-zlai in #1316

Full Changelog: v1.23.2...v1.23.3

What's Changed

Bug fixes

• [Spark 4] Add Platform source compatibility CI by @tchow-zlai in #1295
• [Spark 4] Publish and select AWS Hub and Eval images by @tchow-zlai in #1296
• Expose batch-table GC days override to AWSCleanupVerticle by @piyush-zlai in #1299

Full Changelog: v1.23.1...v1.23.2

What's Changed

Bug fixes

• Redact sensitive UI environment variables by @sean-zlai in #1291
• fix(orchestration): ask a producer about its own grid everywhere by @kumar-zlai in #1280
• Convert Chronon training requests to Crucible Ray requests by @diya-zipline in #1283
• [Spark 4] Make Platform eval source-compatible with Spark 3 and 4 by @tchow-zlai in #1293
• fix(eval): proj and non-proj allowed in wheres by @cristian-zlai in #1294

New Contributors

• @diya-zipline made their first contribution in #1283

Full Changelog: v1.23.0...v1.23.1

What's Changed

Minor features

• feat(eval): bigquery setups allowed + tests for aliases by @cristian-zlai in #1274
• CleanupVerticle + AWS DynamoDB batch-table sweep by @piyush-zlai in #1287
• UI: Monitor chart filters by @sean-zlai in #1289
• UI: Add date range filtering to home and workflow views by @sean-zlai in #1290
• Site: Add /docs/changelog page for GitHub releases by @sean-zlai in #1284

Bug fixes

• Upgrade frontend deps (notable Better Auth to 1.7) by @sean-zlai in #1288

Full Changelog: v1.22.0...v1.23.0

What's Changed

Minor features

• UI: Add schedule view filtering and sorting by @sean-zlai in #1265
• Add schedule state filtering to conf list by @sean-zlai in #1277

Bug fixes

• Pin httpcore5 to fix Grype scans by @tchow-zlai in #1268
• Check cancel streaming eligibility server-side by @sean-zlai in #1270
• bug(eval): revise fields available for derivations by @cristian-zlai in #1273
• Patch Platform images for Grype HTTP advisories by @nikhil-zlai in #1271
• ci: increase Mill heap for eval assembly by @tchow-zlai in #1275
• ci: isolate Mill assembly builds by @tchow-zlai in #1276
• ci: exclude NVMe stress from standard AWS Canary by @tchow-zlai in #1278
• ci: use large runner for jar builds by @tchow-zlai in #1279

Full Changelog: v1.21.1...v1.22.0

What's Changed

Bug fixes

• UI: UTC dates by @sean-zlai in #1263

Full Changelog: v.1.21.0...v1.21.2

What's Changed

Minor features

• UI: Log level improvements by @sean-zlai in #1252

Bug fixes

• Determine streaming node skipped state by latest step CTRL-301 by @kumar-zlai in #1251
• Add StepTracker regression test for async cluster creation failure by @david-zlai in #1245
• fix(orchestration): don't finish daily deploy workflows while batch uploads are unfilled by @nikhil-zlai in #1255
• Run NVMe stress in nightly AWS e2e CTRL-201 by @tchow-zlai in #1197
• Fix frontend Grype vulnerabilities by @nikhil-zlai in #1254
• fix(orchestration): gate batch children of a running stream on the stream's own upstream deps by @kumar-zlai in #1257
• fix(orch): don't demote Running adhoc streaming step to Submitted on transient PENDING by @david-zlai in #1253
• Resolve latest branch node hash on streaming crash-recovery retriggers by @kumar-zlai in #1261
• fix(orchestration): express node plan ranges on the producer's own grid by @nikhil-zlai in #1258
• UI: UTC dates by @sean-zlai in #1263

Full Changelog: v1.20.1...v1.21.1

What's Changed

Bug fixes

• Surface streaming semanticHash conflict on the failing workflow CTRL-278 by @kumar-zlai in #1248
• ci: serialize nightly AWS e2e runs to prevent shared-state collisions by @kumar-zlai in #1247
• Auth: Fix zipline CLI auth behind JWT-only API gateways by @sean-zlai in #1153

Full Changelog: v1.20.0...v1.20.1

What's Changed

Minor features

• UI: Dynamic workflow/conf status by @sean-zlai in #1233

Bug fixes

• Fix Flink cold-start ValidationError: propagate savepoints dir in CrucibleSubmitter by @piyush-zlai in #1235
• Progress the steps that satisfy constraints when one step in a batch violates one by @nikhil-zlai in #1237
• Dedupe retriggered streaming steps by node in StreamingStepExecutor CTRL-272 by @kumar-zlai in #1224
• fix(orch): reconcile StreamingRetrying steps when deploy schedule is removed by @david-zlai in #1238
• feat(aws): add Redis/ElastiCache KV store support for hub and orchestration by @piyush-zlai in #1236
• Terminate workflow on conflicting active streaming step CTRL-278 by @kumar-zlai in #1239
• UI: Fix status refresh, improve node details view by @sean-zlai in #1241

Full Changelog: v1.19.1...v1.20.0

What's Changed

Bug fixes

• ci: nightly tracks latest release + Slack on failure only by @kumar-zlai in #1219
• Crucible Flink ports for label support + jmDeploymentStatus = error transitions by @piyush-zlai in #1214
• ci: gate nightly on zipline doctor infra-health + Slack warnings by @kumar-zlai in #1220
• Fix nightly Grype failures: eval jackson drift + jetty-security ignore by @piyush-zlai in #1227
• Revert "Annotate registered steps with call site for failure tracing" by @david-zlai in #1232

Full Changelog: v1.19.0...v1.19.1

What's Changed

Minor features

• Show node metadata in details / compare view (along with contents) by @sean-zlai in #1216

Bug fixes

• Fix admin upgrade data-plane not picking up the latest engine version by @piyush-zlai in #1211
• UI: Timeline overview status order by @sean-zlai in #1212

Full Changelog: v1.18.1...v1.19.0

What's Changed

Bug fixes

• Annotate registered steps with call site for failure tracing by @david-zlai in #1205
• ci: nightly AWS e2e (apply → admin install → full UC suite → destroy) by @kumar-zlai in #1202
• Treat cross-workflow submitted streaming steps as active by @tchow-zlai in #1203
• UI: Streamline status data payloads by @sean-zlai in #1207
• Add kill before retrigger for streaming jobs + port job restart fixes to Crucible Flink submitter by @piyush-zlai in #1208

Full Changelog: v1.18.0...v1.18.1