ProjectX Data

Alerts in this tenant, with their current state.

Create an alert on a query result, or redefine one.

Two kinds of rule. A **threshold** judges every returned row and breaches if any of them does, which is the shape most useful alerts have — “which accounts are over their limit”. An **anomaly** compares one number against the median of its own recent history, using the same detector a pipeline monitor uses, and so needs a query that returns a single row. `for_minutes` is the dwell: the condition must hold continuously for that long before anything is said, and until then the alert is `pending`, which is a state you can see. `renotify_minutes` rations the repeat while it stays firing. Both exist because an alert that notifies per evaluation sends forty messages for one event and the fortieth is read by nobody. Redefining the query or the rule resets the state. An alert whose condition changed is a different alert, and carrying a `firing` across the edit would produce a resolution notice for an incident that never happened.

Body

SaveAlertRequest

Delete an alert.

One alert: its rule, its state and when it entered it.

Evaluate this alert now and fold the reading into its state.

Returns the alert, the reading and the transition, if any. An empty transition means nothing was said, which is the common and correct outcome. A failed evaluation does not read as healthy. After the tolerated number of consecutive failures the alert moves to `unknown` and that notifies, because an alert whose query has been broken for a week is worse than no alert — somebody is relying on it. Its condition state is not cleared while it is blind: an alert that was firing is still, as far as anybody knows, firing. Notifications are audit events, like a pipeline's. They carry the number, the count of breaching rows and the query id, and never the rows themselves — whoever is entitled to the detail can open that query under their own permissions, which keeps the decision about what a chat channel may hold out of this pillar.

Set how often this is evaluated, and as whom.

Evaluations run as whoever enabled the schedule. An alert is a standing query that executes when nobody is present, so it carries the risk a scheduled notebook does: the shortest implementation gives the scheduler a service identity and quietly grants every alert author the ability to read anything. There is no fallback, and an alert whose owner loses access stops working loudly rather than keeping on.

Body

Schedule

Hold or release a company's compute for an empty credit balance.

Pay as you go: when a company's credit balance stays empty past the grace period, the platform's balance sweep holds its compute — every running SQL warehouse, cluster and serverless pool is stopped and marked held, keeping its definition and data — and releases it when credits arrive, starting exactly the products it stopped. A held product cannot be started by hand until then. Platform tenant only.

Body

BillingHold

List this tenant's buckets.

Create a bucket, with public access blocked and versioning on.

Neither of those is a parameter. There is no field on this request and no call in the object-store interface behind it that can make a bucket readable by the internet, which is what makes “we do not create public buckets” a property of the code rather than a convention that drifts. Versioning defaults on because it is the difference between an accidental overwrite being an inconvenience and being data loss. It can be turned off; nobody has to remember to turn it on. `region` is fixed at creation. No object store can move a bucket — the operation is a copy and a switch of every reference — so a later attempt to change it is refused rather than silently ignored.

Body

CreateBucketRequest

Remove an empty bucket.

409 naming what is still in it. Emptiness is judged by the tables and file sets the platform put there rather than by a listing: a bucket a table's location points at is not empty even before the first write, and deleting it would break that table in a way nobody would connect to this call.

Describe a bucket: region, versioning, encryption, lifecycle.

The four answers a residency review, a recovery test and a cost question need, in one place.

Change versioning, encryption, lifecycle rules or replication.

Lifecycle rules transition objects to a colder class and may not delete them. A rule that expires objects is refused with a pointer to retention policies, which is where deletion lives — putting a second, unaudited deletion mechanism next to the audited one is how data disappears with nothing to show who agreed to it. The substrate is asked first and the record written second. A bucket recorded as versioned before the object store agreed is one the catalog describes as recoverable while it is not.

Body

ConfigureBucketRequest

List token budgets with the usage consumed in each current period.

Set a token budget for a window.

The budget is reserved before a call against an estimate and charged the real usage after, so a call that would cross the ceiling is refused rather than discovered afterwards (DATA-GAI-017).

Body

SetBudgetRequest

List the catalogs this tenant can read.

One per tenant in this phase. Multiple catalogs — a sandbox, a catalog shared in from another tenant — arrive with data sharing.

`getSchemas` and `getColumns`, as this principal will experience them.

The single most important endpoint for BI governance, because a tool asks once and then writes the answer into the workbook. Every measure, relationship and visual an analyst builds is written against this list. So it is not the table's schema. It is what the query planner will expose to *you*, derived from the same obligations: a column policy denies is absent, and the count of absent columns is reported so a modeller who can see the table is not told a five-column table has five columns. Filtering results instead would leave the model already built on a column that never arrives. A masked column is reported as `VARCHAR` whatever the schema says, because every mask renders as text — a masked `bigint` arrives as `[redacted]`, and a tool told it was a number fails the refresh with a type-conversion error nobody can trace back to a policy. Recomputed every call and never cached on the connection. A cached catalogue would make the one surface where a revocation must be visible the one surface where it is not.

The tables and columns this principal may read, for browse and complete.

One call, filtered per principal. The editor's schema tree and its completion list are the same question — what may I name in this statement — and answering it with the tenant's whole catalog would put tables somebody cannot open into a menu in front of them. Table and column names are themselves a disclosure: `hr.redundancy_model_q3` tells you something whether or not you can read a row of it. Unlike the catalog's list endpoints, which are authorized per namespace and leave each table's own check for when it is opened, every table here is authorized individually before it is returned. `withheld` says how many existed and were left out, so a shortened list is visibly shortened rather than looking like an empty namespace, and `truncated` says the catalog was larger than one answer holds — a different fact from being denied, and kept separate so a large tenant does not look like a locked-down one.

Everything you may ask about, in words.

The metrics whose table you may read, and their dimensions. A metric over a table you cannot open is absent, so asking about it gets the same answer as asking about a word nobody defined — a box that said “you are not allowed to know about salaries” would have told you the company records salaries, to anybody who guessed the word. This is what a completion menu beside the question box should be built from. Offering a word that will be refused is the same disclosure as answering it.

Clean rooms in this tenant, live and closed.

Revoked and expired rooms are included, because a list of only live ones cannot answer whether joint analysis ever happened — which is the question a review starts from.

Agree a joint analysis whose inputs neither party may read.

A clean room fixes, up front, the entire space of questions it will answer: each participant contributes one table, one join column, named governed metrics, and the dimensions a run may group by. Runs compile per-key metric cells for each leg, intersect them in memory, suppress every group below `min_group_size`, and return group values, cohort sizes and rollups — the join key never appears in an answer. **Runs execute as you.** Each leg is authorized separately through the ordinary compute path under your read access, checked when the room is made and again on every run, so a room stops working the moment you lose a contributed table. Metrics must exist in the semantic registry and belong to the participant's table; `count_distinct` is refused because it cannot be combined from per-key parts without exchanging the keys' values. `avg` rolls up as the mean of the per-key means. The ceremony matches the delegation: a written reason on every room, an expiry always — thirty days by default, a year at the outside — and revocation that takes effect on the next run.

Body

CreateCleanRoomRequest

Revoke a clean room. Dead on the next run.

Nothing is deleted and no sweep is waited on: the run path consults the revocation. Revoking an already-revoked room succeeds — two people who both wanted it closed should not get an error that reads like a failure.

One room: its participants, metrics, minimum cohort, reason and expiry.

Who ran this room, when, and the shape of what came out.

The shape of each run, never its content: the grouping, the groups returned and suppressed, and the query ids of the legs. A history that kept the answers would be a second copy of them held against no expiry.

Run the room's fixed analysis. Aggregate rows only.

The request names optional `group_by` columns, which every participant must have declared — a run is refused rather than trimmed when it asks for more. Each leg is compiled from the room's definition and the metric registry and executed under the room creator's re-checked read access; there is no way to submit SQL to a room. The reply carries group values, the cohort size and each participant's rolled-up metrics, plus how many groups were suppressed for falling below the room's minimum. It never carries a join-key value.

Body

RunCleanRoomRequest

The thread on one subject, oldest first.

Requires `?kind=` and `?subject=`, and checks the subject's own read permission. There is no tenant-wide list of comments, because there is no permission that would fit one.

Comment on a table, query, dashboard or notebook.

The comment inherits its subject's read permission — you may write on a table you may read, and not otherwise. A comment surface open to the tenant would be a side channel around every row filter in the platform, and a good one: nobody writes “have a look at the numbers”, they write the numbers. Mentions are a list of principals, never something read out of the body. A server that scanned prose for names would notify people nobody meant to involve, most reliably on the comment that pasted in an error message. A mention grants nothing. If the mentioned principal cannot read the subject they come back in `unreachable_mentions` and are notified anyway — they may be about to be given access, or asking for it may be the point — but they will follow the notification to a refusal, and the author is the only person who can act on that.

Body

PostCommentRequest

Delete your own comment, or somebody's as a moderator.

A tombstone rather than a removal. The row stays so the shape of the thread does — a reply that outlived the comment it answered reads as an answer to whatever is above it now, which misrepresents what was said more than an acknowledged gap does.

Edit your own comment. The edit count travels with it.

Body

EditCommentRequest

Resolve a thread, or reopen it with `{"resolved": false}`.

Anybody who may read it may do either. A thread only its opener can close stays open forever once they change teams, and the person who knows a question has been answered is usually the one who answered it.

Body

ResolveCommentRequest

Every compute product in the workspace, with its state and SKU.

A compute product is the metered unit a workspace orders: a SQL warehouse, a job cluster, an all-purpose cluster or a serverless pool, in one of five T-shirt sizes. Each is billed per second against the rate card by its `<product_type>.<compute_type>` SKU while it is `running`. The workspace is the caller's own, from the gateway, so the list is this workspace's products and not the tenant's.

Create a SQL warehouse, cluster or serverless pool.

`product_type` chooses the product and its lifecycle. A `sql_warehouse` and an `all_purpose_cluster` are created `stopped` and started on demand; a `job_cluster` and a `serverless_pool` are created `running` — the first because it is spun up for a job, the second because being warm is the whole product. `auto_stop_seconds` and `min_clusters`/`max_clusters` belong to a `sql_warehouse` alone: it is the only product that stops itself when idle and the only one that scales. Setting either on another type is a 422, so each type stays honest about what it promises. `compute_type` is the size, from `x_small` to `x_large`, and together with the product type it is the SKU the per-second price is read from.

Body

CreateComputeProduct

Returns

ComputeProduct

Delete a compute product.

Terminates the product and removes it. Destructive because it ends the product rather than pausing it — the name is freed for the next product that wants it. A running product's per-second metering stops with it, because the sweep bills only what is still `running`.

One compute product: its type, size, state and SKU.

Returns

ComputeProduct

Start a stopped SQL warehouse or all-purpose cluster.

Brings a `stopped` product up and stamps when it started running. Starting one that is already running is a success and changes nothing. Refused for a `job_cluster` and a `serverless_pool`: neither is started by hand — a job cluster runs for its job and a serverless pool is always warm — so asking to start one is a 422 that says so rather than a silent no-op that leaves the caller wondering.

Returns

ComputeProduct

Stop a running product to take it off the invoice.

Takes a `running` product down and stamps when it stopped. Stopping one that is already stopped changes nothing. Refused for a `serverless_pool`: it is managed warm capacity, and stopping it by hand would be operating a product sold as not needing operation. The same `manage` permission as start, because starting what you cannot stop is a bill with no brake.

Returns

ComputeProduct

Your open connections.

Yours, not the tenant's. The question this answers is “what have I left running”; “who is connected” is a different question with a different permission, and it belongs on an operator surface.

Open the connection a JDBC or ODBC driver holds.

The driver is a jar or a shared library on the analyst's laptop; this is the endpoint it bridges to. Everything interesting is on this side, because the driver's job is to translate shapes and this side's job is to decide what may be seen. The connection carries no entitlement. It is a handle for cursors and a name for the tool, and every statement and every catalogue read authorizes from scratch — so a revocation reaches the workbook that has been open on somebody's desk since Tuesday, which is exactly the case a cached decision would miss. `client` is recorded and never acted on. The day something behaves differently for Tableau than for Excel is the day this stops being a label and becomes a compatibility mode.

Body

OpenConnectionRequest

Close it, dropping any unread rows with it.

A cursor that outlived its connection would be rows nobody can reach and nobody will free.

Every connector in this tenant, with its state and last error.

Point one source at one table and let it keep up.

Seven families of source — `file`, `database`, `cdc`, `nosql`, `stream`, `internal` and `saas` — behind one contract, because what differs between reading yesterday's CSV drop and following a Postgres replication slot is the driver, not the promise. Every one of them is a sequence you resume: the connector holds an opaque `cursor`, asks for the next bounded batch, lands it, and only then moves the cursor. A crash between the landing and the move replays the batch — at-least-once, said out loud, because a connector claiming exactly-once over a source that cannot promise it is lying on the source's behalf. Rows land through the same door a client's `POST .../rows` uses, so a data contract that would reject the write rejects the ingestion, and a sealed column stays sealed. The target table is created from the declared schema if it does not exist; the schema is declared rather than sniffed, for the reason a federated table's is — an undeclared column is one no policy can mask, and so this drops it and counts it in `dropped_fields` rather than quietly widening the table. `write` decides how a batch meets what is already there. `append` suits the sources that emit facts — a file drop, a topic, an event feed. `upsert` suits the ones that emit state: a change feed and a Salesforce account are both "here is the current version of record 41", and appending those produces a table where the answer to a question about a customer is a list. It also settles what at-least-once costs, since a replayed batch of upserts writes the same values over themselves. `saas` is one kind for Salesforce, HubSpot, Workday, NetSuite, ServiceNow and the rest, with the vendor in `address.provider`. One kind rather than one per vendor, because an enum that grows whenever a sales team asks for Zendesk is an enum nobody reviews. Those sources rate-limit, and a 429 is the source working as designed rather than a fault: an adapter that comes back with a wait sets `resume_at` and is skipped until it passes, without counting toward the failure limit. Credentials are named, never carried: `secret` is a reference the runtime resolves. An `address` field that looks like a password is refused outright. Registration probes the source. A connector that cannot reach what it was pointed at fails here, loudly, rather than at 3am on its first scheduled pass.

Body

ConnectorDefinitionRequest

Stop the feed. The rows it already landed stay.

Deleting a connector deletes a subscription, not a table. What was ingested is ordinary data in an ordinary table and outlives the thing that put it there — dropping it here would make removing a misconfigured feed indistinguishable from destroying a year of history.

One connector: where it is in the sequence and how it is doing.

`cursor` is the resume point, opaque on purpose — a file connector's is a key and a byte offset, a CDC connector's is an LSN, and code that parses either will break when the driver changes its mind. `failures` counts consecutive stumbles, not lifetime ones; it resets on the first batch that lands. A connector that keeps failing is disabled rather than retried forever, because the usual cause is one record the source will hand back identically every time.

Replace the definition, keeping the place in the sequence.

The cursor survives, so widening a mapping or slowing a schedule does not re-read the source from the beginning. Changing the target or the kind does not: those make the old cursor meaningless and are refused, since a resumed LSN pointed at a different table silently loses everything before it. A disabled connector comes back on a successful replace, on the grounds that editing it is how an operator says they have dealt with the thing that broke it.

Body

ConnectorDefinitionRequest

Ask the source whether it is there, without moving the cursor.

Returns what the source says its fields are, which is the fastest way to see why a mapping is dropping everything: the answer is usually that the column is called `customer_id` and the mapping says `customerId`.

Run one pass now.

One bounded batch, the same one the sweep would have taken, synchronously so the caller sees the error rather than finding it in a state field later. Used to prime a new connector and to confirm a fix before re-enabling the schedule. A manual pull re-checks write permission on the target. A connector whose owner has since lost `data:table:write` stops here, because the alternative is a revoked principal still writing through something they set up while they had it.

List this tenant's corpora.

Create a corpus — a named collection of documents to retrieve over.

A corpus is what a RAG pipeline retrieves over, and the thing access to the documents within it is granted on. Naming and describing it up front is what lets a later pipeline cite it and a reviewer say what it holds.

Body

CreateCorpusRequest

Describe one corpus.

Dashboards in this tenant, without their tiles.

Create a dashboard, or replace its tiles and filters.

A filter is a declared, typed parameter and a tile refers to it as `${name}`, which is rendered into the tile's SQL as a literal by the same function the row-filter compiler uses. Dashboard filters are the notebook parameter problem with a combo box in front of it and a worse exposure — a notebook's parameters are supplied by somebody who could write the notebook, and a dashboard's are supplied by every viewer. Series colours are not accepted. They come from the platform's colour-blind-safe palette in series order, along with the dash pattern and marker that keep the chart readable when colour does not reach the reader, and a chart is refused above eight series because past that two of them look the same to everybody. Saving clears every warmed copy, not just the saver's: the queries changed, so what anybody is holding answers a question that no longer exists.

Body

SaveDashboardRequest

Delete a dashboard and every warmed copy of it.

One dashboard's definition: tiles, charts, filters, drills.

Warm this dashboard on a schedule, for you.

The refresh runs as whoever enabled it, taken from the caller and never from the body, and warms only that principal's copy. There is no fallback: a schedule whose principal is not recorded is disabled rather than run as the service. This is deliberately a smaller feature than it looks. A refresh that warmed one copy for everybody would be a cache that hands the refresher's rows to every viewer, past every filter and mask in the plan — fast, invisible, and the most common way a governed lakehouse leaks.

Body

Schedule

Run this dashboard's tiles as the caller and return the results.

**Rendered per viewer.** Every tile is an ordinary query issued as you, so it carries your row filters, your column masks, your cost gate, your queue slot and a line in your query history. Two people opening the same dashboard see different numbers when they are entitled to different rows, which is the whole point. A warmed copy is returned only when it was produced for this principal under these filter values — the cache key contains both. The scheduled refresh warms the copy belonging to whoever set the schedule and nobody else's, so it is a latency feature for one person rather than a way past the plan for everybody. The response says whether it was cached and when it was rendered; send `live` to bypass it. The permission in the path governs whether you may open the page. It grants nothing on any table. A tile that fails is one broken panel on a working page. Tiles are independent queries, unlike notebook cells, and eleven true numbers beside one error tell whoever is on call more than a blank page.

Body

RenderDashboardRequest

Make a dashboard openable by somebody who could not open it.

`workspace` and `link` shares convey the right to open the page and nothing else. The dashboard is already rendered per viewer, so two colleagues following one link see their own rows — a share inside the tenant has nothing difficult to do. `external` is the different thing. An outside recipient has no principal and therefore no entitlements, so the link runs as the person who created it: a real delegation of one person's read access to whoever holds a URL. That is why an external share needs a written reason, expires in days rather than months, and may carry at most `internal` data — a ceiling that is fixed rather than declared, because a link in an inbox has no compliance properties to weigh against one and the person choosing it is the person who wants the link to work. The token comes back once. It is stored only as a digest, so a reader of the share list cannot open every live link as its creator, and a lost link is re-shared rather than recovered.

Body

CreateDashboardShareRequest

Follow a tile's drill-down for one clicked data point.

The point is request input, not the row the server sent: only the columns the tile declared it carries survive, and their values are bound into the detail query as literals. The query then runs as you, because clicking through to the rows behind a bar is a read and is checked like one — a drill that showed detail the viewer could not have selected would be the shared-cache leak arrived at from the other side.

Body

DrillTileRequest

The marketplace: what this tenant publishes, most subscribed first.

Metadata only, and that is what lets discovery be open while the tables behind it are not. A listing carries an owner, a version, a promise and a subscriber count — never rows, never a sample. A marketplace that previewed data would be a way around every grant in the platform. Ordered by subscriber count, because that is the only signal a browser has about whether a product is trusted, and alphabetically within a tie so the order does not move between requests. `subscribed` is about you rather than the product.

Publish a table as a data product.

A data product is a table somebody has agreed to be held to: an owner, a description aimed at whoever is deciding whether to build on it, and optionally a freshness SLA. Publishing exposes **metadata and no rows**. It makes the table discoverable in the tenant's marketplace and states a promise about it; it grants nobody a read. The publisher does have to be able to read the table, though — a promise about data you cannot see is not one you are in a position to make. `sla_seconds` is a promise rather than a measurement, and omitting it is a decision the listing states rather than hides. It has a floor: a freshness promise finer than the pipelines feeding it is always breached, and an SLA nobody can meet devalues the ones that are met. A name that already exists is refused. Publish a new version instead, so subscribers keep the one they are on — an upsert here would let a breaking change arrive as a fresh v1 and skip the version check.

Body

PublishProductRequest

Withdraw the promise. The table stays.

Deprecating does not delete anything — it withdraws the promise, which is the thing subscribers were relying on, and stops the product taking new ones. Refused while anybody is still subscribed, naming the workspaces, because a deprecation that silently stranded them is the same failure as a silent breaking change one step later. `?force=true` is for the case where that conversation has already happened, and it records who was stranded.

One product: its table, owner, version, promise and status.

Who depends on this, now and formerly.

Ended subscriptions are included. An owner deciding whether a deprecation is safe needs to know who used to depend on it, and a list of only current subscribers cannot answer whether anybody ever did.

Leave a product. Recorded, not deleted.

The same permission as subscribing, because it is the same decision reversed — a separate one could be denied to somebody who would then still be counted as a subscriber. The row stays with an end timestamp, so the owner can still see who used to depend on it.

Subscribe your workspace to a product.

**This grants nothing.** It records that a workspace depends on the product, which is what makes a breaking change refusable and a deprecation reviewable. Reading the data still goes through the policy decision point on the table, or through a data share the owner made — a subscription that conferred access would be a grant issued by the person who wanted it. Held by a workspace rather than a person, because one held by an individual dies when they change jobs, and that is the moment somebody re-subscribes in a hurry. The version you are on is recorded, and a new major release leaves you on it.

Body

SubscribeProductRequest

Publish the next version, declaring what changed.

The endpoint that enforces the phase's rule: a breaking schema change is blocked or versioned, and **never silently applied**. `changes` uses the same vocabulary as table evolution. `drop_column` and `rename_column` break anybody selecting those columns; `add_column`, `widen_column` and `reorder_column` do not, because consumers select by name. Renaming is the worse of the two — a `SELECT *` keeps working and quietly returns something else. A breaking change at the same version number is refused, and the refusal names the change, the version to publish as instead, and how many subscribers are on the current one. Bump the version and the old one keeps working until each of them moves. The alternative — apply it and let a subscriber find out from a failed query — tells them later and somewhere else, by which point whoever could explain it has usually moved on.

Body

ReleaseProductVersionRequest

Data shares in this tenant, live and closed.

Filter by `?workspace=`. Revoked and expired shares are included, because a list of only live ones cannot answer whether anything was ever lent — which is the question a review starts from. Every share names the principal it renders as, since a delegation that is not obvious in this list is one nobody revokes.

Lend named tables to another workspace, read as yourself.

A workspace is the unit of isolation inside a tenant, so a team needing another team's table has two options without this: be granted the table, which is too wide because a grant follows the person everywhere it appears, or copy it, which is how a tenant ends up with four disagreeing copies of its revenue figures. This is the third. **Reads through the share run as you.** The recipient workspace has no entitlements on these tables — that is why they needed a share — so rendering as them would render nothing. The consequence is worth knowing rather than discovering: two colleagues in the recipient workspace see identical rows, and those rows are your view of the table, including any row filter you are subject to. A share is you saying "what I see, you may see". So the ceremony is proportionate to that. A written reason, on every share, because they are all delegations and it is what somebody reviewing live shares in six months actually reads. An expiry, always — thirty days by default, a year at the outside — because a delegation goes stale in a way a grant does not: your own entitlements move and the share does not notice. And a fixed list of tables, checked against your access now and again on every read.

Body

CreateDataShareRequest

Revoke a data share. Dead on the next read.

Nothing is deleted and no sweep is waited on: the read path consults the revocation. A sweep and a check usually agree, and they disagree exactly when the sweep is behind — which is the moment a revocation matters most. Revoking an already-revoked share succeeds. Two people who both wanted it gone should not get an error that reads like a failure.

One share: its tables, its reason, who it renders as, when it ends.

Who consumed this share, when, and how much.

Per recipient, busiest first. Recorded here rather than read back from query history, because a delegated read is submitted as the lender — so history would report you consuming your own share, every time. The shape of the traffic, not the traffic: read counts, row counts and bytes scanned, never values and never the predicate. A share nobody has read comes back empty rather than refusing, and that is the interesting answer — it is the one to revoke.

List the documents ingested into a corpus, with each one's status.

Submit a document for ingestion into a corpus.

The item is recorded, not parsed: it is addressed by a location in the object store, never by its bytes, and starts in status `ingesting`. The parse-chunk-embed-index pipeline reads it and moves it to `indexed` once it is retrievable — within 15 minutes of ingestion completing (Req 9.7). A document over 200 MB, of an unsupported kind, or with no location is refused here at the door. Returns 202: accepted for ingestion, not yet retrievable.

Body

SubmitDocumentRequest

Describe one ingested document and where it is in the pipeline.

Query a corpus using dense, sparse, or hybrid retrieval.

Retrieves chunks matching the query. Strategy: 'dense' uses vector similarity, 'sparse' uses BM25 keyword matching, 'hybrid' fuses both with reciprocal rank fusion. Hybrid requires fusion weights.

Body

QueryRequest

List the embedding models a corpus may pick from.

Register an embedding model a corpus may pick.

The registry is what makes switching a corpus's embedding model a data change and not a deploy (DATA-GAI-005): a corpus selects a model by name from this list. A model declares its vector width, which the per-tenant index is built for, and where it runs — self-hosted or an external provider, the fact Sovereign_Mode and the gateway route on.

Body

RegisterEmbeddingModelRequest

List this tenant's keys and their versions.

Fingerprints, states and dates. No material: there is no endpoint on this service that returns key material, because the port it would have to call has no method that produces any.

Create a tenant encryption key, or import your own.

`origin: projectx` generates the key here. `origin: byok` takes a base64 256-bit key in `material` and is the mode for a tenant whose obligation is to hold their own keys. This key never encrypts data directly. It wraps a per-table data key, and that indirection is what makes rotation cheap — see the rotate endpoint.

Body

CreateKeyRequest

Describe one key and its version history.

Rotate the key by re-wrapping, not re-encrypting.

A new version is created and every data key wrapped by the old one is unwrapped and wrapped again under the new one. The data keys themselves do not change, so no data file is read, rewritten or moved and no snapshot is produced. Rotating a petabyte costs the same as rotating a row. The response carries a sealed receipt naming the tables re-wrapped and the number of data files rewritten, which is zero. Its digest is a SHA-256 over the canonical form, so an auditor can check the claim rather than take it. The previous version is retired rather than destroyed: it can still unwrap, which is what makes a partial failure retryable instead of an outage. Destroy it separately once you are satisfied.

Body

RotateKeyRequest

Destroy one key version's material, irreversibly.

Refused while any table is still wrapped by it, and refused for the active version. Destroying a version that still wraps a data key is silent, permanent data loss: the tables keep reading until the cached data key is dropped, and then they never read again. This is also the mechanism behind crypto-shredding — destroy the key and the ciphertext is unrecoverable everywhere it was ever copied, including backups nobody can reach.

Get an evaluation run by ID, with its per-case results.

List evaluation sets.

Create an evaluation set for RAG quality assessment.

Body

CreateEvaluationSetRequest

Execute an evaluation set against a RAG pipeline.

Body

RunEvaluationRequest

List the runs executed against an evaluation set, newest first.

The comparison a set exists for: how this run's metrics differ from the last one's, over the same questions.

List this tenant's experiments.

Create an experiment to group training runs.

An experiment is a named question — “does more regularisation help” — and the runs logged against it are the attempts to answer it. Naming and describing it up front is what makes the runs interpretable later; a comparison of runs nobody can say the point of is a table of numbers.

Body

CreateExperimentRequest

Describe one experiment.

List the runs of an experiment — the comparison, as data.

Every run with its params and metrics, unranked. A caller ranks by the metric they care about or diffs the params; the service does not pick one, because picking one would be guessing which comparison was wanted.

Log a training run: its params, metrics and artifacts.

Params are the settings the run used, metrics are what it scored, and artifacts name where its outputs live — a location in the object store, never the bytes, because the run record is metadata. A run gets a generated id, so two jobs logging at once do not collide on one the caller chose. The experiment must already exist: a run with no experiment is an orphan nobody will find.

Body

LogRunRequest

Describe one run.

List this tenant's feature views, searchable by owner, entity and tag.

Optional `q` (free text across a view's name, description, entity, owner, features and tags), `owner`, `entity` and `tag` narrow the list. Owner, entity and tag are exact matches — the questions a catalog is asked are whose features these are and which key on a given entity, not whose name sort of looks like one.

Define a feature view: an entity key and the features on it.

A feature view groups the features that share one entity key and are computed together, so they can be read in a single round trip. Each feature declares a kind, which is what lets a value be refused when it does not match — a float feature that quietly accepts a string is a model served nonsense. The owner is who to ask about it, and what the catalog is searchable by: an unowned feature in production is a dependency with no one to page.

Body

RegisterFeatureViewRequest

Search the feature catalog: every feature and who owns it.

The unit of search here is the feature, not the view, because the question a catalog answers is “is there already a feature for a customer's thirty-day spend, and whose is it” — the one that stops two teams building it twice. Each row is a feature with its view, entity, kind, owner and tags. The same `q`, `owner`, `entity` and `tag` filters as the view list apply; a text query returns just the features that answer to it, or every feature of a view when the view itself matches by name or tag.

Describe one feature view and its features.

Read feature values point-in-time — the training path.

The request is a spine: the entity rows a training set is built from, each with the moment its label was true. For each row, each feature is returned as of that moment — the latest value at or before it, and nothing after. That `<=` is point-in-time correctness: a feature computed after a label's moment is the future, and training on it teaches the model to lean on something it will not have when it runs, so it looks brilliant in the lab and fails the day it ships. A row may carry its own `as_of` or fall back to a request-level one; a row with neither is refused, because a point-in-time read with no point in time is an online read at the wrong endpoint. Read as POST because the spine is a body, not a query string.

Body

HistoricalFeaturesRequest

Read the latest feature values for entities — the serving path.

Given entity ids and, optionally, the features to read, returns the current value of each by key. An entity the store has never seen a feature for is omitted rather than nulled: it has no value, and inventing one would be its own wrong answer. Read as POST because a serving request carries many entities and many features, not because it changes anything.

Body

OnlineFeaturesRequest

Write feature values — to the offline history and the online latest.

Each observation is an entity, the moment the value became true, and one or more feature values. The write lands in both stores at once: the offline history keeps every value with its moment, for training to read point-in-time; the online store advances to the latest, for serving to read by key. Writing both from the one batch is what keeps them in parity, so the online value is always exactly the most recent of the offline history and never a second number drifting out of step.

Body

WriteFeatureValuesRequest

List this tenant's file sets.

Catalogue a named collection of objects that is not a table.

Exports, partner drops, document archives, training corpora — the data every estate has in buckets and almost none of it has in a catalog, which is why the honest answer to “where is our customer data” is usually wrong. A file set gives that data the same sentences a table gets: an owner, a classification, a retention policy, an entry in the unified catalog and a place in the offboarding sweep. It is not queryable, and the catalog entry says so. Overlapping prefixes are refused: two file sets over the same bytes are two owners and two retention policies, and the sweep would apply whichever it reached first.

Body

CreateFileSetRequest

Forget a file set. Deletes no objects.

The catalog entry goes and the bytes stay, which is the safe default for a resource whose contents this service did not create. Destroying them happens under a retention policy somebody agreed to, and is audited as a deletion rather than as an unregistration. Refused while the file set is under a legal hold.

Describe a file set and what it last measured.

The object counts are a photograph with a date on it, like table statistics: counting a prefix is a listing over storage that may hold millions of objects, so it happens on request and on the sweep rather than on every read of this endpoint.

Change the owner, classification, tags, format or retention.

Not the bucket and not the prefix. A file set that can be re-pointed is a catalog entry whose history describes one pile of objects and whose contents are another — and its retention clock would carry over to bytes it never applied to.

Body

AmendFileSetRequest

Count the objects now and stamp the count with the time.

Also records when the newest object was written, which is what a retention policy on the `last_written` basis counts from. A set created in 2019 and appended to yesterday is not five years stale.

Read the tenant's GenAI governance settings.

A tenant that has never set this gets sovereign mode off rather than a 404, because no config and sovereign mode off are the same state.

Set the tenant's GenAI governance settings.

Turning sovereign mode on refuses every model not served by this deployment, at registration and at generation both (DATA-GAI-014).

Body

SetGenAIConfigRequest

List configured guardrails.

Configure a guardrail over LLM input or output.

Guardrails scan prompts and answers for PII, credential shapes or the tenant's own blocked terms, and block, redact or warn on a finding (DATA-GAI-015).

Body

CreateGuardrailRequest

List vector indices in the tenant.

Create a vector index for a corpus.

One index per corpus: the index is built for the corpus's embedding model's dimensions and holds the embeddings of the corpus's documents. Creating an index is stating that the corpus is ready to be retrieved over; the actual indexing runs asynchronously in the ingestion pipeline (DATA-GAI-006).

Body

CreateVectorIndexRequest

Get a vector index for a corpus.

Trigger a vector index rebuild.

Rebuilds the index asynchronously (e.g., after the corpus changes its embedding model). Returns 202 Accepted; the rebuild completes in the background.

Jobs in this tenant, newest first, each with derived health.

Submit a Spark, streaming or Ray job. Returns immediately.

`kind` is `batch` for Spark, `streaming` for continuous ingest and transform, or `fanout` for distributed Python. They share a record and a lifecycle; what differs is the question worth asking about them, which is why `health` is derived per kind rather than read off `state`. **Name the tables.** `reads` and `writes` are required — a job declaring neither is refused — and each is authorized separately before anything starts, read on the inputs and write on the outputs. This route's own permission covers submitting a job, not the tables the body happens to name, and without the second check the first would amount to read access to everything. **They are checked again while it runs.** A query lives for four seconds so a check at submission covers its whole life; a streaming job lives for a month, and access revoked in week two would otherwise leave a process reading a table its owner may no longer read. The sweep re-authorizes and stops what is no longer allowed. An unreachable policy service does not stop anything — unavailable is not the same answer as no. A `fanout` job may declare a `failure_budget`: how many of its tasks may fail before the job has. Without one, a thousand tasks with three failures and a thousand with four hundred are the same event, and a platform has to pick which lie to tell.

Body

SubmitJobRequest

One job, with the question worth asking about it answered.

`state` says what happened. `health` says how it is doing, and is computed on read rather than stored, because the characteristic failure of a streaming job is that nothing happens — and a status somebody has to remember to write down is one that never gets written during a stall. For a stream that means `stalled` when the numbers have not moved, separately from `lagging` when they are moving and behind: the causes differ, one being a stuck consumer and the other too much data. For a fan-out it means `degraded` while failures are inside the declared budget and `over budget` past it.

Stop a job. It will not be retried afterwards.

Cancellation is final: a cancelled job cannot be retried, only submitted again. Somebody decided this should stop, and a retry that quietly overruled them would make cancellation advisory.

Engine output for a job, read live and never stored here.

A separate permission from reading the job, because they are different exposures. A Spark executor's stderr routinely contains row values from whatever it was processing, so somebody who may see that a job failed is not automatically somebody who may see the data it failed on. Nothing is cached on the way past. A control-plane copy of those lines would be customer data sitting outside the governed store, outside its retention rules and outside its residency guarantees, acquired by accident. The cost of not keeping one is that logs last as long as the engine keeps them and no longer, which is said here rather than discovered during an incident. `cursor` is the engine's, not ours. Paging a log correctly needs the thing that wrote it; a control plane inventing offsets would be guessing at line boundaries in a stream it does not own.

Run a failed job again, from its last checkpoint.

Only a failed job. The attempt counter carries over rather than resetting, because a job on its fourth attempt that presents as a first run hides the flapping, which is usually the actual problem.

List registered LLM models.

Register an LLM model.

Models declare their provider and capabilities. The gateway routes generation requests to the appropriate provider based on registration.

Body

RegisterLLMModelRequest

Get an LLM model by name.

Generate a completion from an LLM.

Sends messages to the LLM and returns generated completion. Routes to the model's registered provider.

Body

GenerateRequest

Every metric defined in this tenant.

The definitions, which is a different question from what you may ask about — see `/v1/data/vocabulary` for that.

Define a metric, and the dimensions it may be sliced by.

One aggregate over one governed table, with a name, synonyms and a certification. Dimensions are declared alongside it and are on the same table by construction, which is the constraint the question compiler rests on. The metric's own `filter` is SQL written by its author. That is not a hole: the statement it lands in goes through the planner, which guards every table in it or refuses the query, so a filter naming a second table produces a second guarded relation authorized against whoever asked the question.

Body

DefineMetricRequest

Remove a metric definition.

One metric and its dimensions.

List this tenant's registered models.

Register a model — the name a deployment points at.

A registered model is a stable handle; its versions are the successive things that handle has meant. This creates the handle. Versions are added to it, and a serving endpoint points at the name and follows whichever version is in production.

Body

RegisterModelRequest

Describe a model and its versions, with each version's stage.

Register a new version of a model.

The version number is assigned — one past the highest the model has — rather than sent, so two registrations cannot claim the same version. A version may cite the training run that produced it (`source_experiment` and `source_run`), which is the thread lineage follows back to the data. It starts in stage `none`: registered, but serving nothing.

Body

RegisterModelVersionRequest

Approve a version for production.

The sign-off the production gate checks for. Held apart from `promote` on purpose: the person who trained a model is not the person who approves it serving real traffic, and one permission for both would make the review the trainer's to grant themselves. Records who approved and when.

Body

ApproveModelVersionRequest

Attach a model card to a version — required for production.

A model card answers what the model is for, what it was trained on, where it falls down, and who it could affect — the four questions whose absence is how a model turns out to have learned something nobody wrote down it might. Each section must be filled in words, because a card with blank sections is the box ticked without the thinking done. Production promotion is refused until a version is both documented and approved; documenting is the author's job and approving is the reviewer's, held apart on purpose.

Body

SetModelCardRequest

Check a version for data, prediction or concept drift.

A model is trained on a snapshot and then the world moves. Data drift is the inputs shifting, prediction drift the outputs shifting — both measured as the population stability index between a baseline distribution and a current one, the usual bar being 0.1 to watch and 0.2 that it moved. Concept drift is the relationship shifting, invisible to inputs and outputs and visible only as accuracy falling, so it is measured as the drop against the baseline the model was accepted at. The threshold is configurable because how much drift is too much is a judgement about the cost of being wrong. Returns the numbers and the verdict; monitoring surfaces movement, it does not decide what to do about it.

Body

DriftCheckRequest

List a version's evaluations.

Record an evaluation of a model version.

Takes the predictions the model made — the true label and the predicted label per sample, and the segment each belongs to — and returns what a leaderboard number leaves out: the confusion matrix (which mistakes, not just how many) and per-slice accuracy (for whom, not just overall). The headline precision, recall and F1 are the macro average across classes, so a rare class — the fraud, the defect — is not drowned by a common one. Computed from the supplied outcomes rather than by running a model, so the same harness scores any model.

Body

RecordEvaluationRequest

Describe one evaluation: metrics, confusion matrix and slices.

Run a bias-and-fairness check over a protected attribute.

Slice analysis with a line drawn on it: per-group accuracy and, given a positive label, positive-prediction rates for a protected attribute, checked against the limits the caller sets — a minimum group accuracy, a maximum accuracy gap, a minimum four-fifths disparate-impact ratio. Only the thresholds supplied are checked, because fair is a policy about how unequal a model's errors may be and not a property a platform can hard-code for you; and a group below the support floor is reported but held out of the maths so a limit does not fire on three examples. Returns `passed`, but does not itself block anything — a report is the gate's evidence, not its verdict, which is why this is a read over supplied predictions, not a stored decision.

Body

FairnessCheckRequest

Trace a version back to the data it was trained on.

The chain version → training run → table at a snapshot. The snapshot is the point of it: “trained on sales.orders” is a habit, “trained on sales.orders at snapshot 4211” is a fact you can reproduce. Read from what the version and its run recorded, so the answer does not depend on the catalog graph being reachable — the same edges are also registered there when the version is created, for the cross-pillar graph to walk.

Move a version to a stage: staging, production or archived.

Promotion to `production` is refused unless the version has been approved — a version reaching production without a reviewer's sign-off is exactly the event the approval exists to prevent, so it is blocked rather than allowed and recorded after the fact. Every other transition is ungated.

Body

TransitionModelVersionRequest

List this tenant's namespaces.

Create a namespace.

The tenant is taken from the authenticated context and prefixed to the name given here, so there is no namespace a caller can name that belongs to anyone else.

Body

CreateNamespaceRequest

Drop an empty namespace.

409 if it still holds tables. Dropping them along with it would make an unrecoverable action available by typo.

Describe a namespace and its properties.

Notebooks in this tenant, without their cells.

Create a notebook, or add a revision to one.

Cells are ordered and each names one of `python`, `sql`, `scala` or `r`. Every save produces a new immutable revision, including one that changes nothing — a `digest` is returned so a no-op is visible without diffing. Revisions have to be immutable because a scheduled notebook runs source nobody is watching, and “which version ran at 03:00 on the 14th” cannot be answered by a document that is edited in place. Parameters are declared here and bound at run time. A `${name}` in a SQL cell is substituted as a typed literal by the same renderer the row-filter compiler uses, so a value can never become an identifier, an operator or a second statement. A `${name}` in a Python, Scala or R cell is refused outright: substituting a value into code would let whoever supplies the parameter choose what executes. Those cells receive the values as a binding instead. Placeholders naming an undeclared parameter are refused at save rather than at run, because a notebook that refers to a parameter nobody declared is broken from the moment it is written and the alternative is learning that from a failed run at three in the morning.

Body

SaveNotebookRequest

Read a notebook out of a repository and save it as a revision.

The notebook's name comes from the file's own header, never from this request. Otherwise “pull `team/report.py` into `payroll_export`” would let anybody who can write any file in the repository replace the source of a scheduled notebook that runs as somebody else. Content arriving from Git is untrusted input and is validated exactly as a request body is — the whole point of the feature is that somebody edited the file in an editor this platform has never seen. Treating it as trusted because it came through source control would make the repository a way to write past every check on the write path, including the rule that a code cell may not interpolate parameters. The revision is attributed to whoever ran the pull, with an origin of `git`, because “who changed this” has a different answer when the change arrived through a sync.

Body

PullNotebookRequest

Delete a notebook and its revisions.

One notebook: its head revision's cells, parameters and history.

Push the head revision's source to a bound repository.

Source, and only source. The exported format is a flat text document of cells and parameter declarations, and it has nowhere to put an output — not “outputs are stripped”, which is a step that can be skipped, but a format in which a result cannot be represented. The thing being prevented is a rendered table of customer records committed to a repository, cloned everywhere and permanent, in a system with none of this platform's governance. Text rather than JSON because the reason to use version control is the diff, and a document whose sources are escaped strings reviews as a wall of `\n`. Idempotent against content: committing a notebook whose source is already on the remote does nothing and says so, rather than making an empty commit.

Body

CommitDocumentRequest

One revision, exactly as it was saved.

What a run record's `revision` points at. Immutable, so the source a run executed can be read back long after the notebook moved on.

Runs of this notebook, newest first.

Somebody else's run shows what happened — the revision, as whom, when, how long, which queries it issued and whether each cell worked — and not its outputs or its parameter values. Both of those are data: an output is rows the runner was entitled to and you may not be, and a parameter is frequently the customer identifier they were looking into. Parameter names are kept so the run is still legible.

Run the head revision now, as you.

Every SQL cell goes through the ordinary submitted-query path, so it gets your row filters, your column masks, the cost gate, a queue slot and a line in the query history — the cell result carries the `query_id`, which is where the plan and the applied obligations live. Code cells go to a kernel that is handed your principal. The permission in the path governs whether you may run notebooks at all. It grants nothing on any table: each one the notebook reads is authorized separately, as you, at the moment it is read. Cells run in order and a failure stops the run, leaving the rest `skipped`. Notebook cells are not independent — the third usually needs the second to have created something — so continuing produces a page of errors whose first line is the only real one, or worse a plausible answer computed from a stale intermediate.

Body

RunNotebookRequest

One run, with per-cell state and timings.

Set the schedule, and with it the principal its runs execute as.

There is no “run as” field, deliberately. Scheduled runs execute as whoever set the schedule, taken from the caller — a field would be a field somebody sets to the account with the most access, which is exactly the outcome this avoids, and it would be an escalation primitive besides, since anybody who may schedule a notebook could then run one as anybody they can name. There is also no fallback. A schedule whose principal is not recorded is disabled rather than run under a service identity, and a scheduled run whose principal has since lost access fails. Failing is correct: a nightly report that keeps working after its owner leaves the company is the bug this prevents.

Body

Schedule

Run history, newest first.

One run, step by step, with the query id behind each step.

`query_id` on a step opens the query history entry for it: the rewritten SQL, the obligations that were applied and what it cost. `snapshot_id` is the version the target ended on, which is what a reader time-travels to in order to reproduce the answer.

Create the optional Bronze/Silver/Gold pipeline default.

This is an onboarding shortcut, not a required storage architecture. It returns an ordinary pipeline on the existing governed execution path. Bronze appends the declared source projection, Silver applies the required quality rules and quarantines failures by default, and Gold replaces the requested grouped aggregate. Outputs are `<layer>_<pipeline>` tables in the source's existing namespace, so no hidden namespace-creation privilege is needed. Existing custom pipelines and tables are unaffected.

Body

MedallionPipelineRequest

This tenant's pipelines, each with its execution levels.

Define a pipeline: steps, and what each of them writes.

**The graph is derived.** A step says what it writes and its SQL says what it reads, and the ordering falls out of the two. There is no `depends_on`, because a declared order that disagrees with the queries is a pipeline where every step goes green and the numbers are yesterday's. `after` exists for the ordering data flow cannot express and can only hold a step back, never let one start early. **dbt models work.** A step of kind `dbt` may write `{{ ref('other_step') }}` and `{{ source('sales', 'orders') }}`, and both are resolved here — on this request, addressed to the person who can fix a typo, rather than at 03:00 to whoever is on call. Anything else in braces is refused rather than passed through to the engine. **A target may need declaring.** A step writing a table that does not exist yet must carry a `schema` for it; column types cannot be read off a SELECT. A step writing a table that already exists uses it, and is checked against it now rather than when it runs. **`mode` decides what a run costs.** `replace` empties the target and writes the new answer, and is the default because a step that recomputes an aggregate and appends it produces a table with two of every row. `append` adds. `incremental` also needs `incremental_on`, a column the step selects: the run takes only rows above the highest value already in the target, and skips the step entirely when no input table has committed a new snapshot since it last ran. A row that arrives below the mark is not picked up — the way back is a `replace`, which is a full refresh. **Bad rows are moved, not dropped.** A step may carry a `quality` block: `expectations`, each a rule every row must satisfy, and `on_violation`. Under the default `quarantine` the rows that fail are written to `dq_`-prefixed table beside the target, stamped with the run, the step and the first rule they broke, and the rest land as normal — so a million-row load with four bad rows lands 999,996 and leaves four somebody can look at. The alternative every hand-rolled pipeline reaches for, a `WHERE` in the transformation, does the same arithmetic and leaves no record that the four ever existed. **A run that does not look like the ones before it is remarked on, never refused.** Volume and freshness are watched from the run record and cost nothing, so they are on unless a step's `monitor` block turns them off. Naming columns in `monitor.columns` adds their mean and null rate to the same pass the expectations are counted in, so drift costs one scan and not two. The baseline is the median of recent successful turns of that step, and the step runs to completion regardless: an expectation says what is allowed and can refuse a row, an anomaly says what is usual and cannot — Black Friday is unusual and not wrong. YAML is the same document. This endpoint takes JSON and the CLI reads the file, because a YAML parser inside a service that accepts untrusted input buys nothing the client cannot do safely.

Body

PipelineDefinitionRequest

Forget the pipeline. The tables it wrote stay.

Deliberately unlike dropping a materialized view. Those are platform-owned copies; a pipeline's outputs are the tenant's own tables, which other things read and policies are written against. Deleting the recipe is not permission to delete the dinner.

One pipeline, with its compiled SQL and its execution levels.

`compiled_sql` is the statement after `ref` and `source` were resolved, and it is what actually runs. `levels` groups the steps into the sets that could go at once — the DAG, in the order the runner will take it.

Replace the whole definition.

The whole document, not a patch on one step. Whether a graph is valid is a property of all of it — a step edited alone can close a cycle two steps away — so the unit that is validated is the unit that is sent.

Body

PipelineDefinitionRequest

Reprocess a date range, one window at a time.

Cuts `[from, to)` into windows of `window_hours` and queues a run for each, oldest first. One run per window rather than one run for the range: five hundred days as a single execution is a single thing to fail, restart from the beginning, and have no idea how far it got. Each window is cleared from the target before it is written, so backfilling the same range twice leaves the same rows as doing it once. That is what the `incremental_on` column is used for here, and why every step must have one before a pipeline can be backfilled — a backfill that reprocessed three steps of five would leave a table that is half-corrected, which is worse than one uniformly wrong because nothing about it looks unfinished. `concurrency` bounds how many windows are in flight at once. Four hundred queued windows would otherwise be four hundred tickets at the front of the query queue and every other pipeline in the tenant waits behind them.

Body

BackfillPipelineRequest

Push a pipeline definition to a bound repository.

Canonical JSON, because a pipeline is structured configuration rather than code. Canonical so that saving through the API and committing does not produce a diff purely from key ordering.

Body

CommitDocumentRequest

Tell a pipeline the thing it is waiting for happened.

For what neither a clock nor a snapshot can express: a partner's drop finished, a month closed, a job that is not ours reported done. A pipeline not listening for that event gets `started: false` and a 200, not a 404. The caller is usually announcing to everything that might care, and "not for you" is an answer rather than a failure.

Body

AnnouncePipelineEventRequest

Where each checked step keeps the rows it refused.

The index, not the rows. A quarantine table is a real table in the catalog, classified like the target it came from, and it is read through `POST /v1/data/tables/:table/queries` with this principal's row filters and column masks applied. A browser that served the rows from here would be a second door onto data the first one governs. Every row in one carries `dq_run`, `dq_step`, `dq_rule` and `dq_at`: which run held it back, which step, the first rule it broke and when. `dq_rule` is what makes the pile countable by cause — a hundred thousand rows failing one rule is a schema change upstream, and a hundred thousand failing forty is a different morning. A step whose expectations have never been broken has no table yet and is listed with `exists: false`, so the answer describes every check the pipeline makes rather than only the ones that have caught something.

Ask for a run. Returns immediately with a queued one.

202 and a `pending` run; the scheduler picks it up. A pipeline's work is measured in minutes, and a request that waited for it would be a connection held open across a deploy, a timeout somebody retries, and a run started twice. Poll `/v1/data/pipeline-runs/:run` for where it got to. Each step is a submitted query with a destination, and goes through the same machinery one typed by hand does: authorized per table, rewritten with this principal's row filters and masks, priced, admitted to a queue, killable, and recorded in the query history with the rewritten SQL an auditor can read. A pipeline cannot read what the principal running it cannot. A step that fails stops the steps that depend on it and nothing else. The rest run, and the ones that did not are reported as `skipped` naming what blocked them — because "did not run" and "was not reached" send different people to look at different things. A step with a `retry` policy is tried again before it is called failed, and sits `pending` with a `not_before` while it waits out the backoff.

Body

RunPipelineRequest

Turn a schedule off, or back on, without losing it.

The state a pipeline goes into while somebody works out why it has been failing. Deleting the schedule would also stop it and lose the expression, and a cron retyped from memory is how a daily job becomes an hourly one. Re-enabling computes the next fire from now, so a schedule that was off for a week does not immediately run for the Tuesday it missed.

Body

SetPipelineEnabledRequest

Where this workspace's lakehouse runs: its place, region and root.

The caller's own workspace, taken from the gateway, never a path or body. A workspace that has not been bound to its deployment is a 404 — the same 404 an absent one gets — and a request the gateway gives no workspace is a 400, because a placement belongs to one. The warehouse root is returned in the clear rather than masked: which object store a workspace's tables sit on is a residency answer its owner is entitled to, not a secret.

Returns

WorkspacePlacement

Bind this workspace's lakehouse to its deployment place.

Reads the workspace's cloud deployment (M3) and records where its lakehouse lives — the object-store root its tables are created under, and the REST catalog and compute endpoints of its data plane — then registers the placement in the unified catalog. The control-plane catalog stays the single source of truth for table metadata regardless of which place the bytes land on. `place` and `region` are optional and the deployment is authoritative: the lakehouse follows where the workspace was deployed and does not choose. Naming a place or region that disagrees with the deployment is a 422, and a workspace with no deployment is a 404 pointing at the cloud plane. Binding is idempotent — a second bind re-reads the deployment, which is how a placement catches up to a move.

Body

BindPlacement

Returns

WorkspacePlacement

Every compute pool, its size, and what it is being asked for.

A pool is the capacity a queue runs on. Three kinds, and the kind is a promise rather than a label: `ephemeral` may scale to zero, `dedicated` keeps at least one worker and is refused at zero, and `gpu` needs a GPU in its shape and runs one query per worker. `status.workers` is what is running and `status.desired` is what was asked for. They differ while a resize is in flight, which is the honest answer to “why is my query still queued” — a pool that cannot get capacity would otherwise look exactly like one that has.

Define a compute pool in a workspace.

`pool` is `workspace.name`. The workspace is required because a workspace is the unit of isolation within a tenant — data, compute, network and budget — and a pool outside one is outside all four. `max_workers` bounds how large the pool may get and `burst_workers` bounds how many workers may join in a single scaling decision. Both are limits and they limit different things: without the second, a pool is permitted to go from two workers to two hundred in one tick. `warmup_seconds` is what waking this pool costs the next caller. It is declared rather than measured because scale-to-zero does not remove that latency, it moves it onto somebody's afternoon, and a pool that claims to start instantly produces a query that is inexplicably slow once a day.

Body

CreatePoolRequest

Remove a pool, once no queue runs on it.

Refused while a queue is still bound, because the alternative is a queue whose capacity silently reverts to whatever `max_concurrency` was left over from before it was bound — a number nobody chose and everybody will assume was deliberate.

One pool, with what the runtime says is actually running.

Resize a pool, change its shape, or pause it.

Applied immediately rather than at the next scaling pass, because the reason to raise a floor during an incident is that things are stuck now. `paused` gives every worker back and keeps the definition. A pause that left the workers running would be a pause that saved nothing, which is the only thing pausing is for. The kind cannot be changed. It decides which constraints apply to the record, and changing it would re-interpret a pool that already has queues and running work on it.

Body

UpdatePoolRequest

Place a piece of work on a pool, on the right kind of worker.

`checkpoint_every` is both the promise that this work can resume and the bound on how much a reclaim will repeat. Zero — the default — means it cannot say where it got to, and work that cannot resume is never placed on preemptible capacity. That rule is the whole of spot safety. A workload that restarts from nothing has paid for the discount with somebody's afternoon, and nobody decided that: the decision would be made implicitly by whichever scheduler had a spot worker free. The reply says whether the work may be interrupted and how much it would redo, so a caller who would rather not can ask for a pool with no spot in it instead of finding out later.

Body

PlaceAssignmentRequest

Where a piece of work is running and how far it got.

Record where a workload has got to, so a reclaim does not lose it.

`position` is the platform's number — how far through, used for the resume point and the redo arithmetic. `token` is the workload's own and is opaque here: a control plane that understood it would need changing every time an engine did. Only the latest is kept, and a checkpoint behind the one on record is ignored rather than applied. A late write from a worker that has already been replaced would otherwise rewind the resume point and redo work a newer worker has since committed.

Body

SaveCheckpointRequest

Put displaced work back on a worker, at its last checkpoint.

A provider is taking a worker back. Save the work on it.

Called by the runtime when a reclaim notice arrives, which gives thirty seconds to two minutes. In that window three things happen: the worker stops being given work, a replacement is requested, and everything on it is handed back with the position of its last checkpoint. Deliberately not “let it finish” — no amount of grace reliably fits into thirty seconds. The replacement is requested before the drained worker is written off, so the pool never dips below the size the autoscaler decided on and a scaling pass landing in that window does not read the dip as demand falling. What this guarantees is that nothing is lost. Work between the last checkpoint and the reclaim is done **again** — a checkpoint is a position, not a transaction — and the amount is bounded by the `checkpoint_every` the workload declared.

Body

ReportPreemptionRequest

Projects in this tenant, with their environment names.

Create a project, or replace its environments.

A project is named `workspace.project` and lives inside a workspace the policy service knows about — this endpoint refuses a workspace nobody created, because a project outside the isolation boundary is not isolated by anything. An environment declares the namespaces its assets may read. That is a constraint and not a default: SQL is never rewritten to point somewhere else, so a production dashboard querying a development table is refused when it is saved rather than quietly redirected. The mistake is worth catching at save time because it is silent — the wrong table usually returns plausible numbers. Libraries are pinned to exact versions; a range is refused. The list is code that runs with the permissions of everybody who runs a notebook in the environment, so it has to be something a reviewer can read and something that means the same thing tomorrow. Marking an environment `protected` moves writes into it behind `data:project:promote`. Removing that mark needs the same permission, or the protection would be liftable by whoever it was meant to constrain.

Body

SaveProjectRequest

Delete a project.

Needs `data:project:promote` as well when any environment is protected, since deleting the container is a way to remove the protection from everything inside it.

One project: its environments, namespaces and pinned libraries.

List the names of published prompt templates.

Publish a prompt template version.

Publishing under a name that already exists creates the next version rather than editing the current one (DATA-GAI-012). There is no update and no delete: an evaluation run recorded against version 3 has to keep meaning the text that produced it.

Body

PublishPromptTemplateRequest

List every published version of one prompt template.

The platform-default provider keys, each a fingerprint and never a key.

Remove the platform-default key for a provider.

Set or rotate the platform-default key for a model provider.

Staff set the deployment's default key here. It is used by every tenant that has not entered its own key for the provider, and overridden the moment one does. The key goes to the vault and never onto the record, so a later read shows its fingerprint and never the material (G2). Rotating to the same material is refused, because that is not a rotation.

Body

SetProviderKeyRequest

This tenant's provider keys, each as a fingerprint and never a key.

Revoke this tenant's key for a provider, falling back to the default.

Set or rotate this tenant's key for a model provider.

The key goes to the vault and never onto the record, so a later read shows its fingerprint and never the material (G2). Setting a provider that already has a key rotates it: the vault path is unchanged, so nothing holding a reference has to be updated, and the fingerprint changing is how the rotation is confirmed. Rotating to the same material is refused, because that is not a rotation. A tenant key overrides the platform default for this provider at the next generation; removing it falls back to the default rather than disabling generation. The platform default itself is set by staff, not here.

Body

SetProviderKeyRequest

Run the tool's own SQL and return the first page.

A thin wrapper over the same submitted-query path the SQL editor uses. The query is planned, priced, admitted to a queue, subject to your row filters and column masks, and lands in the query history under your name. There is no BI dialect, no rewriting for compatibility and no relaxation — the SQL a tool generates is ugly and is otherwise ordinary. `SELECT *` expands against the guarded relation, so it returns the columns you may read rather than the columns the table has. A result larger than the row ceiling comes back marked `truncated`. Never inferred from a short page: a page is short at the end of every result, and a report that quietly excludes December is worse than one that refuses to load.

Body

ExecuteStatementRequest

The next page of a running cursor.

Forward-only. The cursor advances on read rather than on acknowledgement, because nobody ever sends the acknowledgement for the last page. A driver that loses one re-runs the query, which is what every driver already does when a coordinator is replaced.

Search the query history, newest first, with plans and profiles.

Every submission, including the refused ones. A history holding only the queries that succeeded cannot answer the question people actually bring to it, which is what somebody tried to read. Your own entries carry the SQL as submitted and the SQL as rewritten, so the row filter that applied is visible without taking the platform's word for it. Somebody else's carry the submitted SQL, the tables and the full profile, but not the rewrite or the obligations: those describe *their* entitlements rather than the query, and the history is not the place to learn what a colleague is restricted to. Entries shown that way say so in `redacted`. Filters narrow and never widen, so no combination of them returns an entry that the unfiltered list would not. Paging is by `cursor` rather than by offset, because the list grows at the head while you read it and an offset would skip an entry for every query anybody runs meanwhile.

Run submitted SQL, with entitlements applied inside the plan.

Write ordinary Trino SELECT against `namespace.table`. Before anything runs, every table the query names is authorized separately and replaced by a relation that already carries that table's row filter and column masks — so a join, a window or a subquery cannot compose its way to a row the policy excluded, because the excluded rows are not in what it is composing. The permission in the path (`data:query:run`) governs whether you may submit SQL at all. It grants nothing on any table: each one still needs `data:table:read`, and a query naming a table you cannot read fails as though it were not there. The rewritten SQL comes back with the rows, always. Queries outside the supported subset are refused rather than run unguarded — a statement the planner cannot resolve against the tables it names is one whose column entitlements it cannot claim to have applied.

Body

SqlQueryRequest

Rewrite submitted SQL, show what it would read and what it would cost.

Same authorization and the same rewrite, no execution and no meter. Returns the SQL that would run and, per table, the obligations that shaped it and the columns policy withheld. The cost preview comes back with it. `bytes_scanned` is an upper bound rather than a guess — a preview exists to stop an accidental petabyte, and an estimate that sometimes comes in under is no use for that — with `floor_bytes` beside it for what a well-pruned run would read. Both are computed from published metadata and the query's shape, never by evaluating its predicates: a preview that counted matching rows would be an unaudited oracle over data the caller may not read. The estimate is per principal, like the plan. Columns policy denies you are not in the relation, so they are not in the estimate, and the same SQL previews cheaper for someone who may read less of the table.

Body

SqlPlanRequest

One query from the history, with its plan and profile.

The id is the share link. Sending somebody a query id is how a colleague sees what you ran, what it read and what it cost, without a second object with its own lifetime and its own revocation. The plan is the one that ran, stored at the time, not one recomputed now. Recomputing would plan it against today's policy and today's schema and quietly answer a different question. The profile splits the elapsed time into `planned_ms`, `waited_ms` and `executed_ms`. They have different causes and different fixes: planning is the authorization round trips, waiting is capacity, and only the third is the query being slow. A single elapsed figure makes all three look like the same complaint.

Stop a query, whether it is running or still waiting for a slot.

Answers with which of the two it was, because “cancelled” without saying whether anything had started is not something an operator can act on. Cancelling your own query needs `data:query:cancel`. Cancelling somebody else's also needs `data:query:manage`, so that an analyst can stop their own runaway query without being able to clear the queue during a colleague's deadline. A cancelled query is recorded in the history as `killed`, with who stopped it and what it had cost by then. The engine is asked to stop and the caller is freed immediately; the queue slot is held until the engine actually finishes, because releasing it earlier would admit the next query while the last one is still running.

Ask, and get the answer with the SQL that produced it.

This endpoint takes a question and never a statement. It compiles the question again rather than accepting the SQL `questions:plan` returned, which is what makes “the generated SQL is always shown” a guarantee instead of a gesture — compilation is deterministic, so there is nothing to substitute and nowhere to put it. Accepting a reviewed statement back would also be the submitted-query endpoint rebuilt behind a permission chosen for a chat box. The statement runs through the ordinary path, so it is guarded, priced, queued and recorded in the query history like a hand-written one. A question can therefore be refused after being compiled, and the refusal still shows what would have run.

Body

AskQuestionRequest

Turn a question into SQL and plan it. Nothing runs.

The question is resolved against the vocabulary — it cannot name a table, a column or a predicate, only things somebody defined — and a word that resolves to nothing is refused with the vocabulary listed. Nothing is silently dropped: “revenue by widget” with no `widget` defined is refused rather than answered as “revenue”, because an answer to part of a question is indistinguishable on the page from an answer to the question. There is no model in this path and no call to one. Resolution is a deterministic walk of a lexicon, which is inspectable, identical on every run, and cannot invent a metric that does not exist. It cannot understand a sentence either, and it says so rather than guessing. Planning here is the permission check. What comes back is the planner's own disclosure: every table the statement touches, the columns it will expose to you and the obligations it applied.

Body

PlanQuestionRequest

What is running, what is waiting, and under which limits.

Capacity is per tenant, and each queue has a share of it. A query runs when both have room; when a slot frees, the waiting query with the highest effective priority takes it. Effective priority rises the longer a query waits. Priority without aging is starvation with extra steps — a batch query behind a steady drip of interactive ones never runs, and the queue depth looks healthy the whole time.

Set a queue's priority, concurrency and limits.

Creates the queue if it is new. Raising a limit takes effect at once and admits anything already waiting, because an operator widening a queue during an incident is doing it precisely because things are stuck. `max_bytes_scanned` and `max_runtime_seconds` are ceilings a policy may tighten and never raise. A `query_budget` obligation on a principal's decision narrows what they may spend here; it cannot widen it, or a configured limit would be advice rather than a limit. `pool` binds the queue to a compute pool. From then on the pool owns `max_concurrency` and overwrites it on every resize, including down to zero while the pool is asleep — submissions wait rather than being refused, and the board says which pool they are waiting for.

Body

ConfigureQueueRequest

List registered RAG pipelines.

Register a RAG pipeline.

Pipelines orchestrate retrieval from a corpus and generation with an LLM, making the RAG workflow reproducible and auditable. The retrieval strategy and LLM parameters are fixed per pipeline.

Body

RegisterRAGPipelineRequest

Get a RAG pipeline by name.

Execute a RAG pipeline with a query.

Retrieves relevant chunks from the corpus and generates a response with the LLM. Returns the generated response, retrieved chunks, and metadata.

Body

ExecuteRAGPipelineRequest

External feeds, live and closed, without their tokens.

Closed ones are included: a review of what leaves the organization needs to know what used to, and every entry names the principal it renders as.

Grant an outside organization a feed of named shares.

The most consequential delegation in the product, and the ceremony matches. A written reason, a named organization, and a term of at most ninety days — shorter than an internal share on purpose, because the counterparty is outside the company and nobody here will notice them quietly still consuming it. **Reads run as you.** The recipient is not a principal — there is no entry for them in ProjectX ID and nothing for the policy decision point to reason about — so there is nobody to render as. Your own access is re-checked on every read through the feed, not just now, because a feed that outlived the access it was made from has to stop on its own. The token comes back once and is stored only as a digest: a leak of the recipient list must not be a leak of every live feed.

Body

GrantFeedRequest

Revoke a feed. Dead on the next read.

Read on the request rather than swept, and the reason is stronger than it is internally: there is no colleague outside the company to ask to stop using it. Revoking twice succeeds.

One feed: who holds it, what it reaches, until when.

Set the tables a named share lends.

Separate from granting, so the contents of a feed can change without reissuing anybody's token — and so a token is not a claim on one table list forever. Every table is checked against your own access as it is added.

Body

DefineFeedShareRequest

The shares a token reaches.

The protocol surface, reached by an outside organization holding a guest principal with `data:recipient:consume` and nothing else. The PDP is still asked — invariant 2 admits no exceptions — but it cannot be asked anything useful about a caller who has no identity here, so the token in the body decides which tenant this is and which shares exist. **Built from the recipient's own grants, never from the tenant's catalogue.** That is what stops this becoming a directory of the warehouse. An unknown token and a revoked one get the same answer, because anything finer is a way to test tokens.

Body

DeltaSharingTokenRequest

Schemas inside one share.

A share the token does not reach and a share that does not exist return the same 404, for the reason every other 404 in this platform does: the alternative is an oracle.

Body

DeltaSharingTokenRequest

Tables inside one schema of one share.

Addressed by schema and name rather than by our dotted qualified form, which is translated at the edge so it stays out of a wire contract we do not own.

Body

DeltaSharingTokenRequest

Authorize one read through a feed.

The producer's access is re-checked here, on every read, and not only when the feed was granted. Nobody outside the organization will tell us they are still consuming something the producer lost, so a feed that outlived its access has to stop by itself. Every read is audited against the feed and names the organization it was served to — the chain would otherwise record only the producer, with nothing saying the rows left the company.

Body

DeltaSharingTokenRequest

Repository bindings, with their branch and last synced commit.

Bind a Git remote this tenant's source can be kept in.

Credentials are named, never sent. This takes the name of a credential the deployment resolves; a token posted to a configuration API ends up in an audit log, a backup and a support ticket, and a request carrying one is refused rather than quietly accepted with the secret dropped. Remotes must be `https://` or `git@`. A binding that accepted a local path would be an arbitrary-file-read primitive wearing a repository binding.

Body

BindRepositoryRequest

List the rerankers a pipeline may pick from.

Register a cross-encoder a query may rerank with.

A cross-encoder scores the query and a passage together, catching relevance that independently-encoded vectors lose (DATA-GAI-008). It is accurate and slow, so it reranks what the cheap retriever proposed rather than replacing it. A sovereign tenant may register self-hosted rerankers only, because a reranker sees the query and the retrieved passages.

Body

RegisterRerankerRequest

List this tenant's retention policies.

Define how long something is kept, and what happens after.

`duration_days` and a `basis` of `created` or `last_written`, which defaults to the latter — most obligations are about the age of the data, and a table written to every day is not seven years stale because it was created seven years ago. `action` defaults to `review`, which reports and changes nothing. `archive` moves the data to the coldest tier. `delete` destroys it. The default is the harmless one deliberately: a retention system that deletes by default is how an estate is lost on the afternoon somebody types 30 where they meant 30 years.

Body

CreateRetentionPolicyRequest

Delete a policy nothing is bound to.

409 naming what still points at it. Deleting a bound policy would silently un-govern every one of those resources while they carry on displaying its name.

A policy, everything it governs, and what it is about to do.

Each binding carries its own expiry date and whether it is due, computed and returned without acting on any of it. Somebody turning on a 90-day deletion policy over four years of history deserves to read that sentence before they agree to it rather than afterwards.

Change a policy that is already in force.

Shortening the duration or hardening the action brings destruction forward for everything bound to it, so both are refused while a legal hold is on — not because the change is wrong, but because it is the change somebody makes by accident during the week the lawyers are watching.

Body

AmendRetentionPolicyRequest

Place or lift a legal hold over everything this policy governs.

A hold beats a deletion schedule, always and with no override. While it is on: the sweep skips and records the skip, dropping a bound table is refused, deleting rows from one is refused, and expiring its snapshots is refused. Those are the only ways this service can lose data and all four go through one gate, so a hold cannot be walked around with a `DELETE WHERE true`. `reason` is required in both directions and is kept on the policy. A hold that can be lifted silently is a hold that will be.

Body

SetLegalHoldRequest

List this tenant's batch inference jobs.

Run a batch inference job: score a dataset with a model version.

Batch inference scores a dataset all at once and writes the predictions back — the nightly re-score, not the one-request-at-a-time endpoint. The job names the model version it scores with, which must exist, because scoring against a version nobody registered produces predictions nobody can trace. The output table defaults to the input suffixed, so a job that names only its input still writes somewhere predictable. The tested runtime scores in-process and reports the row count; the real one submits to Spark or Ray.

Body

RunBatchInferenceRequest

Describe one batch inference job and its result.

List this tenant's serving endpoints.

Stand a model version up behind a real-time endpoint.

An endpoint answers one request at a time, and what makes it a platform feature is autoscaling: it holds a replica band and a target load per replica, and the replica count tracks demand between those bounds — up when a spike arrives so latency holds, down when it passes so the tenant is not billed for idle GPUs. It starts at the floor, warm the moment it exists. The version it serves must exist.

Body

RegisterServingEndpointRequest

Describe one serving endpoint, its band and its replicas.

Roll a version onto an endpoint: blue-green, canary, shadow or A/B.

Which strategy is a judgement about risk. Blue-green cuts all traffic over at once. Canary sends a slice to the new version and the rest to the current, so a regression shows on a fraction before it is on all of it — and it needs a single current version as its baseline, so a canary onto an endpoint that is already mid-rollout is refused. Shadow sends the new version a copy of every request and serves none of its answers, comparing it against production at zero risk. A/B splits by weights that must sum to 100. Every version the resulting split references must exist, because routing traffic to a version nobody registered would 404 at serving time. The live weights always sum to 100, so no request falls through a gap.

Body

DeployVersionRequest

Scale an endpoint to the replicas its observed load calls for.

The caller reports the observed concurrency and the autoscale band decides the replica count, so a scale request cannot ask for more than the ceiling or fewer than the floor no matter what load it claims — the autoscaler's decision made explicit and auditable rather than a raw replica count a caller sets by hand.

Body

ScaleEndpointRequest

Live and revoked shares, without their tokens.

Filter by `?dashboard=`. Delegated shares name the principal they render as on every representation, because a delegation that is not obvious in this list is one nobody revokes.

Open a share by its token and render what it points at.

The token in the body is the credential. A revoked or expired one is refused here rather than by a sweep, and an unknown token gets the same answer as a closed one so this endpoint cannot be used to learn which tokens exist. An external recipient reaches this with a guest principal holding `data:share:open` and nothing else — the token decides what they see, not their entitlements, which is exactly why the ceiling and the expiry are where they are.

Body

OpenShareRequest

Revoke a share. Dead on the next request.

Nothing is deleted and no cache is cleared, which is what makes this immediate rather than eventually-immediate: the open path reads the revocation. A revocation implemented by tidying up is widest open exactly when the tidying is behind.

One share: who made it, for whom, until when, opened how often.

List the remote relations registered in this tenant.

Register a remote relation under a name in this tenant.

The caller names it the way they name any table — `ext.orders` — and the connector address is held here rather than written into anybody's SQL. That is what lets a federated table go through the same guard, the same row filters and the same masks as a native one. The schema is declared, not discovered: nothing here scans a Postgres table for card numbers, so a column that was not declared is a column nothing can mask and therefore one nothing can read. `scan_bytes` is required for the same kind of reason — cost estimation has no manifests to read, and a table that estimates at zero passes every budget gate there is.

Body

RegisterFederatedTableRequest

Deregister one remote relation. Nothing at the source is dropped.

Describe a registered remote relation.

List the federated sources registered in this tenant.

Register a federated source: Postgres, MySQL, Delta Lake or Hive.

Declares that the query engine has a connector mounted, and names what may be read through it. It does not provision one — the catalog is deployment configuration, and a source may not be pointed at the platform's own catalog, which would be a way to address a governed table with a schema of the registrant's choosing. Everything reached through a source is read-only. The platform's promises about a table — retention, erasure, encryption, snapshot lineage — are promises about storage it owns, and accepting a write here would extend them over storage nobody here controls. Delta support is read compatibility for exactly that reason.

Body

RegisterSourceRequest

Deregister a source and every table registered against it.

Nothing at the far end is touched. This removes our names for somebody else's rows; the rows carry on existing.

Describe a federated source.

List this tenant's stream subjects.

Register a stream subject.

A subject is one logical stream and the unit a schema is versioned against. `compatibility` defaults to `backward`, which is the mode for a stream whose consumers are upgraded before its producers — most of them. Registered in the unified catalog like any other resource. A topic whose owner is discovered by asking around is not a governed one.

Body

CreateSubjectRequest

Remove a subject and its schema history.

The history goes with it, which is why this is destructive and separate from everything else a producer does.

Describe a subject and its latest registered schema.

Change what future versions are allowed to break.

`backward` lets a consumer on the new schema read old data: remove a field, add an optional one. `forward` lets a consumer on the old schema read new data: add a field, remove an optional one. `full` is both. `none` checks nothing and is recorded on the subject, so a production stream running unchecked is visible rather than discovered. Nothing is re-checked retroactively. Tightening the mode on a subject whose history already violates it constrains the next version, which is exactly what somebody should be able to do on finding a stream set to `none`.

Body

SetCompatibilityRequest

Hold a batch of records to the registered shape.

Contract enforcement on ingest. A registry that checks schema changes and not records documents the shape a producer promised while the producer sends whatever it likes. A field the schema does not have is a failure naming it, not something to ignore: a producer that renamed `user_id` to `userId` is sending records this can prove are wrong. Failures are reported as counts and field names with the index of the first bad record. Never the record.

Body

ValidateRecordsRequest

Register a schema version, if it is compatible.

409 with the offending fields named when the change is not allowed under the subject's compatibility mode. Re-registering the schema that is already current answers 200 with the existing version rather than minting a new one. A producer does this on every deploy, and a version number that counts deployments is a version number nobody can reason about.

Body

SubjectSchemaRequest

Would this schema be accepted? Registers nothing.

What a producer's CI calls. The value of a registry is that the answer arrives before the deploy rather than during it.

Body

SubjectSchemaRequest

Every sync in this tenant, and what left through it.

Send a governed table's rows back into an operational system.

Reverse ETL is usually built as ingestion pointed the other way, and that framing is the mistake. A connector writes into a table this platform governs, so the question is whether the writer may write. A sync *reads* a governed table and puts the answer somewhere that governs nothing — so it is the read question, and it is asked the way every other read is. A run is an ordinary query executed as the sync's owner: their row filters narrow what leaves, the scan is billed, and the statement appears in the query history beside the ones people typed. Rows the policy held back are reported as `withheld`, because a push that delivers forty of a hundred and says nothing looks exactly like a destination that lost sixty. A **masked** column stops the run instead of narrowing it. A mask works on a screen because the reader can see the value is redacted; the same value in a Salesforce field called `card_number` is a card number to everything downstream, and the mask has laundered itself into data. `destination_classification` is a ceiling over every mapped column. Nobody exports a card number on purpose — this is for the sync written last year against a table that has since grown a `pii` column. Encrypted columns are refused outright: the point of holding the key is that a copy elsewhere is not readable. `key` is required. Delivery is at-least-once, so an unkeyed retry creates the record a second time, and the customer-facing symptom of that is two emails.

Body

SyncDefinitionRequest

Stop the export. What already left is already gone.

The one asymmetry with deleting a connector. That leaves rows in a table this platform still governs; this leaves rows in a system it never did, and no endpoint here can recall them.

One sync: how far through the table it is and what came back.

`rejected` and `rejections` are the destination's complaints, counted and quoted in its own words — never the rows, which are the tenant's governed data and this is an audit surface. Partial acceptance is normal rather than exceptional: a destination validates per record, so a batch of two hundred comes back as a hundred and ninety-six and four picklist arguments.

Replace the definition, keeping the mark.

The source and the watermark cannot change. A mark is a value of one column of one table, and carried to another it skips everything below an arbitrary number — which, for an export, looks precisely like a destination that is already up to date.

Body

SyncDefinitionRequest

Send one batch now.

Synchronous, and bounded by `batch_size`. These rows leave over somebody else's API under their rate limit rather than ours, which is why the ceiling is lower than an ingest batch.

List the tables in a namespace.

Create an Iceberg table.

Registered in the unified catalog as part of the same call, with its owner and a classification derived from the strictest of its columns. A table that exists but is not in the catalog is invisible to everyone who searches for it, so the two are not separate steps.

Body

CreateTableRequest

Drop a table and remove it from the catalog.

Describe a table: schema, partitioning, snapshots and size.

Open a branch at a snapshot.

A branch is a movable pointer. This one is readable with `?ref=name` and is a valid rollback target; writes still go to `main`, because the engine this service delegates writes to has no way to address another branch. Write-audit-publish arrives with the pipeline runtime that needs it.

Body

CreateRefRequest

Sample the columns and look for personal data.

Every finding carries the rule that fired, how many values were sampled and how many matched. `email` is not a verdict; "94 of 100 sampled values are addresses and the column is called contact" is evidence somebody can act on. Send `apply: true` to write the labels. Applying can only tighten: a finding below the confidence floor, or one that would loosen a classification a person declared, is reported and left alone. The rule id is recorded on the column, so a later review can tell the labels a human stands behind from the ones a sampler guessed. Encrypted columns are skipped — their stored values are ciphertext, and decrypting a table's worth of personal data to look for personal data would be the wrong trade.

Body

ScanClassificationRequest

Describe a column, classify it, or link it to glossary terms.

Changes no data and breaks no reader, which is why it does not need the permission that can drop a column. Documentation that requires a destructive grant is documentation nobody writes. Send `classification: null` to remove one. Curating by hand clears the detection provenance: once a person has said what the column is, the label is theirs and the next scan will not overwrite it. A table's classification is the strictest of its columns, so this can change what the whole table is — the response is the table, and the audit event carries both the old value and the new.

Body

CurateColumnRequest

The active data contract, every version of it, and freshness.

Versions are kept rather than replaced. A contract consumers built against is evidence of what they were promised, and deleting it makes “you were never promised that” unfalsifiable.

Publish a new version of the producer's commitments.

Each expectation names a column and one or more of `required`, `allowed`, `pattern`, `minimum` and `maximum`. `freshness_seconds` is checked by whoever asks rather than on write, because a freshness promise is broken by the absence of a write and there is no request to refuse. `enforcement: block` refuses a write that breaks the contract, naming the column, the rule and how many rows failed — never a value. `warn` allows it and records the breach on the audit event, which is what makes turning a contract on over a live pipeline a safe thing to do. The version number is assigned here, not sent: two teams publishing “v2” of one contract is exactly the confusion versioning exists to prevent.

Body

PublishContractRequest

Deprecate a contract version.

Nothing is deleted. The version is marked deprecated, stops being enforced, and stays readable as the record of what was promised.

Compact, rewrite manifests, expire snapshots or remove orphans.

A lakehouse degrades on its own: streaming writes leave thousands of small files, every write leaves a snapshot, and a failed write leaves files nothing points at. None of that shows up in a query result — it shows up in the latency six months later and in a storage bill nobody can explain. `compact` and `rewrite_manifests` lose nothing: the rows are the same and every older snapshot still resolves. `expire_snapshots` deletes history and `remove_orphan_files` deletes files, so both enforce a retention floor. The orphan floor is three days because a file no metadata points at looks exactly like a file a write is still producing. Set `projectx_maintenance` on the table to `compact` or `full` to have the sweep run these without being asked.

Body

TableMaintenanceRequest

Replace the partition spec.

The whole spec, not a diff, so that changing it requires having looked at it. Iceberg applies a new spec to new data only. Files already written keep the layout they were written with and stay readable, so this rewrites nothing and takes effect from the next write — which also means a query spanning the change reads two layouts, and is correct rather than fast.

Body

EvolvePartitioningRequest

Run a query and return the rows.

The request describes what is wanted — columns, filters, a limit — and the platform writes the SQL. The generated SQL and the list of policy obligations applied to it come back alongside the rows, because a platform that filters your query and will not show you the query it ran is asking to be trusted about the one thing it should be proving. `filters` is a list of comparisons, all of which must hold. A predicate that needs an OR, a NOT or nesting goes in `expression` instead, written in the same language a policy row filter uses; supplying both means both.

Body

TableQueryRequest

Compile the query and return the plan without running it.

Same authorization, same obligations, no execution and no meter. What an analyst uses to find out which columns a policy will mask before spending anything to find out.

Body

TableQueryRequest

List the table's branches and tags.

`main` is always among them. It is the branch a write goes to, and the snapshot it points at is the table's current state — which is not the newest snapshot once the table has been rolled back.

Remove a branch or tag.

Removes the name, not the snapshot. `main` cannot be removed: a table without it has no current state.

Ask for readable copies of archived partitions.

Answers with when the data will be readable, not with the data. Archival storage measures a restore in hours, so a synchronous version of this would be a request that holds a connection open for half a day and then times out. The restored copy is readable for a week and the partition stays archived underneath, which is what the storage actually does — a restore makes a copy, it does not move the object back. Query it in the meantime and the answer is a 409 naming the partition and the time it becomes readable.

Body

RestorePartitionsRequest

Bind a table to a retention policy, or unbind it.

Its own permission rather than part of altering the table. Binding a `delete` policy schedules the destruction of everything in here, which is a governance decision and not a schema one — and the person who tunes partitioning is not automatically the person who should be able to make it. Send `retention_policy: null` to unbind. A policy name that does not resolve is refused: a table claiming a policy nobody defined reads as governed in every list it appears in.

Body

SetTableRetentionRequest

Move the table back to an earlier snapshot.

Moves `main`. Nothing is deleted — the snapshots stepped over stay in the history until they expire, so a rollback that turns out to be the wrong call is itself reversible. Reads that do not ask to travel follow `main`, so this changes what the table returns and what the catalog reports it holds. Requires `data:table:alter` rather than `write`, because undoing a day of loads is not a kind of loading.

Body

RollbackTableRequest

Append rows, producing one snapshot.

A column the schema does not have is a 422 naming it, never a silently dropped field: a write that reports success having discarded data is the failure nobody notices until an audit.

Body

AppendRowsRequest

Delete the rows matching a condition.

`filters` and `expression` are the same two spellings the query route takes, so the condition that reviewed the rows is the condition that removes them. A request with neither is a 422. `DELETE` with no `WHERE` is the most expensive typo in the language, and reading it as "delete everything" decides on the caller's behalf that they meant it.

Body

DeleteRowsRequest

Apply a source set, choosing what happens on each side of the match.

`when_matched` is one of update, delete or nothing; `when_not_matched` is insert or nothing. The defaults are update and insert, which is an upsert. A source that names the same row twice is a 409. SQL refuses it too: two rows with one key means the caller has two answers and does not know it.

Body

MergeRowsRequest

Insert rows that are new and replace rows that are not.

Matches on the table's `schema.identity` columns, so a table that declared none is a 409 rather than a silent append — an event log has no such thing as the same row twice, and pretending otherwise is how a log acquires duplicates nobody can distinguish. An update replaces the whole row. A source row that omits a column sets it to null rather than leaving the old value, because the alternative means the same request does different things here and in SQL.

Body

UpsertRowsRequest

Add, drop, rename, reorder or widen columns.

`changes` is an ordered list and is applied in order, so dropping `email` and then renaming `email_v2` to `email` is one request. A rename keeps the column's Iceberg field id, which is why it costs nothing and why it changes what a read of an *older* snapshot returns: the data files carry the id, and the name was only ever metadata. A drop followed by an add of the same name gives you a different column. The new one gets a fresh id and reads null over the old data, because reusing the id would resurrect values somebody deleted. A widening is refused unless the new type can read every value the old one wrote. Narrowing a column is a table rebuild, and calling it an evolution would make it sound reversible.

Body

EvolveSchemaRequest

List the table's snapshots, newest first.

Tag a snapshot so it can be read back by name.

Body

TagSnapshotRequest

Row counts, null counts, distinct values and how stale they are.

A photograph with a date on it, not a live view. Computing statistics costs a scan, so they are computed on a schedule and the response says when — a number with no timestamp on it gets read as current forever. Freshness is computed live, from the snapshot history, and is the one figure here that is never stale. A table's data being 18 hours old and its statistics being 18 hours old are different problems. The bounds of a classified column are withheld and the response says so. The minimum and maximum of an `email` column are two customers' addresses, and the catalog is not where those go.

Recompute the statistics now.

Reads every row of the columns named, or of all of them. Needs `data:table:profile` rather than `read`, because the counts are over the whole table and not over the rows a row filter would have left the caller. The scheduled sweep does this unattended for tables whose numbers have gone stale, and skips tables nobody has written to since they were last profiled — their counts have not changed.

Body

RefreshStatisticsRequest

Name a snapshot, permanently.

A tag does not move. `q3-close` means the same rows next year as it does today, which is what makes it usable as evidence. Read it back with `?ref=q3-close` on a query. Defaults to the current snapshot, because naming where you are is the common case and stating the id invites pasting a stale one.

Body

CreateRefRequest

Where each partition of this table is stored, and what is due to move.

One entry per partition: which tier it is in, how much it holds, the newest row in it, and whether a query can read it right now. `due` is what the next sweep would move given the policy and the ages as they are. It is the answer to the question anybody has when they read a tiering policy, which is what it is about to do.

Set the age-based tiering policy, as `warm:30,cold:90,archive:365`.

Days are counted from the newest row in a partition, not from the date in its name — a partition backfilled a year late holds fresh data, and archiving it on its name would put this morning's rows twelve hours out of reach. Thresholds must increase. Omit a tier to never move to it, which is what lets a policy be adopted a step at a time. Setting a policy moves nothing by itself; the sweep does that. Refused on a table that is not partitioned by year, month, day or hour, because such a table has no part that is older than another part and the first move would take all of it offline. Send an empty `tiering` to turn it off. Nothing comes back up: partitions already moved stay where they are until restored.

Body

SetTieringRequest

Move everything the policy says is due, now.

The same work the scheduled sweep does, for a table somebody does not want to wait for. Idempotent: a partition already in the tier the policy wants is not moved again.

Move one named partition to a colder tier.

For the partition a policy does not cover — an export nobody will read again, a backfill that arrived cold. Downwards only. Moving data back up is a restore, which is a different endpoint because it costs money and takes hours.

Body

MovePartitionRequest

Read a view.

Addressed at the base table on purpose. The framework authorizes the resource in the path, so this call is authorized as a read of the table and the obligations that come back are the table's row filters and column masks — which is exactly what a view must not be able to sidestep. Needs `data:table:read`, therefore. A view is a convenience over a table you can already read, not a grant of one you cannot. A view whose definition names a different table than the path is a 422 rather than a redirect. The body takes a `purpose` and a smaller `limit`, and nothing else: a view read that accepted `filters` or `select` would be a query with a view attached, and the definition would stop being the thing that was reviewed. The bytes are metered against the table. If the view is materialized the rows may come from the copy, and the reply always says which — `served_from` is `materialization` or `table`, with a reason. That is not decoration: a materialization can be unusable for one reader and fine for another, and a silent fallback is a performance cliff nobody can see. `max_staleness` says how old an answer you will accept, in seconds; past it, the table is read instead.

Body

ViewQueryRequest

Show the SQL a view read would run, and what policy applied.

Executes nothing and costs nothing. The honest answer to “what does this view actually select for me” is the compiled plan, masks and row filters included.

Body

ViewQueryRequest

List this tenant's training jobs.

Submit a distributed training job.

The job names its framework (Spark for data-parallel, Ray for task-parallel), the experiment its run is logged under, an entrypoint, its hyperparameters and the resources it needs. GPUs are scheduled against the cluster's declared capacity: a job asking for more than the cluster has is refused now, because a request that can never be placed is a mistake and saying so is kinder than a job that is pending until someone asks why. When it finishes it logs a run in its experiment — the same record a hand-logged run is, and one a model version can cite the same way. The metric comes from executing the job, never from the submitter: a training job whose score the caller could set is not a training job.

Body

SubmitTrainingJobRequest

Describe one training job, including the run it produced.

List this tenant's hyperparameter sweeps.

Run a hyperparameter sweep: many trials, the best one chosen.

A sweep is a training job repeated over a search space, one trial per point, with the winner picked by an objective. Grid enumerates the product of enumerated choices and refuses a continuous range and a product past the trial ceiling — silently trying a third of a grid and reporting a best is a lie. Random and bayesian sample the space up to a budget; the tested backend samples deterministically from a seed so a sweep is reproducible, and the real bayesian backend samples where the trials so far suggest the objective is best. Every trial is a real training job — same GPU admission, same run logging — so the winner is the best of what actually ran, and a trial that failed or never reported the objective metric is not a candidate.

Body

RunSweepRequest

Describe one sweep, its trials and the winning trial.

List this tenant's uploads and their status.

Begin a file upload and get a presigned PUT to the tenant's prefix.

The first step of turning a dropped file into a governed table. It returns an upload id and a short-lived URL to PUT the bytes to; the URL is to the tenant's own prefix and is the only door into that key, which is why it is returned here and never again. In a real deployment the client PUTs the file and then calls `inspect`, which sniffs the uploaded object. In this environment the upload is simulated — no bytes are transferred — so `inspect` reads a `sample` the caller hands through instead. The contract is the same either way: the schema is proposed, then confirmed by a person, so a table made this way is declared and not sniffed.

Body

BeginUploadRequest

Check one upload's status and proposed schema.

Never the presigned URL: it is a credential returned once from the begin call, and a status read that leaked it would widen the one window the upload was meant to be.

Import the file into a new or existing table.

Takes the confirmed schema — the one the person edited — and lands the rows. `new` creates the table under the namespace with that schema; `append` adds to a table that already exists, using the table's own schema. Either way the create and the write go through the ordinary table paths, so the result is a governed table like any other: catalogued, classified and snapshotted, and queryable in the editor.

Body

ImportUploadRequest

Sniff the file and propose a schema.

CSV and JSON lines are inferred from a `sample` — the header names the columns, the first rows type them, and the inference is deliberately cautious: a column that is mostly but not entirely numbers is proposed as `string`, because the first bad row of a numeric column is a silent cast. Parquet carries its schema in its footer, which there is no parser for here, so a `columns` hint is accepted for it instead. The proposal is a starting point. The person edits it and confirms it at import, which is what keeps the table declared rather than sniffed.

Body

InspectUploadRequest

Every view defined over this table.

The question a schema change has to answer before it lands. Dropping a column that three views select is three broken saved queries, and finding that out here is the difference between a warning and an incident.

List this tenant's views.

Define a view: a saved query over one table.

A view here is not stored SQL. It is a column list, an optional predicate and a limit over exactly one base table, and reading it compiles the same plan a caller would have got by writing the query themselves. That is what stops a view being a way around policy. Its predicate is conjoined with the reader's row filters rather than replacing them, and its columns are projected from what the reader may see — so a view can only ever return a subset of what its reader could already have selected. There is no definer's-rights mode, because there is no definer to resolve against. The classification is derived from the columns the view actually projects, not from the whole table: a view selecting `order_id` and `region` out of a table holding card numbers is not `pci`, and labelling it so would train people to ignore the label. No joins, no aggregates, no unions. Those need a SQL surface with an authorization story of its own, which is Phase 05's.

Body

CreateViewRequest

Drop a view. Touches no data.

The definition goes and the table underneath is untouched.

Describe a view: its definition, its base table and its owner.

Metadata only. This returns no rows, and nor does any other endpoint whose resource is a view — reading one is addressed at the base table, so that the decision the framework obtains is the decision the read needs.

Change a view's definition, description, owner or retention.

Re-checked against the base table exactly as a creation is, and the classification is recomputed. A redefinition that adds a `pii` column has made a `pii` view, and the catalog has to say so before anybody queries it.

Body

RedefineViewRequest

Stop materializing, and drop the copy.

The table goes with the record. A materialized table left behind after its view stopped pointing at it is a copy of customer data that nothing refreshes, nothing reads, and nothing is watching the retention of.

Where a view's copy is, how old it is and how the last build went.

`age_seconds` rather than an adjective. Staleness is the trade a materialization makes, and whether it is acceptable depends on the question being asked, so the number is reported and the caller decides. `last_error` survives a failed refresh: the previous copy stays in place and gets older than it claims to be, and the age alone would not say why.

Keep this view's rows in a table, refreshed on a schedule.

Declared here and built by the refresh, which is a separate call because building it is a full scan and this is a configuration change — a create that quietly ran one would be a POST that takes four minutes and sometimes times out half done. **A copy is not exempt from the policy on the original.** Reading a materialization is authorized against the base table exactly as an ordinary view read is, and the reader's row filters and column masks are compiled into the plan over the copy. Materialization is a performance detail, never a permission one, and the alternative — a table with none of the base's filters attached — would make it a laundering step. Which is why a copy is sometimes unusable. A row filter is a predicate over columns, so a reader filtered on `region` cannot be served from a view that does not project `region` — there is nothing to apply the filter to. That read falls back to the table and says so rather than being refused, because one principal's entitlements should not break the feature for everybody. A view projecting every column cannot be materialized: that is a copy of the table wearing a different name, and doubling somebody's storage bill and their PII footprint on the strength of a default is not a thing to do quietly. The copy inherits the view's classification and retention, because a copy of PCI data is PCI data and a copy that outlives an erasure is a deletion that did not happen.

Body

MaterializeViewRequest

Rebuild the copy from the base table now.

The copy is read without obligations, because it is shared by every reader and has to hold the rows before anybody's filter. Building it filtered by whoever happened to trigger the refresh would bake one principal's entitlements into everybody's answer — the same bug as keying a result cache on submitted SQL. What makes it safe is that reading the copy re-applies the reader's own. Dropped and recreated rather than truncated and appended, because a view's projection can change and a table whose schema has drifted from the definition that fills it fails on the next refresh with a message about a column nobody edited.