Generate Code With pg-sourcerer
This guide shows how to generate TypeScript from a PostgreSQL database. For the design, read ARCHITECTURE.md.
Terms
- Introspection file: a JSON file that holds the catalog of one database. It is the output of the query from
@danielfgray/pg-introspection. - Visitor role: the database role that the generated code runs as. The generator produces only the queries that this role can run.
- Smart tag: a line in a PostgreSQL comment that starts with
@, or an entry of thetagssection of the config file. The generator reads the tags@omit,@name,@primaryKey,@foreignKey,@notNulland@brand. - Output root: the directory that receives the generated files.
- Generation: one complete set of generated files. The generator merges it into the output root and records the last pristine output as hashes in
.pg-sourcerer/manifest.json. - Output: a document emitter. The
effectandkyselyoutputs emit code. - Plugin: a name, the plugins it requires, the services it holds and an optional output. You list plugins in the config; the generator also runs the plugins they require.
Requirements
- Bun.
- A PostgreSQL database. The visitor role must exist in that database.
- An output root that sits inside a project that installs the packages the generated code imports:
effectversion 4 for theeffectplugin.kyselyfor thekyselyplugin.
The generator typechecks every generated file before it publishes the files. It resolves each package import from the directory tree above the output root. If the output root has no installed effect version 4, the run fails with SnapshotValidationError and publishes nothing. packages/generated-check/package.json is an example of a project that installs both packages.
Step 1: Make an introspection file
Run these commands from packages/pg-sourcerer. The first command writes the introspection query to a file. The second command runs it.
bun -e 'import { makeIntrospectionQuery } from "@danielfgray/pg-introspection"; process.stdout.write(makeIntrospectionQuery())' > introspection.sql
psql -X -q -t -A -v ON_ERROR_STOP=1 -U <owner-role> -d <database> -f introspection.sql > introspection.json
Replace <owner-role> and <database> with your values. Run the query as the database owner role. The fixture script (scripts/introspection-fixture.ts) does the same.
The generator accepts two input shapes:
- The introspection output. The file has a top-level
classeskey.src/pg-source.tsconverts it. - A
CatalogSnapshotobject.src/domain.tsdefines the shape. It hastables,relations,enums,rolesandmemberships.
To make the demo introspection file, run bun run fixture:introspection. This command needs Docker. It writes examples/introspection.json.
Step 2: Generate the code
With a config file
Put a pgsourcerer.config.ts file in your project. The CLI looks for it in the current directory and in every parent directory, and uses the nearest one.
import { defineConfig, fileSource, webApp } from "pg-sourcerer"
export default defineConfig({
source: fileSource("./introspection.json"),
visitorRole: "visitor",
outputDir: "./src/db",
plugins: [webApp()],
format: ["bunx", "oxfmt"],
})
Then run bun run src/cli.ts from packages/pg-sourcerer, or from any directory below the config file.
The fields are:
source:fileSource(path)for an introspection file orCatalogSnapshotfile, orconnectionSource(url)for a live database. The connection URL is kept as aRedactedvalue, so logs do not show it.visitorRole: the role name. The role must exist in the catalog.outputDir: the output root. See the rules for the output root below.plugins: a list of plugins.effect(),kysely()andsuggestionReport()each build one. A plugin also runs the plugins it requires, so list only the ones you want generated.webApp()is a plugin that requireseffect()andkysely(). A preset is a plugin.format(optional): a formatter command as an argv list (program first, then its arguments). The generator appends the path of a directory that holds the generated files, and runs it. The files in that directory must be formatted in place. The generator runs the formatter before validation and merging. It does not format the merged text.
Relative paths in the config file resolve against the directory of the config file.
The CLI imports the config file with a dynamic import. It fails with ConfigLoadError when the file throws while it loads, has no default export, or holds an invalid value. The data fields are checked with a Schema. Each entry of plugins must be a Plugin.
With flags
A flag replaces the config field with the same meaning. With no config file, the flags work alone.
bun run src/cli.ts ../my-app/src/db visitor --input ./introspection.json --plugins effect,kysely
The arguments are:
- The output root (optional when the config sets
outputDir). Relative paths resolve against the current directory. You can edit generated files and keep other files in this directory. Choose a directory inside a project that installseffectversion 4 (andkysely, if you use that plugin). See Requirements. - The visitor role name (optional when the config sets
visitorRole). The role must exist in the catalog. --input, the introspection file orCatalogSnapshotfile.--connectiontakes a connection URL instead. Pass one of them, not both. With no config and neither flag, the CLI readsDATABASE_URL.--plugins, a comma-separated list of output names. The default iseffect. The known names areeffect,kysely,types,zod,valibot,arktype,typiaandsuggestion-report. The CLI fails when it gets an unknown name or the same name twice.
The CLI reports the number of files written, created and deleted. It lists each merge conflict and exits with code 1 when conflicts exist.
Output layout
The output root holds these files and folders:
generated/
app_public/users/schemas.ts editable generated code
app_public/users/queries.ts
.pg-sourcerer/
manifest.json hashes and import names from the last run
Import files directly from the output root. Commit .pg-sourcerer/manifest.json with your code. Each run merges your edits with the new output. Resolve conflict markers before the next run. See Editable generated code for the merge rules.
With --plugins effect, the generation holds these files. Each table gets its own folder, <schema>/<table>/, that holds the files of its schemas, types and queries, for example app_public/users/schemas.ts. The shared files (errors.ts, rpc.ts, http.ts) stay at the root. The byKind() plugin puts all tables in one file for each kind instead (see "Write a layout"). The list names each file without its folder:
schemas.tsqueries.tserrors.tsrpc.tshttp.ts
When kysely is selected, the generation also holds:
kysely/db.tskysely/queries.ts
Step 3: Tag the schema with smart tags
Smart tags are PostgreSQL comments. Write them with COMMENT ON. Each tag must start its own line.
comment on table app_public.comments is '@name Reply';
comment on column app_public.comments.id is '@omit update,delete';
The generator reads the comment from the catalog. It does not read the database at generation time. Run the introspection query again after you add a tag.
Tags in the config file
Every tag also has a config form, so a schema you cannot comment on can still be tagged. The tags section of defineConfig follows PostGraphile's tags file. Rules are grouped by what they tag, and a key is name or schema.name:
defineConfig({
// ...
tags: {
class: { // tables and views
"app_public.comments": {
description: "A reply to a post.", // replaces the comment's prose
tags: { name: "Reply", omit: "delete" },
attribute: { id: { tags: { omit: "update" } } }, // columns of this table
},
},
attribute: { "comments.body": { tags: { notNull: true } } }, // table.column or schema.table.column
procedure: { "app_public.search_posts": { tags: { omit: true } } }, // every overload of the name
type: { "app_public.ticket_number": { tags: { brand: false } } }, // domains
},
})
A tag value is the text that follows the tag in a comment: omit: "read, create" is @omit read, create. true is the bare tag, false is the tag followed by false, and a list is one tag line per item (foreignKey: ["(a) references t", "(b) references u"]). One parser reads both forms, so the same values are valid in both.
The config wins when both set a tag. A rule's tags replace the comment's tags of the same name and keep the other comment tags. A rule's description replaces the prose of the comment. A bare key applies to every schema; when a bare and a qualified key both match, the qualified one is applied last. A rule that matches no table, column, function or domain fails the run, as does an unknown tag name.
Tag reference
| Tag | On | Effect |
|---|---|---|
@omit | table | Removes every candidate on the table. |
@omit all | table | Same as @omit. |
@omit read,create,update,delete | table | Removes the candidates for the named operations. create is the insert operation. |
@omit update,delete | column | Removes the candidates whose key columns include this column, for the named operations. |
@name Reply | table | Sets the entity type name to Reply, exactly as written: the entityTypeName inflector does not run. Names derived from it (query names, request names, group names, shape names) run their own inflectors over Reply. The URL path keeps the table name. |
@primaryKey id | view | Declares the key columns of a view. A table has constraints, so the tag on a table fails the run. |
@foreignKey (a) references [schema.]t [(x)] | view | Declares a foreign key of a view; one line for each key. |
@notNull | column | Asserts a view column is never null. @notNull false clears it. |
@brand, @brand false | domain | Makes the domain a nominal type, or keeps it plain, whatever domains.brand says. |
@omit, @name Name | function | @omit leaves the function out; @name names its query exactly as written. |
@name thread | column | Sets the field name to thread, exactly as written: the fieldName inflector does not run. The property in every shape and the parameter of every query that uses the column take it. Query names and path segments for keys on this column are derived from it. |
The generator fails when a tag is wrong. These cases fail:
- An unknown operation in
@omit, for example@omit reed. The run fails with aSmartTagErrorbefore any query is selected. - A
@nametag with no name, or with more than one name. - A
@nametag that appears twice on one comment. - A tag on an object it does not apply to, for example
@omiton a domain or@brandon a table. - Two selected queries that get the same name.
What a column tag does not do
A column tag does not hide the column. It does not remove the column from the result of a read. It does not remove the column from an insert or an update. It removes only the candidates that use the column as a key.
Example result
A probe run edited a copy of the demo introspection file. The table comment set @name Reply, and the id comment set @omit update,delete. The generated output had these results:
- The find and list names used
Replyin place ofComments. - The HTTP group was named for
Reply. - The URL paths kept the table name:
/comments/:id. - The update and delete queries for
idwere removed. - The find query for
idremained.
Step 4: Select queries with rules (library use)
The CLI passes no query rules. A rule adds or removes candidates by kind or by entity. Use the library function generateSnapshot from pg-sourcerer to pass rules. The function returns an Effect that you can run with Effect.runSync or Effect.result.
import { Effect } from "effect"
import { generateSnapshot } from "pg-sourcerer"
// `snapshot` is a CatalogSnapshot, for example from databaseSnapshotFromIntrospection.
const result = Effect.runSync(
Effect.result(
generateSnapshot(
snapshot,
"visitor",
[{ action: "exclude", kind: "delete-by-key" }],
["effect"],
),
),
)
A rule has these fields:
action:"include"or"exclude". Rules run in order. The last matching rule sets the state.kind(optional): one offind-by-key,list-by-foreign-key,list-keyset,list-by-index,insert,update-by-key,delete-by-key.entity(optional): an entity id such asapp_public.comments.
A smart tag runs after the rules. A tag @omit always removes its candidates, even when a rule includes them.
Step 5: Use the generated Effect code
HTTP routes
The http.ts file declares one HttpApiGroup for each table. The group name comes from the handlerGroupName inflector, which is the entity type name in camel case by default. Each candidate becomes one endpoint. The route path follows these rules (src/effect/http.ts):
| Candidate | Method | Path |
|---|---|---|
insert | POST | /<table> |
find-by-key, update-by-key, delete-by-key on the primary key | GET, PATCH, DELETE | /<table>/:<key> |
find-by-key, update-by-key, delete-by-key on another unique key | GET, PATCH, DELETE | /<table>/by-<columns>/:<key> |
list-by-foreign-key | GET | /<table>/by-<columns>/:<key> |
list-keyset | GET | /<table>/ordered-by-<columns> |
list-by-index | GET | /<table>/ordered-by-<columns>, or /<table>/by-<columns>/:<key>/ordered-by-<columns> when a prefix filters; a descending column adds -desc |
Table names and column names go through the routeSegment inflector, which is kebab case by default. The table segment comes from the table's catalog name; the column segment comes from the column's @name tag, or its catalog name. When two routed tables share a name in different schemas, the default routeSegment puts the schema first for those tables only: /app-public/users/:id and /app-private/users/:id. names.tableSegments(entityIds) does this over the tables given, with the resolver that also names colliding types, so a transport counts only its own routes (see the schemas option of HttpOptions). Two tables that still get one method and path, for example from a routeSegment override, fail the run. The override receives { schema, table, column, name, qualified }. A non-primary key joins its column names with -and- in the by- segment. A composite key adds one path parameter for each column.
When every permissive RLS policy for a command pins a key column to a session function (for example user_id = app_public.current_user_id()), that column is not a parameter. Names and routes use the remaining columns: findUserEmailsByUserIdAndEmail becomes findUserEmailsByEmail with route /user-emails/by-email/:email. A foreign-key list whose only column is session-bound is omitted (the keyset list already returns the visitor's rows). An insert omits a session-bound column when the column default is that same session function.
When RLS binds every column of a unique key for find, update or delete, the generator emits a parameterless current-row procedure instead of dropping the key: updateCurrentUsers / PATCH /users/current, and likewise findCurrent… / GET /…/current and deleteCurrent… / DELETE /…/current when those commands are fully bound. The generated SQL still compares each bound key column to its session function (WHERE id = app_public.current_user_id()), so the write is limited to the visitor's row even when the role bypasses RLS. HttpApi declares /current before /:key siblings of the same method so the literal segment wins. Only one fully bound unique key per entity and command yields that procedure (the primary key if bound, else the first by index id).
The fixture's comments table produces these endpoints (packages/generated-check/generated/http.ts):
| Endpoint | Method and path | Errors |
|---|---|---|
findCommentsById | GET /comments/:id | NotFound, StoreFailure |
updateCommentsById | PATCH /comments/:id | NotFound, StoreFailure |
deleteCommentsById | DELETE /comments/:id | StoreFailure |
listCommentsByParentId | GET /comments/by-parent-id/:parentId | StoreFailure |
listCommentsByPostId | GET /comments/by-post-id/:postId | StoreFailure |
listCommentsOrderedById | GET /comments/ordered-by-id | StoreFailure |
The fixture has no insert endpoint. Its visitor role has no insert privilege on comments.
Path and query values arrive as strings. The generator converts numbers and dates from strings. A nullable key column has a non-null schema on the wire, because a path segment cannot be null.
Error statuses
src/effect/errors.ts declares two errors:
| Error | HTTP status | When |
|---|---|---|
NotFound | 404 | A find or update endpoint finds no row. |
StoreFailure | 500 | A database error or a row that does not decode. |
Only find and update endpoints can return NotFound. A delete endpoint returns only StoreFailure.
RPC
rpc.ts declares one RPC for each query. The success type depends on the cardinality of the query:
- A find or update query returns
Schema.Optionof the row. - A list query returns an array of rows.
- A delete query returns
Schema.Void.
Every RPC can fail with StoreFailure.
Serve the HTTP and RPC layers
This example uses the generated http.ts and rpc.ts files. bun run check:readme compiles it against the generated output of the fixture. It was not run against a database. Add a SqlClient layer from your Postgres driver. The generated query functions need it.
import { Layer } from "effect"
import { HttpRouter } from "effect/http"
import { HttpApiBuilder } from "effect/http-api"
import { RpcSerialization, RpcServer } from "effect/rpc"
import {
Api as HttpApi,
CommentsHandlers,
PostsHandlers,
UserEmailsHandlers,
UsersHandlers,
} from "./generated/http.js"
import { Api as RpcApi, ApiHandlers } from "./generated/rpc.js"
const HttpRoutes = HttpApiBuilder.layer(HttpApi).pipe(
Layer.provide([CommentsHandlers, PostsHandlers, UserEmailsHandlers, UsersHandlers]),
)
const RpcRoutes = RpcServer.layerHttp({ group: RpcApi, path: "/rpc" }).pipe(
Layer.provide([ApiHandlers, RpcSerialization.layerJson]),
)
export const Routes = Layer.mergeAll(HttpRoutes, RpcRoutes)
// Add an HttpServer layer for your runtime, then run the result with Layer.launch.
export const Server = HttpRouter.serve(Routes)
By default the HTTP handler layers are named <Group>Handlers, for example CommentsHandlers. The handlerLayerName inflector sets the name, and handlerGroupName sets the group.
Step 6: Use the generated Kysely code
The kysely plugin writes kysely/db.ts and kysely/queries.ts in each table folder, for example app_public/comments/kysely/queries.ts. Each table's db.ts holds its table type, and the root kysely/db.ts holds the DB interface. Each query takes a Kysely<DB> instance as its first argument. The application creates the instance with its own dialect.
import type { Kysely } from "kysely"
import type { DB } from "./generated/kysely/db.js"
import { findCommentsById, updateCommentsById } from "./generated/app_public/comments/kysely/queries.js"
export const readAndEdit = async (db: Kysely<DB>) => {
const comment = await findCommentsById(db, { id: "1" })
const edited = await updateCommentsById(db, { id: "1", body: "edited" })
return { comment, edited }
}
bun run check:readme compiles this snippet against the generated output of the fixture. It was not run against a database.
Driver settings
Columns of type date, timestamp and timestamptz are string in every output: the types, Kysely, sql-queries, the Effect schemas, the validators and the HTTP routes. A JavaScript Date keeps milliseconds only, and a keyset cursor read from a Date can skip rows that lie less than a millisecond apart. A string keeps the microseconds, and the cursor query compares the bare column, so an index still serves it.
The value is ISO 8601 text with six fractional digits, for example 2024-01-01T00:00:00.001200Z; a date is 2024-01-01. The offset follows the session TimeZone: set it to UTC to get Z always. infinity and -infinity are kept as they are.
The generated code does not set this. Each driver must return the text form, so set it where you create the client. The files packages/generated-check/support/timestamps.ts in this repository hold a tested version of each setting:
pg(and Kysely): passtypes: { getTypeParser }tonew Pool.getTypeParserreturns the text normalizer for the OIDs 1082, 1114 and 1184, and the default parser for the others.postgres: passtypes: { date: { to: 1184, from: [1082, 1114, 1184], serialize, parse } }. This replaces the built-in entry that decodes toDate.@effect/sql-pg: itsdatecodec already returns text. Register a codec for the OIDs 1114 and 1184 in thetypesregistry ofPgClient.layer; it turns the binary microsecond count into the text above. Theserver.tsthateffectApp()generates does this: itsPgTypesLiveregistry is passed toSqlLive.
A column of an array of these types needs the same setting for the array OIDs 1182, 1115 and 1185. Without the setting a driver returns Date while the types say string, and a cursor taken from a row loses its microseconds again.
Troubleshooting
| Message | Cause |
|---|---|
Visitor role not found: <name> | The visitor role is not in the input file. Check the role name and run the introspection query again. |
Unknown role referenced: <name> | The input file references a role that it does not define. |
SnapshotMergeError: <path> | The base or edited file cannot be parsed. Fix the reported side before you rerun. |
path:declaration: reason | Both sides changed a declaration differently. Resolve the markers in the output file. |
SnapshotValidationError | A generated file does not compile. The usual cause is a missing package near the output root. The CLI prints the error tag, then each diagnostic on its own line. To read them, call validateSnapshot from src/snapshot.ts, which returns the diagnostics in its error. |
Unknown output plugin "<name>" | The name in --plugins is not a known output name. |
Query name <name> is given to both <a> and <b> | Two queries get the same name. Rename one table or column with @name, or change the queryName inflector. |
NamesError: <inflector> gave "<name>" for <object>: <reason> | An inflector or a tag produced a name that generated code cannot use. Fix the inflector in inflection, or the @name tag on the object. |
SmartTagError: <object>: Unknown @omit operation: <name> | The @omit tag names an operation that does not exist. Use read, create, update or delete. Every malformed tag fails the run this way, before any query is selected. |
Write an output plugin
An output is the plugin API. The types output (src/types-plugin/output.ts, about 90 lines) is the smallest complete example inside the package. This section uses it.
An output outside the package imports the same API from two entry points: pg-sourcerer (the registry, the core services, the shape types and the capability helpers of the types output) and pg-sourcerer/conjure (the builders). This output lists the field names of every row shape:
import { Effect, Option } from "effect"
import { noLink, Plugin, Shapes, SymbolRegistry, type Document, type Output, type SymbolError } from "pg-sourcerer"
import * as c from "pg-sourcerer/conjure"
const fieldNamesOutput: Output<SymbolError, Shapes> = Effect.gen(function*() {
const registry = yield* SymbolRegistry
const rows = (yield* Shapes).shapes.filter((shape) => shape.kind === "row")
yield* registry.declare({
capability: "field-names:all",
name: "fieldNames",
file: "field-names.ts",
entity: Option.none(),
kind: "value",
statements: [
c.exportConst("fieldNames", rows.reduce((object, shape) => object.prop(shape.exportName, c.array(...shape.fields.map((field) => c.str(field.name)))), c.obj()).build()),
],
})
return yield* Effect.succeed(noLink.pipe(Effect.as<readonly Document[]>([])))
})
export const fieldNames = (): Plugin => Plugin.make("field-names").pipe(Plugin.output(fieldNamesOutput))
packages/generated-check/custom/columns.ts is the same output with a link step: it types each list as ReadonlyArray<keyof Row> and imports the row interfaces of the types output. bun run typecheck runs it from packages/generated-check/custom/pgsourcerer.config.ts and compiles what it writes:
export const columns: {
readonly Comments: ReadonlyArray<keyof Comments>;
// ...
} = {
Comments: [
"body",
// ...
An output is an Effect value of type Output<E, R> (from src/symbols/registry.ts). It has two phases:
type Output<E, R> = Effect<
Effect<readonly Document[], E, SymbolRegistry>, // phase 2: link
E,
R | SymbolRegistry // phase 1: declare
>
- Declare. The outer effect reads its inputs and calls
registry.declarefor each symbol the output exports. - Link. The generator runs every declare phase first. Then it runs every link phase. A link phase calls
registry.importfor symbols it uses from any output, so the order of outputs does not matter. It returns the documents that the output writes itself. It returns an empty list when the registry prints all the output's files.
An output that imports nothing generated has no link work. Return noLink mapped to an empty document list:
return yield* Effect.succeed(noLink.pipe(Effect.as<readonly Document[]>([])))
Write return yield* Effect.succeed(link), not return link. The Effect language service reports a generator that returns an Effect as a value (returnEffectInGen).
Read the inputs
Name the services the output needs in its R type. Both are provided by the core layer.
SemanticModelgivesrelational: entities, fields, relations, enums and indexes.Plansgives the query plan catalog: oneQueryPlanfor each query the rules select.Shapesgives the row, insert and update shapes of each entity, and the enums they use.Namesgives every generated name. See Names.
A field the model hides, and an entity no plan uses, do not appear.
To describe row, insert and update shapes, read the Shapes service (src/shapes/service.ts). It holds ShapeSpec values (id, kind, entity, exportName, fields; entity is the type name of the shape's entity) and the enums they use. The shapes follow the plans: a row shape holds the columns that some plan projects, and an insert or update shape holds the columns that some plan writes. The effect output builds its Schema.Struct values from the same catalog. An output that reads it agrees with the effect output by construction. An output that derives shapes by itself does not.
Names
Every name in generated code comes from the Names service (src/names/service.ts). An output reads a name there, and does not case, join or suffix one itself. The inflectors that build the names are in src/inflection.ts:
| Inflector | Gives | Default |
|---|---|---|
entityTypeName, enumTypeName, domainTypeName | the type name of a table, enum or domain | pascal case of the catalog name |
qualifiedTypeName | the name of each of several same-named objects in different schemas | pascal case of <schema>_<name> |
shapeTypeName | the row, insert or update type of an entity | Users, UsersInsert, UsersUpdate |
fieldName | the property and parameter name of a column | camel case of the column |
queryName | the name of a query | findUsersById, listPostsByUserId, insertUsers |
requestTypeName | the schema of the parameters of a query | <queryName>Request |
handlerGroupName, handlerLayerName | an HTTP group and its handlers layer | users, UsersHandlers |
componentName | a React component of an entity, by view (list, detail, create-form, update-form) | UsersList, UsersCreateForm |
routeSegment | a path segment for a table or column | kebab case; <schema>/<table> for a table whose name is shared by another routed table |
Use these Names accessors and plan fields:
names.entityTypeName(entityId),names.fieldName(fieldId),names.enumTypeName(enumId)and the matching accessors for the other inflectors.plan.nameandplan.requestNamefor the query and the schema of its parameters.plan.parameters[].namefor a parameter.shape.exportNameandshape.entityfor a shape.
A @name smart tag is the final name for its object, and its inflector does not run. The service checks a name when an output asks for it. A name must be a valid identifier, and a name that is bound (a query, a parameter, a group) must not be a reserved word. A failure is a NamesError that holds the inflector, the object and the result, and the run fails with that error, not with the error of the stage that asked. Two columns of one shape that get one field name fail the run with a message from the shapes stage. Two names that collide in one file fail where the file is declared.
Two same-named tables in different schemas get distinct names from qualifiedTypeName. The check covers every entity in the catalog, so a name does not depend on which queries are selected.
A project changes a name with inflection in its config (see the website's Configuration page). An output that spells a name by itself ignores that setting.
Declare symbols
yield* registry.declare({
capability: "types:row:Users", // <owner>:<kind>:<name>, unique in the run
name: "Users", // the exported identifier
file: "types.ts", // the symbol's default file, a path inside the output root; the layout decides the real path
entity: Option.some("public.users"), // the entity id the symbol derives from; Option.none() for a shared symbol
kind: "type", // "type" is always imported type-only
statements: [c.exportInterface("Users", members)],
imports: [{ from: "effect", types: ["Schema"] }], // packages the statements use (optional); `names`, `types` or `namespace`
})
- The owner is the output's name. Use one capability for each symbol. The
typesoutput usestypes:row:<Entity>,types:insert:<Entity>,types:update:<Entity>andtypes:enum:<Name>. - A symbol says what it is, not where it lives.
capabilitystarts with the owner (the plugin name),fileis its default file name, andentityis the semantic entity id (schema.table, asQueryPlan.entityIdspells it) the symbol derives from. A symbol shared by every entity getsOption.none(): the kyselyDBinterface, the error classes, an Api or router that composes every entity, a function's procedure. A shape or a query carries the id of its table. TheLayoutservice turns the three into the real path (see "Write a layout"). - A symbol that another file imports must be exported. A local
constthat only its own file uses is not a symbol: build it inside the symbol that uses it, or declare it withentity: Option.none()beside the symbols that use it. - The registry rejects a repeated capability (
DuplicateCapability) and two symbols that export one name from one resolved file (SymbolCollision). Two outputs may export the same name from different files, sotypes.tsandschemas.tscan both exportUsers. - Build statements with Conjure (
pg-sourcerer/conjure, source insrc/conjure/index.ts):c.exportInterface(name, members),c.exportType(name, type)and the type builders inc.ts(c.ts.string,c.ts.nullable,c.ts.readonlyArray,c.ts.union,c.ts.ref). Map pg types withscalarTs(type, enumName)(exported bypg-sourcerer). It returns the TypeScript type a pg driver returns for a scalar, and it takes an enum export name when the column is an enum. An output that needs a runtime schema callsscalarKind(type)instead. It returnsstring,number,boolean,bytesorunknown, the same table thatscalarTsuses, so a schema and an interface for one column agree. - A
valuesymbol may declare a constant and a type of the same name (export const Xandexport type X). The validator outputs do this, soz.infer<typeof X>is available asX. - A symbol cannot import a name that its own file exports. The
typiaoutput imports the interfaceUsersfromtypes.ts, so it exports its validator asassertUsers. - The output prints no import for a symbol that another symbol in the same resolved file uses. A column of an enum type refers to the enum by name.
Capabilities of the react output
The react output declares one symbol per view of an entity, each in react.tsx, with the entity id of its table. reactCapability (exported from pg-sourcerer) builds the capability ids, which take the entity's type name, as the types capabilities do:
| Builder | Capability | Component | Props |
|---|---|---|---|
reactCapability.list("Users") | react:list:Users | UsersList | { rows: readonly Users[]; onSelect?: (row: Users) => void } |
reactCapability.detail("Users") | react:detail:Users | UsersDetail | { row: Users } |
reactCapability.createForm("Users") | react:create-form:Users | UsersCreateForm | { initial?: Partial<UsersInsert>; onSubmit: (value: UsersInsert) => void | Promise<void> } |
reactCapability.updateForm("Users") | react:update-form:Users | UsersUpdateForm | { initial: Partial<UsersUpdate>; onSubmit: (value: UsersUpdate) => void | Promise<void> } |
An entity has a view only when Routes.select gives it the route the view needs. An output that mounts the components learns which views exist from viewNames(routes, catalog) (src/views/entities.ts), which applies the rules react() and solid() declare them by. A registry lookup cannot answer this in a declare step, because an output later in the run has not declared its symbols yet. The output then imports each view with registry.import in its link step. The props are the contract: a router supplies rows and row from its loaders and calls its own mutation from onSubmit. A component fetches nothing. The component name comes from the componentName inflector.
The views react and solid share
react() and solid() write the same views, so the rules that decide them live in src/views/, not in either plugin: entityViews (which of List, Detail, CreateForm and UpdateForm an entity has, from Routes.select and the shape catalog), fieldInput (the input a field gets), the <Name>Props aliases (props.ts), and the conjure builders of a cell's text and of a form's draft, starting values and submit conversion (cell.ts, draft.ts). A plugin keeps what depends on the framework: the JSX, its state hooks, its capability ids and its file. solid() has the same props and capabilities under the solid: prefix; solidCapability (exported from pg-sourcerer) builds them.
Capabilities of the tanstack-router output
tanstackRouterCapability (exported from pg-sourcerer) builds the capability ids of the router output. Each page is one symbol in the entity's tanstack-router.tsx, named <Component>Route after the component it mounts. The entity argument is the entity's type name, as in reactCapability:
| Builder | Symbol | File |
|---|---|---|
tanstackRouterCapability.list("Users") | UsersListRoute | tanstack-router.tsx |
tanstackRouterCapability.detail("Users") | UsersDetailRoute | tanstack-router.tsx |
tanstackRouterCapability.new("Users") | UsersCreateFormRoute | tanstack-router.tsx |
tanstackRouterCapability.edit("Users") | UsersUpdateFormRoute | tanstack-router.tsx |
tanstackRouterCapability.context | RouterContext | router/root.tsx |
tanstackRouterCapability.root | rootRoute | router/root.tsx |
tanstackRouterCapability.tree | routeTree | router/tree.tsx |
The output decides which pages exist with viewNames, as the react section above says. The client's type is clientCapability.type (client:type, the Client that createClient returns).
Link symbols
A link phase asks for symbols that other outputs declared:
const link = Effect.gen(function* () {
const registry = yield* SymbolRegistry
yield* registry.import("effect:schema:Users", { into: "effect:query:findUsersById", type: true })
return []
})
registry.import(capability, { into, type? })records that the symbol with capabilityintouses the symbolcapability. Imports belong to symbols, so each file imports exactly what its own symbols use under any layout. It fails withUnknownCapabilitywhen no selected output declaredcapabilityorinto. The registry resolves both files through the layout and writes the import declaration with the right relative path.- Every reference from one declared symbol to another goes through
registry.import, even when both sit in one file underbyKind(). The registry drops an import whose target is in the importing file, so that layout writes no self-import. A reference with no import works only while the layout keeps both symbols in one file. ACannot find name Xerror from a generated project means a reference with no link: add the import in the output that declares the user ofX. registry.importOptionalreturnsOption.none()instead of failing, for an output that works without the other output.- The generator reads
registry.documentsafter the link phase. An output does not call it.
Write a layout
A layout decides the path of each symbol's file. It is a Layout service: a function from a Placement ({ owner, file, entity }) to an output-relative path. The default is by table: each entity's symbols go under <schema>/<table>/<file>, and symbols with no entity stay in file. The byKind() plugin selects the by-kind layout, where the path is file.
import { Layer } from "effect"
import { Layout, Plugin } from "pg-sourcerer"
export const byOwner = (): Plugin =>
Plugin.make("layout").pipe(Plugin.layer(Layer.succeed(Layout, ({ owner, file }) => `${owner}/${file}`)))
- The plugin is named
layout. Two layout plugins in one config fail withPluginSelectionError. - The registry reads
Layoutwhen the run starts, so listing the plugin in the config is enough. No other plugin requires it. - The layout must be pure and give two symbols one path only when they may share a file: the registry rejects two symbols of one name in one path.
- Keep the path relative, normalized and ending in
.ts. The generator validates the files it writes. openapi.jsonis a document an output returns, not a symbol, so it stays at the output root under every layout.- Switching layouts moves files. The merge publish keys its manifest by path, so a project that switches layouts sees its files appear at the new paths. Edits a user made in the files at the old paths stay where they are.
Errors
An output may fail with any tagged error of its own. The generator wraps each failure in OutputFailed { output, error }, where output is the output's name and error is the original tagged error. The CLI prints OutputFailed: output "<name>" failed, then one line with the error's tag and its message. An error must therefore have a message built from its typed fields. A Schema.TaggedError has an empty message by default, so override the getter:
class MyOutputError extends Schema.TaggedError<MyOutputError, { readonly brand: unique symbol }>()(
"MyOutputError",
{ issue: Schema.String },
) {
override get message(): string {
return this.issue
}
}
No union lists the errors of the outputs, so register nothing. Errors from the registry (SymbolError) pass through the same way.
Expose the output
Build a Plugin in steps and return it from a constructor:
const typesPlugin = Plugin.make("types").pipe(Plugin.output(typesOutput))
export const types = (): Plugin => typesPlugin
Plugin.make(name) starts a plugin with no output and no services. Each step after it adds one thing, and the compiler checks every step against what the earlier steps gave the plugin:
Plugin.requires(other)adds a plugin the generator runs with this one, and the servicesotherholds.Plugin.layer(layer)adds services the plugin owns. The layer may read the services the plugin already holds.Plugin.output(output)sets the output. It may read the core services and the services the plugin holds. An output that reads a service no step gave the plugin is a type error.
The name identifies the plugin in failure reports and owns its capabilities. A user adds types() to plugins in pgsourcerer.config.ts. A constructor without options returns one value each time, so two plugins that require types() hold the same plugin and the generator runs it once.
The generator resolves the plugins in the config and every plugin they require into one plugin per name:
- A plugin you list replaces every required plugin of the same name. Files link by name, so the dependents link to yours.
- The services of your plugin override the services of the replaced one, by tag, wherever a dependent reads them: in its layer and in its output. A service yours does not provide still comes from the replaced plugin. A dependent's own
Plugin.layerstep still wins over both. - Your plugin may require the plugin it replaces and add to it. Its own steps read the replaced plugin; every other plugin reads yours.
- Give each layer object to one plugin's
Plugin.layerstep. Effect builds a layer object once per run, so to share a service, require the plugin that holds it. - Two different required plugins of one name fail with
PluginConflict. List one of them in the config to choose it. - Two entries of one name in the config fail with
PluginSelectionError.
To also select the plugin with --plugins types, add the name to BuiltinPluginName and pluginByName in src/pipeline.ts, and export the constructor from src/index.ts. A plugin that lives outside this package needs only the constructor in plugins; --plugins selects built-in plugins only, and it replaces the plugins of the config file.
Validator outputs
The zod, valibot and arktype outputs share one walk (src/validators/shared.ts). A dialect says how a library spells a scalar, enum, array, nullable value and object. The walk reads Shapes, so each schema has the fields of the types interface of the same name. Capabilities are <name>:schema:<Export> and <name>:enum:<Name>. The schemas validate the value a pg driver returns, for example a Date for a timestamp.
Each schema comes with a type of the same name. For the fixture's Username domain, zod.ts holds:
export const Username = z.string().min(2).max(24).regex(new RegExp("^[a-zA-Z][a-zA-Z0-9_-]+$", "i"));
export type Username = z.infer<typeof Username>;
The typia output is different. typia builds a validator from a TypeScript type, so the output imports the interfaces of the types output (registry.import, type-only) and fails with UnknownCapability when types is not selected. The generated typia.ts runs only after the typia transformer rewrites it. Compile it with ttsc or run it with ttsx, with the plugin { "transform": "typia/lib/transform" } in a tsconfig that holds only the files typia needs (see packages/generated-check/tsconfig.typia.json). tsc type-checks the file but does not transform it. The ttsc build compiles a Go plugin and needs a Go toolchain; its cache is large. check:validators keeps the cache in ~/.cache/pg-sourcerer/ttsc (or under XDG_CACHE_HOME), TTSC_CACHE_DIR moves it, and the script stops with a message when go is not on PATH.
Test the output
- Run the plugin with
generatePlugins(snapshot, "visitor", [], [yourPlugin()])on a smallCatalogSnapshotand check the document paths and a few lines of source (seetest/pipeline.test.ts). - Add the output to
packages/generated-check/pgsourcerer.config.ts.bun run typecheckthen compiles its files with the real compiler. - When two outputs must agree, add a type-level assertion to
packages/generated-check/assertions/.types-equal-schemas.tsshows that every plain type equalstypeof Schema.Typeof the same name. A mismatch is a type error.