Skip to main content

Generate Code With pg-sourcerer

This guide shows how to generate TypeScript from a PostgreSQL database. For the design, read ARCHITECTURE.md.

Terms​

  • Introspection file: a JSON file that holds the catalog of one database. It is the output of the query from @danielfgray/pg-introspection.
  • Visitor role: the database role that the generated code runs as. The generator produces only the queries that this role can run.
  • Smart tag: a line in a PostgreSQL comment that starts with @, or an entry of the tags section of the config file. The generator reads the tags @omit, @name, @primaryKey, @foreignKey, @notNull and @brand.
  • Output root: the directory that receives the generated files.
  • Generation: one complete set of generated files. The generator merges it into the output root and records the last pristine output as hashes in .pg-sourcerer/manifest.json.
  • Output: a document emitter. The effect and kysely outputs emit code.
  • Plugin: a name, the plugins it requires, the services it holds and an optional output. You list plugins in the config; the generator also runs the plugins they require.

Requirements​

  • Bun.
  • A PostgreSQL database. The visitor role must exist in that database.
  • An output root that sits inside a project that installs the packages the generated code imports:
    • effect version 4 for the effect plugin.
    • kysely for the kysely plugin.

The generator typechecks every generated file before it publishes the files. It resolves each package import from the directory tree above the output root. If the output root has no installed effect version 4, the run fails with SnapshotValidationError and publishes nothing. packages/generated-check/package.json is an example of a project that installs both packages.

Step 1: Make an introspection file​

Run these commands from packages/pg-sourcerer. The first command writes the introspection query to a file. The second command runs it.

bun -e 'import { makeIntrospectionQuery } from "@danielfgray/pg-introspection"; process.stdout.write(makeIntrospectionQuery())' > introspection.sql
psql -X -q -t -A -v ON_ERROR_STOP=1 -U <owner-role> -d <database> -f introspection.sql > introspection.json

Replace <owner-role> and <database> with your values. Run the query as the database owner role. The fixture script (scripts/introspection-fixture.ts) does the same.

The generator accepts two input shapes:

  • The introspection output. The file has a top-level classes key. src/pg-source.ts converts it.
  • A CatalogSnapshot object. src/domain.ts defines the shape. It has tables, relations, enums, roles and memberships.

To make the demo introspection file, run bun run fixture:introspection. This command needs Docker. It writes examples/introspection.json.

Step 2: Generate the code​

With a config file​

Put a pgsourcerer.config.ts file in your project. The CLI looks for it in the current directory and in every parent directory, and uses the nearest one.

import { defineConfig, fileSource, webApp } from "pg-sourcerer"

export default defineConfig({
source: fileSource("./introspection.json"),
visitorRole: "visitor",
outputDir: "./src/db",
plugins: [webApp()],
format: ["bunx", "oxfmt"],
})

Then run bun run src/cli.ts from packages/pg-sourcerer, or from any directory below the config file.

The fields are:

  • source: fileSource(path) for an introspection file or CatalogSnapshot file, or connectionSource(url) for a live database. The connection URL is kept as a Redacted value, so logs do not show it.
  • visitorRole: the role name. The role must exist in the catalog.
  • outputDir: the output root. See the rules for the output root below.
  • plugins: a list of plugins. effect(), kysely() and suggestionReport() each build one. A plugin also runs the plugins it requires, so list only the ones you want generated. webApp() is a plugin that requires effect() and kysely(). A preset is a plugin.
  • format (optional): a formatter command as an argv list (program first, then its arguments). The generator appends the path of a directory that holds the generated files, and runs it. The files in that directory must be formatted in place. The generator runs the formatter before validation and merging. It does not format the merged text.

Relative paths in the config file resolve against the directory of the config file.

The CLI imports the config file with a dynamic import. It fails with ConfigLoadError when the file throws while it loads, has no default export, or holds an invalid value. The data fields are checked with a Schema. Each entry of plugins must be a Plugin.

With flags​

A flag replaces the config field with the same meaning. With no config file, the flags work alone.

bun run src/cli.ts ../my-app/src/db visitor --input ./introspection.json --plugins effect,kysely

The arguments are:

  1. The output root (optional when the config sets outputDir). Relative paths resolve against the current directory. You can edit generated files and keep other files in this directory. Choose a directory inside a project that installs effect version 4 (and kysely, if you use that plugin). See Requirements.
  2. The visitor role name (optional when the config sets visitorRole). The role must exist in the catalog.
  3. --input, the introspection file or CatalogSnapshot file. --connection takes a connection URL instead. Pass one of them, not both. With no config and neither flag, the CLI reads DATABASE_URL.
  4. --plugins, a comma-separated list of output names. The default is effect. The known names are effect, kysely, types, zod, valibot, arktype, typia and suggestion-report. The CLI fails when it gets an unknown name or the same name twice.

The CLI reports the number of files written, created and deleted. It lists each merge conflict and exits with code 1 when conflicts exist.

Output layout​

The output root holds these files and folders:

generated/
app_public/users/schemas.ts editable generated code
app_public/users/queries.ts
.pg-sourcerer/
manifest.json hashes and import names from the last run

Import files directly from the output root. Commit .pg-sourcerer/manifest.json with your code. Each run merges your edits with the new output. Resolve conflict markers before the next run. See Editable generated code for the merge rules.

With --plugins effect, the generation holds these files. Each table gets its own folder, <schema>/<table>/, that holds the files of its schemas, types and queries, for example app_public/users/schemas.ts. The shared files (errors.ts, rpc.ts, http.ts) stay at the root. The byKind() plugin puts all tables in one file for each kind instead (see "Write a layout"). The list names each file without its folder:

  • schemas.ts
  • queries.ts
  • errors.ts
  • rpc.ts
  • http.ts

When kysely is selected, the generation also holds:

  • kysely/db.ts
  • kysely/queries.ts

Step 3: Tag the schema with smart tags​

Smart tags are PostgreSQL comments. Write them with COMMENT ON. Each tag must start its own line.

comment on table app_public.comments is '@name Reply';
comment on column app_public.comments.id is '@omit update,delete';

The generator reads the comment from the catalog. It does not read the database at generation time. Run the introspection query again after you add a tag.

Tags in the config file​

Every tag also has a config form, so a schema you cannot comment on can still be tagged. The tags section of defineConfig follows PostGraphile's tags file. Rules are grouped by what they tag, and a key is name or schema.name:

defineConfig({
// ...
tags: {
class: { // tables and views
"app_public.comments": {
description: "A reply to a post.", // replaces the comment's prose
tags: { name: "Reply", omit: "delete" },
attribute: { id: { tags: { omit: "update" } } }, // columns of this table
},
},
attribute: { "comments.body": { tags: { notNull: true } } }, // table.column or schema.table.column
procedure: { "app_public.search_posts": { tags: { omit: true } } }, // every overload of the name
type: { "app_public.ticket_number": { tags: { brand: false } } }, // domains
},
})

A tag value is the text that follows the tag in a comment: omit: "read, create" is @omit read, create. true is the bare tag, false is the tag followed by false, and a list is one tag line per item (foreignKey: ["(a) references t", "(b) references u"]). One parser reads both forms, so the same values are valid in both.

The config wins when both set a tag. A rule's tags replace the comment's tags of the same name and keep the other comment tags. A rule's description replaces the prose of the comment. A bare key applies to every schema; when a bare and a qualified key both match, the qualified one is applied last. A rule that matches no table, column, function or domain fails the run, as does an unknown tag name.

Tag reference​

TagOnEffect
@omittableRemoves every candidate on the table.
@omit alltableSame as @omit.
@omit read,create,update,deletetableRemoves the candidates for the named operations. create is the insert operation.
@omit update,deletecolumnRemoves the candidates whose key columns include this column, for the named operations.
@name ReplytableSets the entity type name to Reply, exactly as written: the entityTypeName inflector does not run. Names derived from it (query names, request names, group names, shape names) run their own inflectors over Reply. The URL path keeps the table name.
@primaryKey idviewDeclares the key columns of a view. A table has constraints, so the tag on a table fails the run.
@foreignKey (a) references [schema.]t [(x)]viewDeclares a foreign key of a view; one line for each key.
@notNullcolumnAsserts a view column is never null. @notNull false clears it.
@brand, @brand falsedomainMakes the domain a nominal type, or keeps it plain, whatever domains.brand says.
@omit, @name Namefunction@omit leaves the function out; @name names its query exactly as written.
@name threadcolumnSets the field name to thread, exactly as written: the fieldName inflector does not run. The property in every shape and the parameter of every query that uses the column take it. Query names and path segments for keys on this column are derived from it.

The generator fails when a tag is wrong. These cases fail:

  • An unknown operation in @omit, for example @omit reed. The run fails with a SmartTagError before any query is selected.
  • A @name tag with no name, or with more than one name.
  • A @name tag that appears twice on one comment.
  • A tag on an object it does not apply to, for example @omit on a domain or @brand on a table.
  • Two selected queries that get the same name.

What a column tag does not do​

A column tag does not hide the column. It does not remove the column from the result of a read. It does not remove the column from an insert or an update. It removes only the candidates that use the column as a key.

Example result​

A probe run edited a copy of the demo introspection file. The table comment set @name Reply, and the id comment set @omit update,delete. The generated output had these results:

  • The find and list names used Reply in place of Comments.
  • The HTTP group was named for Reply.
  • The URL paths kept the table name: /comments/:id.
  • The update and delete queries for id were removed.
  • The find query for id remained.

Step 4: Select queries with rules (library use)​

The CLI passes no query rules. A rule adds or removes candidates by kind or by entity. Use the library function generateSnapshot from pg-sourcerer to pass rules. The function returns an Effect that you can run with Effect.runSync or Effect.result.

import { Effect } from "effect"
import { generateSnapshot } from "pg-sourcerer"

// `snapshot` is a CatalogSnapshot, for example from databaseSnapshotFromIntrospection.
const result = Effect.runSync(
Effect.result(
generateSnapshot(
snapshot,
"visitor",
[{ action: "exclude", kind: "delete-by-key" }],
["effect"],
),
),
)

A rule has these fields:

  • action: "include" or "exclude". Rules run in order. The last matching rule sets the state.
  • kind (optional): one of find-by-key, list-by-foreign-key, list-keyset, list-by-index, insert, update-by-key, delete-by-key.
  • entity (optional): an entity id such as app_public.comments.

A smart tag runs after the rules. A tag @omit always removes its candidates, even when a rule includes them.

Step 5: Use the generated Effect code​

HTTP routes​

The http.ts file declares one HttpApiGroup for each table. The group name comes from the handlerGroupName inflector, which is the entity type name in camel case by default. Each candidate becomes one endpoint. The route path follows these rules (src/effect/http.ts):

CandidateMethodPath
insertPOST/<table>
find-by-key, update-by-key, delete-by-key on the primary keyGET, PATCH, DELETE/<table>/:<key>
find-by-key, update-by-key, delete-by-key on another unique keyGET, PATCH, DELETE/<table>/by-<columns>/:<key>
list-by-foreign-keyGET/<table>/by-<columns>/:<key>
list-keysetGET/<table>/ordered-by-<columns>
list-by-indexGET/<table>/ordered-by-<columns>, or /<table>/by-<columns>/:<key>/ordered-by-<columns> when a prefix filters; a descending column adds -desc

Table names and column names go through the routeSegment inflector, which is kebab case by default. The table segment comes from the table's catalog name; the column segment comes from the column's @name tag, or its catalog name. When two routed tables share a name in different schemas, the default routeSegment puts the schema first for those tables only: /app-public/users/:id and /app-private/users/:id. names.tableSegments(entityIds) does this over the tables given, with the resolver that also names colliding types, so a transport counts only its own routes (see the schemas option of HttpOptions). Two tables that still get one method and path, for example from a routeSegment override, fail the run. The override receives { schema, table, column, name, qualified }. A non-primary key joins its column names with -and- in the by- segment. A composite key adds one path parameter for each column.

When every permissive RLS policy for a command pins a key column to a session function (for example user_id = app_public.current_user_id()), that column is not a parameter. Names and routes use the remaining columns: findUserEmailsByUserIdAndEmail becomes findUserEmailsByEmail with route /user-emails/by-email/:email. A foreign-key list whose only column is session-bound is omitted (the keyset list already returns the visitor's rows). An insert omits a session-bound column when the column default is that same session function.

When RLS binds every column of a unique key for find, update or delete, the generator emits a parameterless current-row procedure instead of dropping the key: updateCurrentUsers / PATCH /users/current, and likewise findCurrent… / GET /…/current and deleteCurrent… / DELETE /…/current when those commands are fully bound. The generated SQL still compares each bound key column to its session function (WHERE id = app_public.current_user_id()), so the write is limited to the visitor's row even when the role bypasses RLS. HttpApi declares /current before /:key siblings of the same method so the literal segment wins. Only one fully bound unique key per entity and command yields that procedure (the primary key if bound, else the first by index id).

The fixture's comments table produces these endpoints (packages/generated-check/generated/http.ts):

EndpointMethod and pathErrors
findCommentsByIdGET /comments/:idNotFound, StoreFailure
updateCommentsByIdPATCH /comments/:idNotFound, StoreFailure
deleteCommentsByIdDELETE /comments/:idStoreFailure
listCommentsByParentIdGET /comments/by-parent-id/:parentIdStoreFailure
listCommentsByPostIdGET /comments/by-post-id/:postIdStoreFailure
listCommentsOrderedByIdGET /comments/ordered-by-idStoreFailure

The fixture has no insert endpoint. Its visitor role has no insert privilege on comments.

Path and query values arrive as strings. The generator converts numbers and dates from strings. A nullable key column has a non-null schema on the wire, because a path segment cannot be null.

Error statuses​

src/effect/errors.ts declares two errors:

ErrorHTTP statusWhen
NotFound404A find or update endpoint finds no row.
StoreFailure500A database error or a row that does not decode.

Only find and update endpoints can return NotFound. A delete endpoint returns only StoreFailure.

RPC​

rpc.ts declares one RPC for each query. The success type depends on the cardinality of the query:

  • A find or update query returns Schema.Option of the row.
  • A list query returns an array of rows.
  • A delete query returns Schema.Void.

Every RPC can fail with StoreFailure.

Serve the HTTP and RPC layers​

This example uses the generated http.ts and rpc.ts files. bun run check:readme compiles it against the generated output of the fixture. It was not run against a database. Add a SqlClient layer from your Postgres driver. The generated query functions need it.

import { Layer } from "effect"
import { HttpRouter } from "effect/http"
import { HttpApiBuilder } from "effect/http-api"
import { RpcSerialization, RpcServer } from "effect/rpc"
import {
Api as HttpApi,
CommentsHandlers,
PostsHandlers,
UserEmailsHandlers,
UsersHandlers,
} from "./generated/http.js"
import { Api as RpcApi, ApiHandlers } from "./generated/rpc.js"

const HttpRoutes = HttpApiBuilder.layer(HttpApi).pipe(
Layer.provide([CommentsHandlers, PostsHandlers, UserEmailsHandlers, UsersHandlers]),
)

const RpcRoutes = RpcServer.layerHttp({ group: RpcApi, path: "/rpc" }).pipe(
Layer.provide([ApiHandlers, RpcSerialization.layerJson]),
)

export const Routes = Layer.mergeAll(HttpRoutes, RpcRoutes)

// Add an HttpServer layer for your runtime, then run the result with Layer.launch.
export const Server = HttpRouter.serve(Routes)

By default the HTTP handler layers are named <Group>Handlers, for example CommentsHandlers. The handlerLayerName inflector sets the name, and handlerGroupName sets the group.

Step 6: Use the generated Kysely code​

The kysely plugin writes kysely/db.ts and kysely/queries.ts in each table folder, for example app_public/comments/kysely/queries.ts. Each table's db.ts holds its table type, and the root kysely/db.ts holds the DB interface. Each query takes a Kysely<DB> instance as its first argument. The application creates the instance with its own dialect.

import type { Kysely } from "kysely"
import type { DB } from "./generated/kysely/db.js"
import { findCommentsById, updateCommentsById } from "./generated/app_public/comments/kysely/queries.js"

export const readAndEdit = async (db: Kysely<DB>) => {
const comment = await findCommentsById(db, { id: "1" })
const edited = await updateCommentsById(db, { id: "1", body: "edited" })
return { comment, edited }
}

bun run check:readme compiles this snippet against the generated output of the fixture. It was not run against a database.

Driver settings​

Columns of type date, timestamp and timestamptz are string in every output: the types, Kysely, sql-queries, the Effect schemas, the validators and the HTTP routes. A JavaScript Date keeps milliseconds only, and a keyset cursor read from a Date can skip rows that lie less than a millisecond apart. A string keeps the microseconds, and the cursor query compares the bare column, so an index still serves it.

The value is ISO 8601 text with six fractional digits, for example 2024-01-01T00:00:00.001200Z; a date is 2024-01-01. The offset follows the session TimeZone: set it to UTC to get Z always. infinity and -infinity are kept as they are.

The generated code does not set this. Each driver must return the text form, so set it where you create the client. The files packages/generated-check/support/timestamps.ts in this repository hold a tested version of each setting:

  • pg (and Kysely): pass types: { getTypeParser } to new Pool. getTypeParser returns the text normalizer for the OIDs 1082, 1114 and 1184, and the default parser for the others.
  • postgres: pass types: { date: { to: 1184, from: [1082, 1114, 1184], serialize, parse } }. This replaces the built-in entry that decodes to Date.
  • @effect/sql-pg: its date codec already returns text. Register a codec for the OIDs 1114 and 1184 in the types registry of PgClient.layer; it turns the binary microsecond count into the text above. The server.ts that effectApp() generates does this: its PgTypesLive registry is passed to SqlLive.

A column of an array of these types needs the same setting for the array OIDs 1182, 1115 and 1185. Without the setting a driver returns Date while the types say string, and a cursor taken from a row loses its microseconds again.

Troubleshooting​

MessageCause
Visitor role not found: <name>The visitor role is not in the input file. Check the role name and run the introspection query again.
Unknown role referenced: <name>The input file references a role that it does not define.
SnapshotMergeError: <path>The base or edited file cannot be parsed. Fix the reported side before you rerun.
path:declaration: reasonBoth sides changed a declaration differently. Resolve the markers in the output file.
SnapshotValidationErrorA generated file does not compile. The usual cause is a missing package near the output root. The CLI prints the error tag, then each diagnostic on its own line. To read them, call validateSnapshot from src/snapshot.ts, which returns the diagnostics in its error.
Unknown output plugin "<name>"The name in --plugins is not a known output name.
Query name <name> is given to both <a> and <b>Two queries get the same name. Rename one table or column with @name, or change the queryName inflector.
NamesError: <inflector> gave "<name>" for <object>: <reason>An inflector or a tag produced a name that generated code cannot use. Fix the inflector in inflection, or the @name tag on the object.
SmartTagError: <object>: Unknown @omit operation: <name>The @omit tag names an operation that does not exist. Use read, create, update or delete. Every malformed tag fails the run this way, before any query is selected.

Write an output plugin​

An output is the plugin API. The types output (src/types-plugin/output.ts, about 90 lines) is the smallest complete example inside the package. This section uses it.

An output outside the package imports the same API from two entry points: pg-sourcerer (the registry, the core services, the shape types and the capability helpers of the types output) and pg-sourcerer/conjure (the builders). This output lists the field names of every row shape:

import { Effect, Option } from "effect"
import { noLink, Plugin, Shapes, SymbolRegistry, type Document, type Output, type SymbolError } from "pg-sourcerer"
import * as c from "pg-sourcerer/conjure"

const fieldNamesOutput: Output<SymbolError, Shapes> = Effect.gen(function*() {
const registry = yield* SymbolRegistry
const rows = (yield* Shapes).shapes.filter((shape) => shape.kind === "row")
yield* registry.declare({
capability: "field-names:all",
name: "fieldNames",
file: "field-names.ts",
entity: Option.none(),
kind: "value",
statements: [
c.exportConst("fieldNames", rows.reduce((object, shape) => object.prop(shape.exportName, c.array(...shape.fields.map((field) => c.str(field.name)))), c.obj()).build()),
],
})
return yield* Effect.succeed(noLink.pipe(Effect.as<readonly Document[]>([])))
})

export const fieldNames = (): Plugin => Plugin.make("field-names").pipe(Plugin.output(fieldNamesOutput))

packages/generated-check/custom/columns.ts is the same output with a link step: it types each list as ReadonlyArray<keyof Row> and imports the row interfaces of the types output. bun run typecheck runs it from packages/generated-check/custom/pgsourcerer.config.ts and compiles what it writes:

export const columns: {
readonly Comments: ReadonlyArray<keyof Comments>;
// ...
} = {
Comments: [
"body",
// ...

An output is an Effect value of type Output<E, R> (from src/symbols/registry.ts). It has two phases:

type Output<E, R> = Effect<
Effect<readonly Document[], E, SymbolRegistry>, // phase 2: link
E,
R | SymbolRegistry // phase 1: declare
>
  1. Declare. The outer effect reads its inputs and calls registry.declare for each symbol the output exports.
  2. Link. The generator runs every declare phase first. Then it runs every link phase. A link phase calls registry.import for symbols it uses from any output, so the order of outputs does not matter. It returns the documents that the output writes itself. It returns an empty list when the registry prints all the output's files.

An output that imports nothing generated has no link work. Return noLink mapped to an empty document list:

return yield* Effect.succeed(noLink.pipe(Effect.as<readonly Document[]>([])))

Write return yield* Effect.succeed(link), not return link. The Effect language service reports a generator that returns an Effect as a value (returnEffectInGen).

Read the inputs​

Name the services the output needs in its R type. Both are provided by the core layer.

  • SemanticModel gives relational: entities, fields, relations, enums and indexes.
  • Plans gives the query plan catalog: one QueryPlan for each query the rules select.
  • Shapes gives the row, insert and update shapes of each entity, and the enums they use.
  • Names gives every generated name. See Names.

A field the model hides, and an entity no plan uses, do not appear.

To describe row, insert and update shapes, read the Shapes service (src/shapes/service.ts). It holds ShapeSpec values (id, kind, entity, exportName, fields; entity is the type name of the shape's entity) and the enums they use. The shapes follow the plans: a row shape holds the columns that some plan projects, and an insert or update shape holds the columns that some plan writes. The effect output builds its Schema.Struct values from the same catalog. An output that reads it agrees with the effect output by construction. An output that derives shapes by itself does not.

Names​

Every name in generated code comes from the Names service (src/names/service.ts). An output reads a name there, and does not case, join or suffix one itself. The inflectors that build the names are in src/inflection.ts:

InflectorGivesDefault
entityTypeName, enumTypeName, domainTypeNamethe type name of a table, enum or domainpascal case of the catalog name
qualifiedTypeNamethe name of each of several same-named objects in different schemaspascal case of <schema>_<name>
shapeTypeNamethe row, insert or update type of an entityUsers, UsersInsert, UsersUpdate
fieldNamethe property and parameter name of a columncamel case of the column
queryNamethe name of a queryfindUsersById, listPostsByUserId, insertUsers
requestTypeNamethe schema of the parameters of a query<queryName>Request
handlerGroupName, handlerLayerNamean HTTP group and its handlers layerusers, UsersHandlers
componentNamea React component of an entity, by view (list, detail, create-form, update-form)UsersList, UsersCreateForm
routeSegmenta path segment for a table or columnkebab case; <schema>/<table> for a table whose name is shared by another routed table

Use these Names accessors and plan fields:

  • names.entityTypeName(entityId), names.fieldName(fieldId), names.enumTypeName(enumId) and the matching accessors for the other inflectors.
  • plan.name and plan.requestName for the query and the schema of its parameters. plan.parameters[].name for a parameter.
  • shape.exportName and shape.entity for a shape.

A @name smart tag is the final name for its object, and its inflector does not run. The service checks a name when an output asks for it. A name must be a valid identifier, and a name that is bound (a query, a parameter, a group) must not be a reserved word. A failure is a NamesError that holds the inflector, the object and the result, and the run fails with that error, not with the error of the stage that asked. Two columns of one shape that get one field name fail the run with a message from the shapes stage. Two names that collide in one file fail where the file is declared.

Two same-named tables in different schemas get distinct names from qualifiedTypeName. The check covers every entity in the catalog, so a name does not depend on which queries are selected.

A project changes a name with inflection in its config (see the website's Configuration page). An output that spells a name by itself ignores that setting.

Declare symbols​

yield* registry.declare({
capability: "types:row:Users", // <owner>:<kind>:<name>, unique in the run
name: "Users", // the exported identifier
file: "types.ts", // the symbol's default file, a path inside the output root; the layout decides the real path
entity: Option.some("public.users"), // the entity id the symbol derives from; Option.none() for a shared symbol
kind: "type", // "type" is always imported type-only
statements: [c.exportInterface("Users", members)],
imports: [{ from: "effect", types: ["Schema"] }], // packages the statements use (optional); `names`, `types` or `namespace`
})
  • The owner is the output's name. Use one capability for each symbol. The types output uses types:row:<Entity>, types:insert:<Entity>, types:update:<Entity> and types:enum:<Name>.
  • A symbol says what it is, not where it lives. capability starts with the owner (the plugin name), file is its default file name, and entity is the semantic entity id (schema.table, as QueryPlan.entityId spells it) the symbol derives from. A symbol shared by every entity gets Option.none(): the kysely DB interface, the error classes, an Api or router that composes every entity, a function's procedure. A shape or a query carries the id of its table. The Layout service turns the three into the real path (see "Write a layout").
  • A symbol that another file imports must be exported. A local const that only its own file uses is not a symbol: build it inside the symbol that uses it, or declare it with entity: Option.none() beside the symbols that use it.
  • The registry rejects a repeated capability (DuplicateCapability) and two symbols that export one name from one resolved file (SymbolCollision). Two outputs may export the same name from different files, so types.ts and schemas.ts can both export Users.
  • Build statements with Conjure (pg-sourcerer/conjure, source in src/conjure/index.ts): c.exportInterface(name, members), c.exportType(name, type) and the type builders in c.ts (c.ts.string, c.ts.nullable, c.ts.readonlyArray, c.ts.union, c.ts.ref). Map pg types with scalarTs(type, enumName) (exported by pg-sourcerer). It returns the TypeScript type a pg driver returns for a scalar, and it takes an enum export name when the column is an enum. An output that needs a runtime schema calls scalarKind(type) instead. It returns string, number, boolean, bytes or unknown, the same table that scalarTs uses, so a schema and an interface for one column agree.
  • A value symbol may declare a constant and a type of the same name (export const X and export type X). The validator outputs do this, so z.infer<typeof X> is available as X.
  • A symbol cannot import a name that its own file exports. The typia output imports the interface Users from types.ts, so it exports its validator as assertUsers.
  • The output prints no import for a symbol that another symbol in the same resolved file uses. A column of an enum type refers to the enum by name.

Capabilities of the react output​

The react output declares one symbol per view of an entity, each in react.tsx, with the entity id of its table. reactCapability (exported from pg-sourcerer) builds the capability ids, which take the entity's type name, as the types capabilities do:

BuilderCapabilityComponentProps
reactCapability.list("Users")react:list:UsersUsersList{ rows: readonly Users[]; onSelect?: (row: Users) => void }
reactCapability.detail("Users")react:detail:UsersUsersDetail{ row: Users }
reactCapability.createForm("Users")react:create-form:UsersUsersCreateForm{ initial?: Partial<UsersInsert>; onSubmit: (value: UsersInsert) => void | Promise<void> }
reactCapability.updateForm("Users")react:update-form:UsersUsersUpdateForm{ initial: Partial<UsersUpdate>; onSubmit: (value: UsersUpdate) => void | Promise<void> }

An entity has a view only when Routes.select gives it the route the view needs. An output that mounts the components learns which views exist from viewNames(routes, catalog) (src/views/entities.ts), which applies the rules react() and solid() declare them by. A registry lookup cannot answer this in a declare step, because an output later in the run has not declared its symbols yet. The output then imports each view with registry.import in its link step. The props are the contract: a router supplies rows and row from its loaders and calls its own mutation from onSubmit. A component fetches nothing. The component name comes from the componentName inflector.

The views react and solid share​

react() and solid() write the same views, so the rules that decide them live in src/views/, not in either plugin: entityViews (which of List, Detail, CreateForm and UpdateForm an entity has, from Routes.select and the shape catalog), fieldInput (the input a field gets), the <Name>Props aliases (props.ts), and the conjure builders of a cell's text and of a form's draft, starting values and submit conversion (cell.ts, draft.ts). A plugin keeps what depends on the framework: the JSX, its state hooks, its capability ids and its file. solid() has the same props and capabilities under the solid: prefix; solidCapability (exported from pg-sourcerer) builds them.

Capabilities of the tanstack-router output​

tanstackRouterCapability (exported from pg-sourcerer) builds the capability ids of the router output. Each page is one symbol in the entity's tanstack-router.tsx, named <Component>Route after the component it mounts. The entity argument is the entity's type name, as in reactCapability:

BuilderSymbolFile
tanstackRouterCapability.list("Users")UsersListRoutetanstack-router.tsx
tanstackRouterCapability.detail("Users")UsersDetailRoutetanstack-router.tsx
tanstackRouterCapability.new("Users")UsersCreateFormRoutetanstack-router.tsx
tanstackRouterCapability.edit("Users")UsersUpdateFormRoutetanstack-router.tsx
tanstackRouterCapability.contextRouterContextrouter/root.tsx
tanstackRouterCapability.rootrootRouterouter/root.tsx
tanstackRouterCapability.treerouteTreerouter/tree.tsx

The output decides which pages exist with viewNames, as the react section above says. The client's type is clientCapability.type (client:type, the Client that createClient returns).

A link phase asks for symbols that other outputs declared:

const link = Effect.gen(function* () {
const registry = yield* SymbolRegistry
yield* registry.import("effect:schema:Users", { into: "effect:query:findUsersById", type: true })
return []
})
  • registry.import(capability, { into, type? }) records that the symbol with capability into uses the symbol capability. Imports belong to symbols, so each file imports exactly what its own symbols use under any layout. It fails with UnknownCapability when no selected output declared capability or into. The registry resolves both files through the layout and writes the import declaration with the right relative path.
  • Every reference from one declared symbol to another goes through registry.import, even when both sit in one file under byKind(). The registry drops an import whose target is in the importing file, so that layout writes no self-import. A reference with no import works only while the layout keeps both symbols in one file. A Cannot find name X error from a generated project means a reference with no link: add the import in the output that declares the user of X.
  • registry.importOptional returns Option.none() instead of failing, for an output that works without the other output.
  • The generator reads registry.documents after the link phase. An output does not call it.

Write a layout​

A layout decides the path of each symbol's file. It is a Layout service: a function from a Placement ({ owner, file, entity }) to an output-relative path. The default is by table: each entity's symbols go under <schema>/<table>/<file>, and symbols with no entity stay in file. The byKind() plugin selects the by-kind layout, where the path is file.

import { Layer } from "effect"
import { Layout, Plugin } from "pg-sourcerer"

export const byOwner = (): Plugin =>
Plugin.make("layout").pipe(Plugin.layer(Layer.succeed(Layout, ({ owner, file }) => `${owner}/${file}`)))
  • The plugin is named layout. Two layout plugins in one config fail with PluginSelectionError.
  • The registry reads Layout when the run starts, so listing the plugin in the config is enough. No other plugin requires it.
  • The layout must be pure and give two symbols one path only when they may share a file: the registry rejects two symbols of one name in one path.
  • Keep the path relative, normalized and ending in .ts. The generator validates the files it writes.
  • openapi.json is a document an output returns, not a symbol, so it stays at the output root under every layout.
  • Switching layouts moves files. The merge publish keys its manifest by path, so a project that switches layouts sees its files appear at the new paths. Edits a user made in the files at the old paths stay where they are.

Errors​

An output may fail with any tagged error of its own. The generator wraps each failure in OutputFailed { output, error }, where output is the output's name and error is the original tagged error. The CLI prints OutputFailed: output "<name>" failed, then one line with the error's tag and its message. An error must therefore have a message built from its typed fields. A Schema.TaggedError has an empty message by default, so override the getter:

class MyOutputError extends Schema.TaggedError<MyOutputError, { readonly brand: unique symbol }>()(
"MyOutputError",
{ issue: Schema.String },
) {
override get message(): string {
return this.issue
}
}

No union lists the errors of the outputs, so register nothing. Errors from the registry (SymbolError) pass through the same way.

Expose the output​

Build a Plugin in steps and return it from a constructor:

const typesPlugin = Plugin.make("types").pipe(Plugin.output(typesOutput))

export const types = (): Plugin => typesPlugin

Plugin.make(name) starts a plugin with no output and no services. Each step after it adds one thing, and the compiler checks every step against what the earlier steps gave the plugin:

  • Plugin.requires(other) adds a plugin the generator runs with this one, and the services other holds.
  • Plugin.layer(layer) adds services the plugin owns. The layer may read the services the plugin already holds.
  • Plugin.output(output) sets the output. It may read the core services and the services the plugin holds. An output that reads a service no step gave the plugin is a type error.

The name identifies the plugin in failure reports and owns its capabilities. A user adds types() to plugins in pgsourcerer.config.ts. A constructor without options returns one value each time, so two plugins that require types() hold the same plugin and the generator runs it once.

The generator resolves the plugins in the config and every plugin they require into one plugin per name:

  • A plugin you list replaces every required plugin of the same name. Files link by name, so the dependents link to yours.
  • The services of your plugin override the services of the replaced one, by tag, wherever a dependent reads them: in its layer and in its output. A service yours does not provide still comes from the replaced plugin. A dependent's own Plugin.layer step still wins over both.
  • Your plugin may require the plugin it replaces and add to it. Its own steps read the replaced plugin; every other plugin reads yours.
  • Give each layer object to one plugin's Plugin.layer step. Effect builds a layer object once per run, so to share a service, require the plugin that holds it.
  • Two different required plugins of one name fail with PluginConflict. List one of them in the config to choose it.
  • Two entries of one name in the config fail with PluginSelectionError.

To also select the plugin with --plugins types, add the name to BuiltinPluginName and pluginByName in src/pipeline.ts, and export the constructor from src/index.ts. A plugin that lives outside this package needs only the constructor in plugins; --plugins selects built-in plugins only, and it replaces the plugins of the config file.

Validator outputs​

The zod, valibot and arktype outputs share one walk (src/validators/shared.ts). A dialect says how a library spells a scalar, enum, array, nullable value and object. The walk reads Shapes, so each schema has the fields of the types interface of the same name. Capabilities are <name>:schema:<Export> and <name>:enum:<Name>. The schemas validate the value a pg driver returns, for example a Date for a timestamp.

Each schema comes with a type of the same name. For the fixture's Username domain, zod.ts holds:

export const Username = z.string().min(2).max(24).regex(new RegExp("^[a-zA-Z][a-zA-Z0-9_-]+$", "i"));
export type Username = z.infer<typeof Username>;

The typia output is different. typia builds a validator from a TypeScript type, so the output imports the interfaces of the types output (registry.import, type-only) and fails with UnknownCapability when types is not selected. The generated typia.ts runs only after the typia transformer rewrites it. Compile it with ttsc or run it with ttsx, with the plugin { "transform": "typia/lib/transform" } in a tsconfig that holds only the files typia needs (see packages/generated-check/tsconfig.typia.json). tsc type-checks the file but does not transform it. The ttsc build compiles a Go plugin and needs a Go toolchain; its cache is large. check:validators keeps the cache in ~/.cache/pg-sourcerer/ttsc (or under XDG_CACHE_HOME), TTSC_CACHE_DIR moves it, and the script stops with a message when go is not on PATH.

Test the output​

  • Run the plugin with generatePlugins(snapshot, "visitor", [], [yourPlugin()]) on a small CatalogSnapshot and check the document paths and a few lines of source (see test/pipeline.test.ts).
  • Add the output to packages/generated-check/pgsourcerer.config.ts. bun run typecheck then compiles its files with the real compiler.
  • When two outputs must agree, add a type-level assertion to packages/generated-check/assertions/. types-equal-schemas.ts shows that every plain type equals typeof Schema.Type of the same name. A mismatch is a type error.