API Reference

API Reference#

Transpilation#

giql.transpile.transpile(giql: str, tables: list[str | Table] | None = None, *, dialect: Literal['datafusion'] | None = None) → str[source]#

giql.transpile.transpile(giql: str, tables: list[str | Table] | None = None, *, dialect: Literal['duckdb']) → str

giql.transpile.transpile(giql: str, tables: list[str | Table] | None = None, *, dialect: str) → str

Transpile a GIQL query to SQL.

Parses the GIQL syntax and converts it to portable SQL for the active target (uses LATERAL joins where needed for operations like NEAREST). The output is not strictly SQL-92: depending on the target it may use engine extensions such as LATERAL or SELECT * EXCEPT (see the dialect parameter).

Parameters#

giqlstr: The GIQL query string containing genomic extensions like INTERSECTS, CONTAINS, WITHIN, CLUSTER, MERGE, NEAREST, or DISJOIN.
tableslist[str | Table] | None: Table configurations. Strings use default column mappings (chrom, start, end, strand). Table objects provide custom column name mappings.
dialectstr | None: Optional target engine. Resolves to a giql.targets.Target carrying the engine’s capability set; None selects the generic portable target, "duckdb" / "datafusion" the built-in targets, and any other name a custom giql.targets.Target registered on the plugin hub (see giql.expander.register() / giql.expander.ExpanderRegistry.register_target()). When set to "duckdb", column-to-column INTERSECTS joins (INNER, SEMI, or ANTI) are transpiled into a per-chromosome dynamic-SQL pattern (SET VARIABLE + query(getvariable(...))) that DuckDB plans through its range-join family (IE_JOIN / PIECEWISE_MERGE_JOIN). Shapes the IEJoin path declines (LEFT/RIGHT/FULL joins, self-joins, multiple INTERSECTS, extra predicates, non-base operands, 3+ tables) fall through to the generic naive overlap predicate. "datafusion" and None always emit that naive predicate — a plain ON a.chrom = b.chrom AND a.start < b.end AND b.start < a.end condition the engine’s own optimizer plans as a range join — for both inner and outer column-to-column INTERSECTS joins. Hard-error projection shapes raise ValueError at transpile time; see the performance guide for the full enumeration. The target’s capabilities also choose the coordinate-canonicalization emit form for a non-canonically-encoded table: "duckdb" emits SELECT * REPLACE (...), while the generic (None) and "datafusion" targets emit the portable SELECT * EXCEPT (...), <start>, <end> form, which runs on * EXCEPT-capable engines (the DataFusion family) but not on DuckDB — pass dialect="duckdb" for DuckDB-runnable output. Tables in the canonical 0-based half-open encoding are unaffected (they emit portable SQL on every target).

Returns#

str: The transpiled SQL query.

Raises#

ValueError: If the query cannot be parsed or transpiled, or if dialect is unknown.

Examples#

Basic usage with default column mappings:

sql = transpile(
    "SELECT * FROM peaks WHERE interval INTERSECTS 'chr1:1000-2000'",
    tables=["peaks"],
)

Custom Table configuration:

sql = transpile(
    "SELECT * FROM peaks WHERE interval INTERSECTS 'chr1:1000-2000'",
    tables=[
        Table(
            "peaks",
            genomic_col="interval",
            chrom_col="chrom",
            start_col="start",
            end_col="end",
        )
    ],
)

Column-to-column INTERSECTS join (naive overlap predicate; inner or outer, planned as a range join by the target engine):

sql = transpile(
    "SELECT a.*, b.* FROM peaks a JOIN genes b "
    "ON a.interval INTERSECTS b.interval",
    tables=["peaks", "genes"],
)

DuckDB IEJoin dialect (column-to-column INNER/SEMI/ANTI JOIN only; projections must be qualified):

sql = transpile(
    "SELECT a.chrom, a.start, b.start "
    "FROM peaks a JOIN genes b ON a.interval INTERSECTS b.interval",
    tables=["peaks", "genes"],
    dialect="duckdb",
)

class giql.table.Table(name, genomic_col='interval', chrom_col='chrom', start_col='start', end_col='end', strand_col='strand', coordinate_system='0based', interval_type='half_open')[source]#

Genomic table configuration for transpilation.

This class defines how genomic intervals are stored in a database table, mapping a pseudo-column name (genomic_col) to the physical columns that store chromosome, start, end, and optionally strand information.

Parameters#

namestr: The table name.
genomic_colstr: The pseudo-column name used in GIQL queries to reference the genomic interval (default: “interval”).
chrom_colstr: The physical column name storing chromosome/contig (default: “chrom”).
start_colstr: The physical column name storing interval start position (default: “start”).
end_colstr: The physical column name storing interval end position (default: “end”).
strand_colstr | None: The physical column name storing strand information, or None if the table has no strand column (default: “strand”).
coordinate_systemLiteral[“0based”, “1based”]: The coordinate system used for positions (default: “0based”).
interval_typeLiteral[“half_open”, “closed”]: The interval endpoint convention (default: “half_open”).

Examples#

Using default column names (via transpile):

sql = transpile(query, tables=["peaks"])

Mixing default and custom table configurations:

sql = transpile(
    query,
    tables=[
        "peaks",
        Table(
            "variants",
            genomic_col="position",
            chrom_col="chr",
            start_col="pos_start",
            end_col="pos_end",
            strand_col=None,  # No strand column
            coordinate_system="1based",
            interval_type="closed",
        ),
    ],
)

name: str#

genomic_col: str = 'interval'#

chrom_col: str = 'chrom'#

start_col: str = 'start'#

end_col: str = 'end'#

strand_col: str | None = 'strand'#

coordinate_system: Literal['0based', '1based'] = '0based'#

interval_type: Literal['half_open', 'closed'] = 'half_open'#

__post_init__()[source]#

Validate field values after initialization.

__init__(name, genomic_col='interval', chrom_col='chrom', start_col='start', end_col='end', strand_col='strand', coordinate_system='0based', interval_type='half_open')#

Extension hook#

The registry-based extension point for custom targets and operator expanders (see Extending GIQL: custom targets and operators). Every symbol below is re-exported from the top-level giql package.

giql.expander.register(target, operator)[source]#

Register an operator expander for (target, operator) — the extension hook.

The public decorator by which a built-in or user-supplied expander is added to the process-wide REGISTRY:

@register(DuckDBTarget, GIQLDisjoin)
def expand_disjoin(node, ctx): ...

Parameters#

targettype[Target] | Target: The target the expander handles, given as the target class (the usual form — it is instantiated with its default capabilities) or an explicit instance. Pass GenericTarget for the portable expander every target falls back to.
operatortype: The GIQL operator expression class the expander handles.

Returns#

Callable: A decorator that registers its argument and returns it unchanged, so the decorated object stays usable on its own.

giql.expander.REGISTRY = <giql.expander.ExpanderRegistry object>#: The process-wide registry the register() decorator writes to and the ExpandOperators pass reads from. The built-in expanders register into it at import time via giql.expanders; the pass rewrites every GIQL operator node by resolving its expander here.

class giql.expander.ExpanderRegistry[source]#

Maps (target, operator_type) to the expander that handles it.

Resolution follows the epic’s fallback chain so a generic expander written once covers every target, and a per-target entry overrides it only where the engine genuinely differs:

(target, op) — an expander registered for the exact active target;
(generic, op) — the portable expander registered for GenericTarget;
None — no expander resolves. Every built-in operator registers a (generic, op) expander, so this only arises for a cleared registry or an operator with no generic fallback; the pass raises rather than emitting (there is no legacy *_sql fallback).

The registry keys on the frozen, value-equal Target instance (two DuckDBTarget() compare and hash alike), so registering against DuckDBTarget and resolving against a freshly built DuckDBTarget() hit the same entry. Registering against the class DuckDBTarget is a convenience the decorator instantiates.

__init__()[source]#

register(target, operator, expander)[source]#

Register expander for (target, operator), overriding any prior.

Parameters#

targetTarget: The target instance the expander handles. Use GenericTarget for the portable fallback.
operatortype: The GIQL operator expression class (e.g. GIQLDisjoin).
expanderOperatorExpander | ExpanderFn: The expander object or function. A later registration for the same key replaces an earlier one (last-write-wins override).

Notes#

Registering an expander also declares target by name (see register_target()), so a custom target becomes selectable via transpile(dialect=target.name) as soon as any expander is registered against it — a user overriding one operator need not also declare the target separately.

A later registration for the same (target, operator) key replaces the earlier one (last-write-wins), so a user override of a built-in target-specific expander — notably the DuckDB IEJoin (DuckDBTarget, Intersects) join rewrite (giql.expanders.intersects_duckdb) — simply supersedes it in the registry; the pass then dispatches the operator node to the user’s expander instead. (has_override() reports whether such an exact non-generic entry exists for introspection.)

register_target(target)[source]#

Declare target so transpile(dialect=target.name) resolves it.

The registry doubles as the target plugin hub: a custom Target registered here (directly, or as a side effect of register()) is resolvable by name through giql.targets.resolve_target(). This is the path for a capability-only target — one that overrides no operators (so it never calls register()) but declares a distinct Capabilities set / sqlglot_dialect.

The built-in GenericTarget is intentionally not declared: it is the portable fallback, selected only via dialect=None (see giql.targets.resolve_target()), never by name. The skip is by value (target == GenericTarget()), so registering an expander against GenericTarget is a no-op here — but a custom Target subclass is declared under its own name even if that name happens to be "generic" (it does not compare equal to the built-in). A later registration for the same name replaces an earlier one.

target(name)[source]#

Return the registered custom target named name, or None.

Consulted by giql.targets.resolve_target() for any dialect name that is not a built-in, so a registered custom target is selectable via transpile(dialect=name).

resolve(target, operator)[source]#

Return the expander for (target, operator) via the fallback chain.

Tries the exact (target, op) entry, then the (GenericTarget(), op) fallback, then None. None means no expander is registered; the ExpandOperators pass treats that as an internal invariant violation and raises (there is no legacy emitter).

Resolution is the single dispatch path for every operator, including the DuckDB IEJoin join rewrite: on duckdb this returns the built-in (DuckDBTarget, Intersects) override (giql.expanders.intersects_duckdb), and a user registration for the same key replaces it (last-write-wins) so this returns the user’s expander instead (#169). There is no separate pre-pass for INTERSECTS anymore.

has_override(target, operator)[source]#

Whether an exact non-generic (target, operator) entry is registered.

Returns True only when target is not GenericTarget and an exact (target, operator) entry is registered; the portable (GenericTarget(), operator) fallback is not an override and does not count here.

Such an entry marks a target-specific override that supersedes the (GenericTarget, operator) fallback — for example the built-in DuckDB IEJoin (DuckDBTarget, Intersects) join rewrite (giql.expanders.intersects_duckdb), or a plugin’s own override of it. This is an introspection helper (the dispatch itself flows through resolve()); it is not a gate on any pre-pass, since INTERSECTS join strategy is fully registry-driven (#169) with no pre-pass to bypass.

unregister(target, operator)[source]#

Drop the (target, operator) entry if present.

Part of the registry’s public teardown seam: a caller that registered a custom expander (a plugin, or a test fixture) undoes one registration with this rather than mutating private state. Resolving the same key afterward falls back through the chain to None. Dropping an absent key is a no-op.

clear()[source]#

Drop every registration — both expanders and declared targets.

The bulk form of unregister(), and the public reset for the process-wide REGISTRY — e.g. a test fixture that saves and restores the registry around a body that registers custom expanders or targets.

snapshot()[source]#

Return a shallow copy of the current registrations and targets.

The save half of a save/restore seam that supports test baseline-isolation: capture the baseline with this and hand it back to restore() afterward, so the built-in registrations survive an isolating fixture that would otherwise clear() them permanently.

The returned RegistrySnapshot is a fresh, opaque value bundling both the (target, operator) expander map and the declared-target map; mutating the registry afterward does not affect it, and two snapshots compare equal when the registry is in the same state.

restore(snapshot)[source]#

Replace all registrations with those captured by snapshot().

The restore half of the save/restore seam that supports test baseline-isolation. Drops every current entry and re-installs exactly the snapshot contents (expanders and declared targets), so a fixture can return the registry to a previously captured baseline regardless of what its body registered or cleared.

__contains__(key)[source]#

Whether an exact (target, operator) entry is registered.

Tests the exact key only — it does not walk the generic fallback chain resolve() follows ((GenericTarget(), op) in registry is the only way (target, op) reports present via the generic entry). Part of the public teardown seam, for asserting a registration landed or was torn down.

__len__()[source]#

The number of registered (target, operator) entries.

Part of the public introspection surface (alongside in): emptiness is observable without reaching into private state, so a teardown fixture or leak guard can assert the registry is clear via len(registry) or bool(registry).

__bool__()[source]#

Whether any expander is registered (False when empty).

class giql.expander.RegistrySnapshot(expanders, targets)[source]#

An opaque save/restore token for ExpanderRegistry state.

Bundles the (target, operator) expander map and the declared-target map captured by ExpanderRegistry.snapshot(). Hand it back to ExpanderRegistry.restore() to return the registry to that state, or compare two snapshots for equality to assert the registry returned to a baseline (the leak-guard use).

A snapshot is compared by equality and handed to restore() only; it is not hashable (its two dict fields are mutable), so hash(snapshot) raises — never use one as a set member or dict key.

expanders: dict[tuple[Target, type], Callable[[Expression, ExpansionContext], Expression]]#

targets: dict[str, Target]#

__init__(expanders, targets)#

class giql.expander.ExpansionContext(node, resolution, target, tables, alias_seq=None, registry=None, finalizers=None)[source]#

Everything an OperatorExpander needs to expand one operator.

A fresh context is built per operator node by ExpandOperators. It bundles the node’s pass-1 resolution metadata together with the active target, collision-safe alias minting, and the registered tables, so an expander never has to reach back into the pass or re-resolve anything.

Attributes#

nodeexp.Expression: The GIQL operator node being expanded.
resolutionOperatorResolution | None: The metadata giql.resolver.resolve_operator_refs() attached to node (its resolved reference slots, column operands, deferrals, and de-canonicalization output tables). None only if the node was never annotated — an internal invariant violation an expander may assert on.
targetTarget: The active Target, exposing its capabilities and sqlglot_dialect.
tablesTables: The registered Tables container.
registryExpanderRegistry | None: The registry the pass is resolving against. Carried so a whole-query rewrite (CLUSTER / MERGE) can re-enter expand_operators() over the SELECT it just restructured and expand sibling operators it copied into it, honoring a custom-registry pass run. None for a standalone context built outside the pass.

A node-local expander that needs to rewrite the enclosing statement (rather than just replace its own node) registers a StatementFinalizer via add_statement_finalizer(); the pass applies every finalizer to the statement after all node-local replacements. This is the query-level seam for a target whose expansion must reshape the enclosing statement — for example to project internal helper columns out of a surfacing SELECT *.

__init__(node, resolution, target, tables, alias_seq=None, registry=None, finalizers=None)[source]#

node#

resolution#

target#

tables#

registry#

property capabilities#: The active target’s Capabilities.

add_statement_finalizer(finalizer)[source]#

Register a query-level StatementFinalizer for this run.

The query-level seam: an expander is node-local — its return value replaces only its own node — so a target that must rewrite the enclosing statement (for example to project internal helper columns out of a surfacing SELECT *) registers a finalizer here instead. expand_operators() applies every registered finalizer to the statement, in registration order, after all node-local replacements complete; each receives the current statement root and returns the (possibly new) root.

A finalizer’s returned root is emitted verbatim — beyond a type check that it is an Expression, it is not semantically re-validated — so it must not reference columns or relations absent from the projection it rewrites: a wrapper over an absent column builds without error but fails at engine runtime. Finalizers registered on a standalone context (one built outside the pass) are collected but never applied.

alias()[source]#

Mint a fresh, query-unique alias with the reserved expander prefix.

Draws sequential names from a per-run sqlglot.helper.name_sequence() so two expanders (or two slots of one expander) never collide.

class giql.expander.OperatorExpander(*args, **kwargs)[source]#

Turns one GIQL operator node into standard sqlglot AST for a target.

An expander is any callable-bearing object whose expand() takes the operator node and its ExpansionContext and returns the sqlglot AST the node expands to for the active target. The returned expression replaces the operator node in the tree; the stock per-target serializer renders it.

The protocol is runtime_checkable(), so the registry can assert a registered object satisfies it. A plain function is not an OperatorExpander (it has no expand method); register one by wrapping it (see register(), which accepts either form).

An expander’s return value is node-local: expand(node, ctx) -> exp.Expression returns the one expression that replaces the operator node in place. When a target additionally needs to rewrite the enclosing statement — for example to project internal helper columns away from a surfacing SELECT * — the expander registers a query-level StatementFinalizer via ExpansionContext.add_statement_finalizer(), applied to the statement after every node-local replacement. The DuckDB INTERSECTS IEJoin whole-query fold uses exactly this seam: its (DuckDBTarget, Intersects) expander (giql.expanders.intersects_duckdb, #169) registers a finalizer that replaces the statement root with the per-chromosome dynamic-SQL rewrite.

expand(node, ctx)[source]#

__init__(*args, **kwargs)#

Targets and capabilities#

class giql.targets.Target(name, sqlglot_dialect, capabilities)[source]#

A SQL target engine.

Subclasses declare the engine name, the sqlglot_dialect used to serialize AST for that engine (None selects sqlglot’s default generic serialization), and the engine capabilities.

Targets are frozen, value-equal, and hashable: two DuckDBTarget() instances compare equal and hash alike, so the operator-expander registry (#138) can key on a resolved target by value. Equality is class-scoped — GenericTarget() != DataFusionTarget() even where their fields overlap.

name: str#

sqlglot_dialect: str | None#

capabilities: Capabilities#

__init__(name, sqlglot_dialect, capabilities)#

class giql.targets.Capabilities(supports_lateral, supports_star_replace, supports_qualify)[source]#

Feature set of a SQL target engine.

Each field is a portable choice that later steps of epic #137 turn into a capability lookup instead of a hardcoded dialect branch.

Parameters#

supports_lateralbool: Whether the engine supports LATERAL / correlated joins. Drives the NEAREST LATERAL-vs-window-function strategy (#142): a correlated NEAREST expands to a portable correlated LATERAL subquery where this holds and to a decorrelated window-function form where it does not. This capability is the single source of truth — the former generator-level SUPPORTS_LATERAL attribute has been removed.
supports_star_replacebool: Whether the engine supports SELECT * REPLACE (...). Drives every coordinate-canonicalization site — the canonicalizer wrapper CTE and the DISJOIN / NEAREST passthroughs (#143/#145): * REPLACE where supported, the portable * EXCEPT projection otherwise. Supported by DuckDB / BigQuery / Snowflake / ClickHouse; not by PostgreSQL, SQLite, or DataFusion.
supports_qualifybool: Whether the engine supports the QUALIFY clause. Reserved: no emission path consumes it yet (a future window-function operator port would).

supports_lateral: bool#

supports_star_replace: bool#

supports_qualify: bool#

__init__(supports_lateral, supports_star_replace, supports_qualify)#

class giql.targets.GenericTarget(name='generic', sqlglot_dialect=None, capabilities=Capabilities(supports_lateral=True, supports_star_replace=False, supports_qualify=False))[source]#

Portable SQL-92-ish target with no engine-specific features.

This is the default target (dialect=None). Its capabilities are the conservative, maximally portable baseline, serialized through sqlglot’s dialect-less default (sqlglot_dialect = None).

“SQL-92-ish”, not strict SQL-92: because supports_star_replace=False, every coordinate-canonicalization site over a non-canonical target falls back to a SELECT * EXCEPT (...) projection (re-appending the recomputed interval columns) — the canonicalizer wrapper CTE and the DISJOIN / NEAREST passthroughs alike. A star-projected CLUSTER is a further * EXCEPT site (independent of canonicalization): it hides its synthesized __giql_is_new_cluster flag with * EXCEPT (#184). * EXCEPT is not SQL-92 and is not DuckDB-runnable — it is a DataFusion-family extension — so the generic target’s output from any of these sites runs only on an * EXCEPT-capable engine (transpile with dialect="duckdb" to execute on DuckDB, where it spells the exclusion EXCLUDE). A canonical (0-based half-open) target with no star-projected CLUSTER passes the row through as a plain, fully portable SELECT *.

name: str = 'generic'#

sqlglot_dialect: str | None = None#

capabilities: Capabilities = Capabilities(supports_lateral=True, supports_star_replace=False, supports_qualify=False)#

__init__(name='generic', sqlglot_dialect=None, capabilities=Capabilities(supports_lateral=True, supports_star_replace=False, supports_qualify=False))#

class giql.targets.DuckDBTarget(name='duckdb', sqlglot_dialect='duckdb', capabilities=Capabilities(supports_lateral=True, supports_star_replace=True, supports_qualify=True))[source]#

DuckDB target.

Serializes through sqlglot’s duckdb dialect and uses the IEJoin per-partition plan for column-to-column INTERSECTS joins, registered as the (DuckDBTarget, Intersects) override in the operator-expander registry (giql.expanders.intersects_duckdb, #169).

Serializing through the duckdb dialect makes null ordering explicit on ORDER BY / window terms, emitting NULLS FIRST where the generic (dialect-less) serialization leaves it implicit. This is result-preserving for every migrated operator because each sorts on a non-null coordinate key (chromosome / position, or a same-chromosome-filtered distance), so the explicit NULLS FIRST never reorders a NULL into or out of the top rows. A future operator ordering on a nullable key would need to re-verify this before relying on the DuckDB serialization being a semantic no-op.

name: str = 'duckdb'#

sqlglot_dialect: str | None = 'duckdb'#

capabilities: Capabilities = Capabilities(supports_lateral=True, supports_star_replace=True, supports_qualify=True)#

__init__(name='duckdb', sqlglot_dialect='duckdb', capabilities=Capabilities(supports_lateral=True, supports_star_replace=True, supports_qualify=True))#

class giql.targets.DataFusionTarget(name='datafusion', sqlglot_dialect=None, capabilities=Capabilities(supports_lateral=False, supports_star_replace=False, supports_qualify=False))[source]#

Apache DataFusion target.

sqlglot has no DataFusion dialect, so serialization uses the generic form (sqlglot_dialect = None); this is the finalized strategy (#145) — the portable SQL the generic generator emits runs on DataFusion, verified end-to-end by the cross-target oracle (tests/integration/datafusion/): every operator at the default encoding, and the canonicalizing operators (DISJOIN / NEAREST) across all four coordinate encodings plus custom-column and strand schemas.

The capability values below are validated against a real DataFusion engine by that oracle: supports_lateral=False (no correlated-LATERAL physical plan, so NEAREST takes the decorrelated window fallback), supports_star_replace= False (DataFusion supports * EXCEPT / * EXCLUDE but not * REPLACE, so the canonicalizer and the NEAREST/DISJOIN passthroughs emit the portable * EXCEPT form), and supports_qualify=False. DataFusion registers no INTERSECTS override, so a column-to-column INTERSECTS join falls back to the generic naive overlap predicate — a plain ON condition DataFusion plans as a hash join keyed on chrom with the position inequalities as a residual join filter (#167).

A SELECT * / SELECT b.* over a correlated NEAREST would otherwise expose the decorrelated fallback’s reserved __giql_x_* rank/key columns; a statement finalizer wraps the enclosing SELECT in SELECT * EXCEPT (...) on the fallback path (#160), so the cross-target identity claim holds for star projections over a correlated NEAREST unconditionally. The finalizer re-locates its target join by a reserved meta tag and mints its reserved column names per fallback (#172), so the claim now holds through the two former residual compositions too: a correlated NEAREST re-surfaced by an enclosing SELECT * outside its own SELECT (e.g. a wrapping CLUSTER, whose copy() + transplant the tag survives) and two correlated NEAREST fallbacks in one query (whose now-distinct per-run reserved names each * EXCEPT independently).

name: str = 'datafusion'#

sqlglot_dialect: str | None = None#

capabilities: Capabilities = Capabilities(supports_lateral=False, supports_star_replace=False, supports_qualify=False)#

__init__(name='datafusion', sqlglot_dialect=None, capabilities=Capabilities(supports_lateral=False, supports_star_replace=False, supports_qualify=False))#

API Reference

Contents

API Reference#

Transpilation#

Parameters#

Returns#

Raises#

Examples#

Parameters#

Examples#

Extension hook#

Parameters#

Returns#

Parameters#

Notes#

Attributes#

Targets and capabilities#

Parameters#