lpts
Optimized-plan inspection and cross-system SQL transpilation
Maintainer(s): ila
Installing and Loading
INSTALL lpts FROM community;LOAD lpts;Example
-- Set both input and output dialects to DuckDB for a first round trip.D SET lpts_input_dialect = 'duckdb';D SET lpts_dialect = 'duckdb';
D CREATE TABLE users (id INTEGER, name VARCHAR, age INTEGER);D INSERT INTO users VALUES (1, 'Alice', 30), (2, 'Bob', 22), (3, 'Carol', 28);
-- Return the optimized plan as readable CTE SQL.D PRAGMA lpts('SELECT name FROM users WHERE age > 25');WITHt0_scan (t0_name) AS ( SELECT "name" FROM memory.main.users WHERE (age>25))SELECT t0_name AS "name"FROM t0_scan;
-- The same output, exposed as a first-class EXPLAIN statement.D EXPLAIN (FORMAT SQL) SELECT name FROM users WHERE age > 25;┌─────────────────────────────┐│┌───────────────────────────┐│││ Optimized Logical Plan │││└───────────────────────────┘│└─────────────────────────────┘WITHt0_scan (t0_name) AS ( SELECT "name" FROM memory.main.users WHERE (age>25))SELECT t0_name AS "name"FROM t0_scan;
-- Turn on transparent round-trip checking. Every top-level SELECT is now-- rewritten by LPTS and its result compared against the original; a wrong-- rewrite raises an error. The query otherwise returns its normal rows.D SET lpts_check = true;D SELECT name FROM users WHERE age > 25;┌─────────┐│ name ││ varchar │├─────────┤│ Alice ││ Carol │└─────────┘About lpts
LPTS is a DuckDB extension for optimized-plan inspection and cross-system SQL transpilation. LPTS takes DuckDB’s post-optimizer logical plan and reconstructs equivalent SQL as a sequence of named CTEs.
PRAGMA Syntax
PRAGMA lpts('<query>');EXPLAIN (FORMAT SQL)
LPTS also extends DuckDB’s EXPLAIN with a SQL format. EXPLAIN (FORMAT SQL) <query> returns the optimized logical plan rendered as equivalent CTE SQL — the
same output as PRAGMA lpts, but exposed as a first-class EXPLAIN statement.
The CLI prints the SQL as plain multi-line text, while JDBC, Python, and other
clients receive the standard two-column (explain_key, explain_value) EXPLAIN
result. It honors lpts_dialect just like PRAGMA lpts.
EXPLAIN (FORMAT SQL) SELECT name FROM users WHERE age > 25;┌─────────────────────────────┐│┌───────────────────────────┐│││ Optimized Logical Plan │││└───────────────────────────┘│└─────────────────────────────┘WITHt0_scan (t0_name) AS ( SELECT "name" FROM memory.main.users WHERE (age>25))SELECT t0_name AS "name"FROM t0_scan;LPTS plans the query through DuckDB, optimizes it, then serializes the optimized logical plan.
Supported Dialects
The dialect settings accept these values:
| Dialect | Accepted values |
|---|---|
| DuckDB | duckdb |
| PostgreSQL | postgres, postgresql |
| Spark SQL | spark |
| Hive | hive |
| Trino / Presto | trino, presto |
| Snowflake | snowflake |
| BigQuery | bigquery, bq |
| Redshift | redshift |
| MySQL / MariaDB | mysql, mariadb |
Use Cases
- Inspect optimized DuckDB plans as SQL.
- Debug optimizer rewrites such as filter pushdown, join reordering, top-N, materialized CTEs, and subquery decorrelation.
- Generate a CTE program that communicates the optimized execution shape.
- Emit SQL for another engine with
lpts_dialect. - Convert other SQL dialect syntax to DuckDB SQL with
lpts_input_dialect, then execute or inspect it.
Supported Operators
LPTS is intended to cover all logical operators produced by optimized DuckDB SELECT plans. The current regression suite round-trips all 22 TPC-H queries and exercises joins, aggregates, windows, set operations, CTEs, recursive CTEs, table functions, DuckLake scans, and inserts.
Unsupported optimizer edge cases fail explicitly with NotImplementedException.
Examples
D CREATE TABLE events (id INTEGER, ts TIMESTAMP, name VARCHAR, "order" INTEGER);D INSERT INTO events VALUES (1, TIMESTAMP '2024-01-15 08:09:10', 'alpha', 10), (11, TIMESTAMP '2024-01-16 11:12:13', 'beta', 20);
-- Render generated SQL for PostgreSQL.D SET lpts_dialect = 'postgres';
-- Return generated CTE SQL directly in the shell.D PRAGMA lpts( 'SELECT strftime(ts, ''%Y-%m-%d'') AS day FROM events WHERE id > 10 ORDER BY day');
-- Return generated CTE SQL as a table row, useful in scripts and tests.D SELECT sqlFROM lpts_query( 'SELECT strftime(ts, ''%Y-%m-%d'') AS day FROM events WHERE id > 10 ORDER BY day');WITHt0_scan (ts) AS ( SELECT ts FROM events WHERE (id>10))SELECT to_char(ts, 'YYYY-MM-DD') AS "day"FROM t0_scanORDER BY to_char(ts, 'YYYY-MM-DD') ASC NULLS LAST;-- Switch back to DuckDB rendering.D SET lpts_dialect = 'duckdb';
-- Turn on transparent round-trip checking, then run the query directly.-- LPTS rewrites every top-level SELECT and compares its result bag against-- the original, raising on a mismatch; the query returns its normal rows.D SET lpts_check = true;D SELECT name FROM events WHERE id > 10 ORDER BY name;┌─────────┐│ name ││ varchar │├─────────┤│ beta │└─────────┘-- Print the AST tree to stdout for interactive debugging.D PRAGMA print_ast('SELECT name FROM events WHERE id > 10 ORDER BY name');
-- Return the AST tree as a table row, useful for tools and regression tests.D SELECT astFROM print_ast_query('SELECT name FROM events WHERE id > 10 ORDER BY name');-- Normalize MySQL syntax before DuckDB parses and plans the query.D SET lpts_input_dialect = 'mysql';
-- Return source-dialect SQL normalized to DuckDB SQL.D SELECT sqlFROM lpts_normalize_query( 'SELECT `order`, DATE_FORMAT(ts, ''%Y-%m-%d %H:%i:%s'') AS formatted FROM events LIMIT 5, 10');SELECT "order", strftime(ts, '%Y-%m-%d %H:%M:%S') AS formatted FROM events LIMIT 10 OFFSET 5Added Functions
| function_name | function_type | description | comment | examples |
|---|---|---|---|---|
| lpts | pragma | Return the optimized logical plan for a query as equivalent CTE SQL. | NULL | [PRAGMA lpts(‘SELECT name FROM users WHERE age > 25’)] |
| lpts_normalize_query | table | Normalize SQL from lpts_input_dialect into DuckDB SQL without planning it. | NULL | [SET lpts_input_dialect = ‘mysql’; SELECT sql FROM lpts_normalize_query(‘SELECT order FROM users LIMIT 1, 2’)] |
| lpts_query | table | Table-function form of PRAGMA lpts. Returns the generated CTE SQL as a single sql column. | NULL | [SELECT sql FROM lpts_query(‘SELECT name FROM users WHERE age > 25’)] |
| print_ast | pragma | Print the LPTS AST tree for a query to stdout. | NULL | [PRAGMA print_ast(‘SELECT name FROM users WHERE age > 25’)] |
| print_ast_query | table | Table-function form of PRAGMA print_ast. Returns the rendered AST tree as a single ast column. | NULL | [SELECT ast FROM print_ast_query(‘SELECT name FROM users WHERE age > 25’)] |
Overloaded Functions
This extension does not add any function overloads.
Added Types
This extension does not add any types.
Added Settings
| name | description | input_type | scope | aliases |
|---|---|---|---|---|
| lpts_check | When true, transparently verify every top-level SELECT: the query returns its normal result, but LPTS rewrites it and, on a different result, raises ‘LPTS check failed’ (an unsupported query raises a specific ‘LPTS check: unsupported’ error). If the LPTS_CHECK_LOG environment variable points to a file, it instead logs one ‘<query#> OK|WRONG|UNSUPPORTED|FAIL’ line per SELECT and never raises (suite-coverage mode). | BOOLEAN | GLOBAL | [] |
| lpts_dialect | SQL dialect for lpts output. Valid values: ‘duckdb’ (default), ‘postgres’, ‘spark’, ‘hive’, ‘trino’, ‘presto’, ‘snowflake’, ‘bigquery’, ‘redshift’, ‘mysql’, ‘mariadb’ | VARCHAR | GLOBAL | [] |
| lpts_enable_data_dependent_optimizers | Enable LPTS planning optimizers that depend on current data, statistics, cardinality estimates, row groups, or runtime dynamic filters. | BOOLEAN | GLOBAL | [] |
| lpts_input_dialect | SQL dialect for lpts input normalization. Valid values: ‘duckdb’ (default), ‘postgres’, ‘spark’, ‘hive’, ‘trino’, ‘presto’, ‘snowflake’, ‘bigquery’, ‘redshift’, ‘mysql’, ‘mariadb’ | VARCHAR | GLOBAL | [] |
| lpts_merge_pipeline | Fuse chains of single-child pipeline operators (Limit/OrderBy/Project/Aggregate/Filter, and pushdown-free base-table scans) into one flat SELECT per query block instead of one CTE per operator. | BOOLEAN | GLOBAL | [] |