lpts
Inspektion optimierter Pläne und SQL-Transpilierung zwischen Systemen
Maintainer: ila
Installation und Laden
INSTALL lpts FROM community;LOAD lpts;Beispiel
-- Set both input and output dialects to DuckDB for a first round trip.D SET lpts_input_dialect = 'duckdb';D SET lpts_dialect = 'duckdb';
D CREATE TABLE users (id INTEGER, name VARCHAR, age INTEGER);D INSERT INTO users VALUES (1, 'Alice', 30), (2, 'Bob', 22), (3, 'Carol', 28);
-- Return the optimized plan as readable CTE SQL.D PRAGMA lpts('SELECT name FROM users WHERE age > 25');WITHt0_scan (t0_name) AS ( SELECT "name" FROM memory.main.users WHERE (age>25))SELECT t0_name AS "name"FROM t0_scan;
-- The same output, exposed as a first-class EXPLAIN statement.D EXPLAIN (FORMAT SQL) SELECT name FROM users WHERE age > 25;┌─────────────────────────────┐│┌───────────────────────────┐│││ Optimized Logical Plan │││└───────────────────────────┘│└─────────────────────────────┘WITHt0_scan (t0_name) AS ( SELECT "name" FROM memory.main.users WHERE (age>25))SELECT t0_name AS "name"FROM t0_scan;
-- Turn on transparent round-trip checking. Every top-level SELECT is now-- rewritten by LPTS and its result compared against the original; a wrong-- rewrite raises an error. The query otherwise returns its normal rows.D SET lpts_check = true;D SELECT name FROM users WHERE age > 25;┌─────────┐│ name ││ varchar │├─────────┤│ Alice ││ Carol │└─────────┘Über lpts
LPTS ist eine DuckDB-Erweiterung für die Inspektion optimierter Pläne und SQL-Transpilierung zwischen Systemen. LPTS nimmt den logischen Plan nach dem Optimierer von DuckDB und rekonstruiert äquivalentes SQL als Folge benannter CTEs.
PRAGMA-Syntax
PRAGMA lpts('<query>');EXPLAIN (FORMAT SQL)
LPTS erweitert außerdem DuckDBs EXPLAIN um ein SQL-Format. EXPLAIN (FORMAT SQL) <query> gibt den optimierten logischen Plan als äquivalentes CTE-SQL zurück — dieselbe
Ausgabe wie PRAGMA lpts, jedoch als erstklassige EXPLAIN-Anweisung.
Die CLI gibt das SQL als einfachen mehrzeiligen Text aus, während JDBC, Python und andere
Clients das übliche zweispaltige EXPLAIN-Ergebnis (explain_key, explain_value)
erhalten. Es berücksichtigt lpts_dialect genau wie PRAGMA lpts.
EXPLAIN (FORMAT SQL) SELECT name FROM users WHERE age > 25;┌─────────────────────────────┐│┌───────────────────────────┐│││ Optimized Logical Plan │││└───────────────────────────┘│└─────────────────────────────┘WITHt0_scan (t0_name) AS ( SELECT "name" FROM memory.main.users WHERE (age>25))SELECT t0_name AS "name"FROM t0_scan;LPTS plant die Abfrage über DuckDB, optimiert sie und serialisiert anschließend den optimierten logischen Plan.
Unterstützte Dialekte
Die Dialekteinstellungen akzeptieren diese Werte:
| Dialekt | Akzeptierte Werte |
|---|---|
| DuckDB | duckdb |
| PostgreSQL | postgres, postgresql |
| Spark SQL | spark |
| Hive | hive |
| Trino / Presto | trino, presto |
| Snowflake | snowflake |
| BigQuery | bigquery, bq |
| Redshift | redshift |
| MySQL / MariaDB | mysql, mariadb |
Anwendungsfälle
- Optimierte DuckDB-Pläne als SQL prüfen.
- Optimizer-Umschreibungen debuggen, etwa Filter-Pushdown, Join-Umordnung, Top-N, materialisierte CTEs und Subquery-Dekorrelation.
- Ein CTE-Programm erzeugen, das die optimierte Ausführungsform vermittelt.
- SQL für eine andere Engine mit
lpts_dialectausgeben. - Syntax anderer SQL-Dialekte mit
lpts_input_dialectnach DuckDB-SQL umwandeln und anschließend ausführen oder prüfen.
Unterstützte Operatoren
LPTS soll alle logischen Operatoren abdecken, die von optimierten DuckDB- SELECT-Plänen erzeugt werden. Die aktuelle Regressionssuite führt alle 22 TPC-H-Abfragen im Roundtrip aus und prüft Joins, Aggregate, Fenster, Mengenoperationen, CTEs, rekursive CTEs, Tabellenfunktionen, DuckLake-Scans und Inserts.
Nicht unterstützte Optimizer-Sonderfälle schlagen explizit mit NotImplementedException fehl.
Beispiele
D CREATE TABLE events (id INTEGER, ts TIMESTAMP, name VARCHAR, "order" INTEGER);D INSERT INTO events VALUES (1, TIMESTAMP '2024-01-15 08:09:10', 'alpha', 10), (11, TIMESTAMP '2024-01-16 11:12:13', 'beta', 20);
-- Render generated SQL for PostgreSQL.D SET lpts_dialect = 'postgres';
-- Return generated CTE SQL directly in the shell.D PRAGMA lpts( 'SELECT strftime(ts, ''%Y-%m-%d'') AS day FROM events WHERE id > 10 ORDER BY day');
-- Return generated CTE SQL as a table row, useful in scripts and tests.D SELECT sqlFROM lpts_query( 'SELECT strftime(ts, ''%Y-%m-%d'') AS day FROM events WHERE id > 10 ORDER BY day');WITHt0_scan (ts) AS ( SELECT ts FROM events WHERE (id>10))SELECT to_char(ts, 'YYYY-MM-DD') AS "day"FROM t0_scanORDER BY to_char(ts, 'YYYY-MM-DD') ASC NULLS LAST;-- Switch back to DuckDB rendering.D SET lpts_dialect = 'duckdb';
-- Turn on transparent round-trip checking, then run the query directly.-- LPTS rewrites every top-level SELECT and compares its result bag against-- the original, raising on a mismatch; the query returns its normal rows.D SET lpts_check = true;D SELECT name FROM events WHERE id > 10 ORDER BY name;┌─────────┐│ name ││ varchar │├─────────┤│ beta │└─────────┘-- Print the AST tree to stdout for interactive debugging.D PRAGMA print_ast('SELECT name FROM events WHERE id > 10 ORDER BY name');
-- Return the AST tree as a table row, useful for tools and regression tests.D SELECT astFROM print_ast_query('SELECT name FROM events WHERE id > 10 ORDER BY name');-- Normalize MySQL syntax before DuckDB parses and plans the query.D SET lpts_input_dialect = 'mysql';
-- Return source-dialect SQL normalized to DuckDB SQL.D SELECT sqlFROM lpts_normalize_query( 'SELECT `order`, DATE_FORMAT(ts, ''%Y-%m-%d %H:%i:%s'') AS formatted FROM events LIMIT 5, 10');SELECT "order", strftime(ts, '%Y-%m-%d %H:%M:%S') AS formatted FROM events LIMIT 10 OFFSET 5Hinzugefügte Funktionen
| function_name | function_type | description | comment | examples |
|---|---|---|---|---|
| lpts | pragma | Return the optimized logical plan for a query as equivalent CTE SQL. | NULL | [PRAGMA lpts(‘SELECT name FROM users WHERE age > 25’)] |
| lpts_normalize_query | table | Normalize SQL from lpts_input_dialect into DuckDB SQL without planning it. | NULL | [SET lpts_input_dialect = ‘mysql’; SELECT sql FROM lpts_normalize_query(‘SELECT order FROM users LIMIT 1, 2’)] |
| lpts_query | table | Table-function form of PRAGMA lpts. Returns the generated CTE SQL as a single sql column. | NULL | [SELECT sql FROM lpts_query(‘SELECT name FROM users WHERE age > 25’)] |
| print_ast | pragma | Print the LPTS AST tree for a query to stdout. | NULL | [PRAGMA print_ast(‘SELECT name FROM users WHERE age > 25’)] |
| print_ast_query | table | Table-function form of PRAGMA print_ast. Returns the rendered AST tree as a single ast column. | NULL | [SELECT ast FROM print_ast_query(‘SELECT name FROM users WHERE age > 25’)] |
Überladene Funktionen
Diese Erweiterung fügt keine Funktionsüberladungen hinzu.
Hinzugefügte Typen
Diese Erweiterung fügt keine Typen hinzu.
Hinzugefügte Einstellungen
| name | description | input_type | scope | aliases |
|---|---|---|---|---|
| lpts_check | When true, transparently verify every top-level SELECT: the query returns its normal result, but LPTS rewrites it and, on a different result, raises ‘LPTS check failed’ (an unsupported query raises a specific ‘LPTS check: unsupported’ error). If the LPTS_CHECK_LOG environment variable points to a file, it instead logs one ‘<query#> OK|WRONG|UNSUPPORTED|FAIL’ line per SELECT and never raises (suite-coverage mode). | BOOLEAN | GLOBAL | [] |
| lpts_dialect | SQL dialect for lpts output. Valid values: ‘duckdb’ (default), ‘postgres’, ‘spark’, ‘hive’, ‘trino’, ‘presto’, ‘snowflake’, ‘bigquery’, ‘redshift’, ‘mysql’, ‘mariadb’ | VARCHAR | GLOBAL | [] |
| lpts_enable_data_dependent_optimizers | Enable LPTS planning optimizers that depend on current data, statistics, cardinality estimates, row groups, or runtime dynamic filters. | BOOLEAN | GLOBAL | [] |
| lpts_input_dialect | SQL dialect for lpts input normalization. Valid values: ‘duckdb’ (default), ‘postgres’, ‘spark’, ‘hive’, ‘trino’, ‘presto’, ‘snowflake’, ‘bigquery’, ‘redshift’, ‘mysql’, ‘mariadb’ | VARCHAR | GLOBAL | [] |
| lpts_merge_pipeline | Fuse chains of single-child pipeline operators (Limit/OrderBy/Project/Aggregate/Filter, and pushdown-free base-table scans) into one flat SELECT per query block instead of one CTE per operator. | BOOLEAN | GLOBAL | [] |