lpts

Inspektion optimierter Pläne und SQL-Transpilierung zwischen Systemen

Maintainer: ila

Installation und Laden

INSTALL lpts FROM community;
LOAD lpts;

Beispiel

-- Set both input and output dialects to DuckDB for a first round trip.
D SET lpts_input_dialect = 'duckdb';
D SET lpts_dialect = 'duckdb';
D CREATE TABLE users (id INTEGER, name VARCHAR, age INTEGER);
D INSERT INTO users VALUES (1, 'Alice', 30), (2, 'Bob', 22), (3, 'Carol', 28);
-- Return the optimized plan as readable CTE SQL.
D PRAGMA lpts('SELECT name FROM users WHERE age > 25');
WITH
t0_scan (t0_name) AS (
SELECT "name"
FROM memory.main.users
WHERE (age>25)
)
SELECT t0_name AS "name"
FROM t0_scan;
-- The same output, exposed as a first-class EXPLAIN statement.
D EXPLAIN (FORMAT SQL) SELECT name FROM users WHERE age > 25;
┌─────────────────────────────┐
│┌───────────────────────────┐│
││ Optimized Logical Plan ││
│└───────────────────────────┘│
└─────────────────────────────┘
WITH
t0_scan (t0_name) AS (
SELECT "name"
FROM memory.main.users
WHERE (age>25)
)
SELECT t0_name AS "name"
FROM t0_scan;
-- Turn on transparent round-trip checking. Every top-level SELECT is now
-- rewritten by LPTS and its result compared against the original; a wrong
-- rewrite raises an error. The query otherwise returns its normal rows.
D SET lpts_check = true;
D SELECT name FROM users WHERE age > 25;
┌─────────┐
name
varchar
├─────────┤
│ Alice │
│ Carol │
└─────────┘

Über lpts

LPTS ist eine DuckDB-Erweiterung für die Inspektion optimierter Pläne und SQL-Transpilierung zwischen Systemen. LPTS nimmt den logischen Plan nach dem Optimierer von DuckDB und rekonstruiert äquivalentes SQL als Folge benannter CTEs.

PRAGMA-Syntax

PRAGMA lpts('<query>');

EXPLAIN (FORMAT SQL)

LPTS erweitert außerdem DuckDBs EXPLAIN um ein SQL-Format. EXPLAIN (FORMAT SQL) <query> gibt den optimierten logischen Plan als äquivalentes CTE-SQL zurück — dieselbe Ausgabe wie PRAGMA lpts, jedoch als erstklassige EXPLAIN-Anweisung. Die CLI gibt das SQL als einfachen mehrzeiligen Text aus, während JDBC, Python und andere Clients das übliche zweispaltige EXPLAIN-Ergebnis (explain_key, explain_value) erhalten. Es berücksichtigt lpts_dialect genau wie PRAGMA lpts.

EXPLAIN (FORMAT SQL) SELECT name FROM users WHERE age > 25;
┌─────────────────────────────┐
│┌───────────────────────────┐│
││ Optimized Logical Plan ││
│└───────────────────────────┘│
└─────────────────────────────┘
WITH
t0_scan (t0_name) AS (
SELECT "name"
FROM memory.main.users
WHERE (age>25)
)
SELECT t0_name AS "name"
FROM t0_scan;

LPTS plant die Abfrage über DuckDB, optimiert sie und serialisiert anschließend den optimierten logischen Plan.

Unterstützte Dialekte

Die Dialekteinstellungen akzeptieren diese Werte:

Dialekt Akzeptierte Werte
DuckDB duckdb
PostgreSQL postgres, postgresql
Spark SQL spark
Hive hive
Trino / Presto trino, presto
Snowflake snowflake
BigQuery bigquery, bq
Redshift redshift
MySQL / MariaDB mysql, mariadb

Anwendungsfälle

  • Optimierte DuckDB-Pläne als SQL prüfen.
  • Optimizer-Umschreibungen debuggen, etwa Filter-Pushdown, Join-Umordnung, Top-N, materialisierte CTEs und Subquery-Dekorrelation.
  • Ein CTE-Programm erzeugen, das die optimierte Ausführungsform vermittelt.
  • SQL für eine andere Engine mit lpts_dialect ausgeben.
  • Syntax anderer SQL-Dialekte mit lpts_input_dialect nach DuckDB-SQL umwandeln und anschließend ausführen oder prüfen.

Unterstützte Operatoren

LPTS soll alle logischen Operatoren abdecken, die von optimierten DuckDB- SELECT-Plänen erzeugt werden. Die aktuelle Regressionssuite führt alle 22 TPC-H-Abfragen im Roundtrip aus und prüft Joins, Aggregate, Fenster, Mengenoperationen, CTEs, rekursive CTEs, Tabellenfunktionen, DuckLake-Scans und Inserts.

Nicht unterstützte Optimizer-Sonderfälle schlagen explizit mit NotImplementedException fehl.

Beispiele

D CREATE TABLE events (id INTEGER, ts TIMESTAMP, name VARCHAR, "order" INTEGER);
D INSERT INTO events VALUES
(1, TIMESTAMP '2024-01-15 08:09:10', 'alpha', 10),
(11, TIMESTAMP '2024-01-16 11:12:13', 'beta', 20);
-- Render generated SQL for PostgreSQL.
D SET lpts_dialect = 'postgres';
-- Return generated CTE SQL directly in the shell.
D PRAGMA lpts(
'SELECT strftime(ts, ''%Y-%m-%d'') AS day
FROM events
WHERE id > 10
ORDER BY day'
);
-- Return generated CTE SQL as a table row, useful in scripts and tests.
D SELECT sql
FROM lpts_query(
'SELECT strftime(ts, ''%Y-%m-%d'') AS day
FROM events
WHERE id > 10
ORDER BY day'
);
WITH
t0_scan (ts) AS (
SELECT ts
FROM events
WHERE (id>10)
)
SELECT to_char(ts, 'YYYY-MM-DD') AS "day"
FROM t0_scan
ORDER BY to_char(ts, 'YYYY-MM-DD') ASC NULLS LAST;
-- Switch back to DuckDB rendering.
D SET lpts_dialect = 'duckdb';
-- Turn on transparent round-trip checking, then run the query directly.
-- LPTS rewrites every top-level SELECT and compares its result bag against
-- the original, raising on a mismatch; the query returns its normal rows.
D SET lpts_check = true;
D SELECT name FROM events WHERE id > 10 ORDER BY name;
┌─────────┐
│ name │
│ varchar │
├─────────┤
│ beta │
└─────────┘
-- Print the AST tree to stdout for interactive debugging.
D PRAGMA print_ast('SELECT name FROM events WHERE id > 10 ORDER BY name');
-- Return the AST tree as a table row, useful for tools and regression tests.
D SELECT ast
FROM print_ast_query('SELECT name FROM events WHERE id > 10 ORDER BY name');
-- Normalize MySQL syntax before DuckDB parses and plans the query.
D SET lpts_input_dialect = 'mysql';
-- Return source-dialect SQL normalized to DuckDB SQL.
D SELECT sql
FROM lpts_normalize_query(
'SELECT `order`, DATE_FORMAT(ts, ''%Y-%m-%d %H:%i:%s'') AS formatted FROM events LIMIT 5, 10'
);
SELECT "order", strftime(ts, '%Y-%m-%d %H:%M:%S') AS formatted FROM events LIMIT 10 OFFSET 5

Hinzugefügte Funktionen

function_name function_type description comment examples
lpts pragma Return the optimized logical plan for a query as equivalent CTE SQL. NULL [PRAGMA lpts(‘SELECT name FROM users WHERE age > 25’)]
lpts_normalize_query table Normalize SQL from lpts_input_dialect into DuckDB SQL without planning it. NULL [SET lpts_input_dialect = ‘mysql’; SELECT sql FROM lpts_normalize_query(‘SELECT order FROM users LIMIT 1, 2’)]
lpts_query table Table-function form of PRAGMA lpts. Returns the generated CTE SQL as a single sql column. NULL [SELECT sql FROM lpts_query(‘SELECT name FROM users WHERE age > 25’)]
print_ast pragma Print the LPTS AST tree for a query to stdout. NULL [PRAGMA print_ast(‘SELECT name FROM users WHERE age > 25’)]
print_ast_query table Table-function form of PRAGMA print_ast. Returns the rendered AST tree as a single ast column. NULL [SELECT ast FROM print_ast_query(‘SELECT name FROM users WHERE age > 25’)]

Überladene Funktionen

Diese Erweiterung fügt keine Funktionsüberladungen hinzu.

Hinzugefügte Typen

Diese Erweiterung fügt keine Typen hinzu.

Hinzugefügte Einstellungen

name description input_type scope aliases
lpts_check When true, transparently verify every top-level SELECT: the query returns its normal result, but LPTS rewrites it and, on a different result, raises ‘LPTS check failed’ (an unsupported query raises a specific ‘LPTS check: unsupported’ error). If the LPTS_CHECK_LOG environment variable points to a file, it instead logs one ‘<query#> OK|WRONG|UNSUPPORTED|FAIL’ line per SELECT and never raises (suite-coverage mode). BOOLEAN GLOBAL []
lpts_dialect SQL dialect for lpts output. Valid values: ‘duckdb’ (default), ‘postgres’, ‘spark’, ‘hive’, ‘trino’, ‘presto’, ‘snowflake’, ‘bigquery’, ‘redshift’, ‘mysql’, ‘mariadb’ VARCHAR GLOBAL []
lpts_enable_data_dependent_optimizers Enable LPTS planning optimizers that depend on current data, statistics, cardinality estimates, row groups, or runtime dynamic filters. BOOLEAN GLOBAL []
lpts_input_dialect SQL dialect for lpts input normalization. Valid values: ‘duckdb’ (default), ‘postgres’, ‘spark’, ‘hive’, ‘trino’, ‘presto’, ‘snowflake’, ‘bigquery’, ‘redshift’, ‘mysql’, ‘mariadb’ VARCHAR GLOBAL []
lpts_merge_pipeline Fuse chains of single-child pipeline operators (Limit/OrderBy/Project/Aggregate/Filter, and pushdown-free base-table scans) into one flat SELECT per query block instead of one CTE per operator. BOOLEAN GLOBAL []