read_lines

Read line-based text files with line numbers and efficient subset extraction. Supports glob patterns, line selection, and context lines around matches.

Maintainer(s): teaguesterling

Installing and Loading

INSTALL read_lines FROM community;
LOAD read_lines;

Example

-- Read all lines from a file
SELECT * FROM read_lines('server.log');

-- Read specific lines (positional argument)
SELECT * FROM read_lines('server.log', '100-200');

-- Read lines with context around matches
SELECT * FROM read_lines('error.log', 42, context := 3);

-- Path-embedded line selection
SELECT * FROM read_lines('error.log:42 +/-3');

-- Last 10 lines (from-end syntax)
SELECT * FROM read_lines('app.log', '+10-');

-- Glob pattern to read from multiple files
SELECT * FROM read_lines('logs/*.log')
WHERE content LIKE '%ERROR%';

-- Parse lines from a string
SELECT * FROM parse_lines('line1
line2
line3');

About read_lines

The read_lines extension provides functions for reading line-based text files with precise control over which lines to extract.

Functions:

• read_lines(path) - Read all lines from file(s) with glob support
• read_lines(path, lines[, trim]) - Read selected lines (positional arguments)
• read_lines_lateral(path[, lines[, trim]]) - Lateral join variant for file paths from table columns
• parse_lines(text, ...) - Parse lines from a string

Sources may be non-seekable (pipes/streams, e.g. shellfs commands), including per-row commands in a correlated lateral join.

Output Schema:

Trimming (trim argument; transforms content only, never changes which rows appear):

• NULL / false / 'none' - Preserve exactly (default)
• true / 'endings' - Strip the line terminator only
• 'right' - Strip terminator and trailing spaces/tabs
• 'left' - Strip leading spaces/tabs, keep terminator
• 'both' - Both sides

Line Selection:

• Single line: 42 or lines := 42
• Range: '100-200' (inclusive)
• List: [1, 5, 10]
• With context: '42 +/-3'
• From end: '+10-' (last 10 lines), '+5' (5th from end)
• Path-embedded: 'file.py:42 +/-3'
• Struct: {start: 100, stop: 200} or {line: 42, context: 3}

Context Lines:

• before := N - Include N lines before each match
• after := N - Include N lines after each match
• context := N - Shorthand for both before and after

Use Cases:

• Extract specific lines from log files
• Get error lines with surrounding context
• Process multiple files with glob patterns
• Parse multi-line string columns

Added Functions

Overloaded Functions

This extension does not add any function overloads.

Added Types

This extension does not add any types.

Added Settings

This extension does not add any settings.