web_search

Query Google Custom Search API directly from SQL with filter pushdowns

Maintainer(s): onnimonni

Installing and Loading

INSTALL web_search FROM community;
LOAD web_search;

Example

-- Setup credentials (get from Google Cloud Console)
CREATE SECRET google_search (
    TYPE google_search,
    key 'YOUR_API_KEY',
    cx 'YOUR_SEARCH_ENGINE_ID'
);

-- Basic web search
SELECT title, link, snippet
FROM google_search('duckdb database')
LIMIT 10;

-- Filter by site
SELECT title, link
FROM google_search('rust tutorial')
WHERE site = 'github.com'
LIMIT 20;

About web_search

DuckDB extension for querying Google Custom Search API with SQL filter pushdowns.

Features

Table Functions

• google_search(query) - Web search
• google_image_search(query) - Image search

Filter Pushdowns (WHERE clause)

• site = 'example.com' / site IN (...) / site != '...'
• language = 'en' / country = 'US' / file_type = 'pdf'
• exact_match = 'phrase' / term IN (...) / term != '...'
• ORDER BY date DESC/ASC
• LIMIT optimization

Named Parameters

• exact_terms, exclude_terms, or_terms
• safe (boolean), rights, sort, structured_data
• Image: img_size, img_type, img_color_type, img_dominant_color

COPY TO

• FORMAT google_pse_annotation - Export URL patterns to PSE annotation XML

Other

• pagemap column as JSON type (arrow operator access)
• Graceful 429 rate limit handling
• JSON extension autoloaded

Setup

Get API credentials:

• API Key: https://developers.google.com/custom-search/v1/introduction
• Search Engine ID (cx): https://programmablesearchengine.google.com/controlpanel/all

Added Functions

Overloaded Functions

This extension does not add any function overloads.

Added Types

This extension does not add any types.

Added Settings

This extension does not add any settings.