Out of Memory Errors

DuckDB has a state of the art out-of-core query engine that can spill to disk for larger-than-memory processing. We continuously strive to improve DuckDB’s scalability and prevent out of memory errors whenever possible. That said, you may still experience out-of-memory errors if you run queries with multiple blocking operators, certain aggregation functions, PIVOT operations, etc., or if you have very little available memory compared to the dataset size.

Types of “Out of Memory” Errors

Out of memory errors mainly occur in two forms:

OutOfMemoryException

Most of the time DuckDB runs out of memory with an OutOfMemoryException. For example:

duckdb.duckdb.OutOfMemoryException: Out of Memory Error: failed to pin block of size 256.0 KiB (476.7 MiB/476.8 MiB used)

OOM Reaper (Linux)

Many Linux distributions have an OOM killer or OOM reaper process whose goal is to prevent memory overcommitment. If the OOM reaper killed your process, you often see the following message where DuckDB was running:

Killed

To get more detailed information, check the diagnostic messages using the dmesg command (you may need sudo):

sudo dmesg

If the process was killed by the OOM killer/reaper, you will find an entry like this:

[Fri Apr 18 02:04:10 2025] Out of memory: Killed process 54400 (duckdb) total-vm:1037911068kB, anon-rss:770031964kB, file-rss:0kB, shmem-rss:0kB, UID:1000 pgtables:1814612kB oom_score_adj:0

Troubleshooting Out-of-Memory Errors

To prevent out-of-memory errors, please consult the “Out-of-Memory” issues page of the Performance Guide.

See Also

For more information on DuckDB’s memory management, see the “Memory Management in DuckDB” blog post.