I love duck db. We use it a ton for indexing and organizing system / kernel level metrics exported by eBPF.
Check out our sandbox:
What’s the situation where this is useful? Seems like ‘replace your remote duckDB instance—used to replace a DB server—with duckDB instance + a flight server (or a bunch of them!)’. Who has a problem for which this is the solution?
This is a cool thought exercise to think that everything that we do in the data world can be done in SQL, from SQL. In a sense this is the MCPs but for the DuckDB world.
Not clear. Will this allow loading ipc files in DuckDB finally? That's been my biggest issue, since I use IPC files for append operations before I turn them into parquet files.
Does this mean the data source and destination both have to set up flight servers? I imagine then this won’t be useful for integration of third-party services.
last monday: https://news.ycombinator.com/item?id=44036343
This is very nice. I also love the fuzzycomplete and lindel from the same org/authors.
I was almost going to build a lakehouse* with DuckDB because I low-key love it, easiest and strongest analytical engine I've found yet: scale from laptops to big metal, while being mostly out-of-core when doing sane stuff, and avoiding distributed computing for SQL in the process (looking at you Spark).
That is until I found out it does not support Iceberg writes[1], big nono as I would need another engine for inserts, and I want a simple stack :(. What a bummer.
[1] https://github.com/duckdb/duckdb_iceberg/issues/37
*that is what they are called now aren't they? I just can't follow the terms anymore haha.