It was and is a life saver. Our django app suffered from runaway memory leaks (quite a story). We were not able to track down the root cause exactly. There are numerous, similar issues with uvicorn or other webservers. Granian contained these problems. Multi process management is also reliable.

What did you try to debug this?

memray and later a custom request wrapper that output python gc statistics. Our main candidates for the leak are: grpc, asgi server itself, psycopg, django channels. All leaked to some degree. Alas, it did not became clear what caused the runaway leak at 30 MB/s. Capturing flamegraphs just before OOM kills would require some more engineering. Granian contained these situations until upgradings later that year made the system more stable to begin with.

I'd love to make it work with flightsql or HTTP endpoints returning arrow IPC [0]! Did you consider using perspective for last-mile charting [1]? Building your own seems like a huge chunk of work. Well done!

[0] https://duckdb.org/docs/stable/clients/wasm/data_ingestion#a... [1] https://github.com/finos/perspective

Perspective is also getting direct support for DuckDB soon! https://github.com/finos/perspective/pull/3062

> The virality of async await is that once you mark a function async, then you can only call it from another async function

That's not exactly true. Many languages return a curried version of it, which can be executed once you have a runtime.

True. Therefore I'm eagerly awaiting an artificially intelligent product manager.

Or I might build that myself.

Looks great! Is flightsql supported over the wire too, so one could hook it up to grafana? Any plans to support iceberg?

Yes! It can connect to FlightSQL compatible servers (see https://docs.spiceai.org/data-connectors/flightsql ) and its also a FlightSQL compatible server

We also have a Grafana plugin we'll continue to improve to make it super easy to connect to Grafana, and Spice has a metrics endpoint and example Grafana dashboard for monitoring itself https://github.com/spiceai/spiceai/blob/trunk/monitoring/gra...

And yes, Iceberg is very high up on our list

We like Rust for its build tooling and safe concurrency primitives. Thus we are eyeballing with candle [1].

OTOH, writing platform specific backends is a huge undertaking. Do you think backends such as MPS may become a shared effort?

[1] https://github.com/huggingface/candle [2] https://github.com/huggingface/candle/issues/313

I think your usage example will be easier to understand, if it explains where nix:0 is setup as CRI image-service. Maybe I glanced over it.

Awesome project!

Hi Alex, I’ve updated the README with a bit more information.

0 is actually an unbindable port, so nix:0 is just an arbitrary string that is unlikely to have conflicts. The Kubelet needs to be configured with —image-service-endpoint to use the nix-snapshotter gRPC socket to handle the “PullImage” RPC. There we can take over resolving the image reference when it’s prefixed with nix:0.

Any pointer to flakehub?

I think it's mostly just a prototype: https://github.com/elohmeier/flakehub

> schema

I'm dreaming of a universal standard based on JSON Schema. Dynamicity is a challenge of course. I guess one needs to strike a balance between static data, say for each subcommand, and a full blown configuration language, say nix.

> write text datalog queries

you may be interested in https://github.com/sqwishy/owoof