> Indeed DataFrames.jl isn't and won't be the fastest way to do many things
Agreed, and the DF.jl developers are aware and very open about this fact - the core design trades off flexibility and user friendliness over speed (while of course trying to be as performant as possible within those constraints).
One thing that hasn't been mentioned so far is InMemoryDatasets.jl, which as far as I know is the closest to polars in Julia-land in that it chooses a different point on the flexibility-performance curve more towards the performance end. It's not very widely used as far as I can tell but could be interesting for users who need more performance than DF.jl can deliver - some benchmarks from early versions suggested performance is on par with polars: https://discourse.julialang.org/t/ann-a-new-lightning-fast-p...
Agreed, and the DF.jl developers are aware and very open about this fact - the core design trades off flexibility and user friendliness over speed (while of course trying to be as performant as possible within those constraints).
One thing that hasn't been mentioned so far is InMemoryDatasets.jl, which as far as I know is the closest to polars in Julia-land in that it chooses a different point on the flexibility-performance curve more towards the performance end. It's not very widely used as far as I can tell but could be interesting for users who need more performance than DF.jl can deliver - some benchmarks from early versions suggested performance is on par with polars: https://discourse.julialang.org/t/ann-a-new-lightning-fast-p...