Evolution of DataFrames.jl load time
Introduction
Julia 1.10 has just been released. You can find a summary of the improvements in this post. Seeing the release I was curious how the DataFrames.jl load time changed in it.
In this post I want to show you how long it takes to install and to load DataFrames.jl.
Experiment setup
The tests were performed on a laptop with i7-1250U processor, 16 GB of RAM and under Windows 11 Pro.
I tested the following Julia versions: 1.6.7, 1.7.3, 1.8.5, 1.9.4, 1.10.0 (the current release), and additionally the latest development version 1.11.0-DEV.1165.
For all setups I have cleaned all Julia temporary files and performed two operations:
- installation fo DataFrames.jl using
using Pkg; Pkg.add("DataFrames")operation; here I collected the reported total precompilation time; - loading DataFrames.jl using
@time using DataFramesas the only operation in a fresh Julia session; here I recorded load time and memory used.
The version of DataFrames.jl used in tests is 1.6.1.
Experiment results
I have collected the results of my test in the following table:
| Julia version | Pkg.add("DataFrames") |
@time using DataFrames |
|---|---|---|
| 1.6.7 | 55 s. | 1.19 s., 2.64 M allocations: 190 MiB |
| 1.7.3 | 44 s. | 1.20 s., 2.63 M allocations: 187 MiB |
| 1.8.5 | 43 s. | 2.04 s., 4.76 M allocations: 338 MiB |
| 1.9.4 | 79 s. | 1.23 s., 1.55 M allocations: 92 MiB |
| 1.10.0 | 60 s. | 0.79 s., 579 k allocations: 44 MiB |
| 1.11.0-DEV.1165 | 72 s. | 0.54 s., 542 k allocations: 35 MiB |
Conclusions
The major things that we can conclude are the following:
- If we excluded Julia 1.8 the load time of DataFrames.jl constantly goes down with newer versions; the same applies to memory usage.
- The precompilation time increases; in general you have to wait for around one minute to get all dependencies of DataFrames.jl compiled. Fortunately, this is a one-time cost.
In summary: Julia 1.10 brings a significant decrease of DataFrames.jl load time. Also we can see that the upcoming Julia 1.11 can be expected to be even faster. This is great news.
Happy New Year!