perf: create directories up front

[baszalmstra]

When linking packages a lot of time is spent creating directories as this flamegraph highlights:

Most of the time is spent checking if a directory (or its parents) already exists. However, we already know upfront all the directories that need to be created so why not do that? Well, say no more! That is exactly what this PR does!

New situation:

Its completely gone. Great success! 👍

In my preliminary dead simple benchmarks, this can easily save 20% of installation times.

[baszalmstra]

This PR is still open because for @ruben-arts this was actually slowing down his installation. We still need to investigate more.

[baszalmstra]

@ruben-arts When you have some time shall we take another look at debugging this? I tested it this weekend and on windows this is a pretty big win.

[baszalmstra]

Here are some benchmarks from my windows machine:

❯ hyperfine --shell nu --prepare "rm -rf .prefix" "target/release/rattler_fast.exe create python" "target/release/rattler.exe create python"
Benchmark 1: target/release/rattler_fast.exe create python
  Time (mean ± σ):      1.303 s ±  0.314 s    [User: 0.472 s, System: 0.328 s]
  Range (min … max):    0.961 s …  1.732 s    10 runs

Benchmark 2: target/release/rattler.exe create python
  Time (mean ± σ):      1.535 s ±  0.420 s    [User: 0.414 s, System: 0.374 s]
  Range (min … max):    1.076 s …  2.412 s    10 runs

Summary
  target/release/rattler_fast.exe create python ran
    1.18 ± 0.43 times faster than target/release/rattler.exe create python

❯ hyperfine --shell nu -w 3 --prepare "rm -rf .prefix" "target/release/rattler_fast.exe create jupyterlab" "target/release/rattler.exe create jupyterlab"
Benchmark 1: target/release/rattler_fast.exe create jupyterlab
  Time (mean ± σ):      5.854 s ±  0.538 s    [User: 2.174 s, System: 0.871 s]
  Range (min … max):    5.177 s …  6.880 s    10 runs

Benchmark 2: target/release/rattler.exe create jupyterlab
  Time (mean ± σ):      6.287 s ±  0.623 s    [User: 2.271 s, System: 1.435 s]
  Range (min … max):    5.346 s …  7.015 s    10 runs

Summary
  target/release/rattler_fast.exe create jupyterlab ran
    1.07 ± 0.15 times faster than target/release/rattler.exe create jupyterlab 

[baszalmstra]

Under WSL:

➜ hyperfine -w 3 --prepare "rm -rf .prefix" "target/release/rattler_fast create python" "target/release/rattler create python"
Benchmark 1: target/release/rattler_fast create python
  Time (mean ± σ):     690.9 ms ±  18.5 ms    [User: 786.7 ms, System: 1411.9 ms]
  Range (min … max):   672.8 ms … 738.0 ms    10 runs

Benchmark 2: target/release/rattler create python
  Time (mean ± σ):     704.9 ms ±  43.2 ms    [User: 870.0 ms, System: 1782.2 ms]
  Range (min … max):   654.2 ms … 798.4 ms    10 runs

Summary
  'target/release/rattler_fast create python' ran
    1.02 ± 0.07 times faster than 'target/release/rattler create python'

➜ hyperfine -w 3 --prepare "rm -rf .prefix" "target/release/rattler_fast create jupyterlab" "target/release/rattler create jupyterlab"
Benchmark 1: target/release/rattler_fast create jupyterlab
  Time (mean ± σ):      2.705 s ±  0.026 s    [User: 2.800 s, System: 4.408 s]
  Range (min … max):    2.663 s …  2.741 s    10 runs

Benchmark 2: target/release/rattler create jupyterlab
  Time (mean ± σ):      2.708 s ±  0.032 s    [User: 2.842 s, System: 6.267 s]
  Range (min … max):    2.679 s …  2.777 s    10 runs

Summary
  'target/release/rattler_fast create jupyterlab' ran
    1.00 ± 0.02 times faster than 'target/release/rattler create jupyterlab'

Looks like at least under WSL there is hardly any difference.

[ruben-arts]

On my windows machine:

❯ hyperfine --shell nu --prepare "rm -rf .prefix" "target/release/rattler_fast.exe create python" "target/release/rattler.exe create python"
Benchmark 1: target/release/rattler_fast.exe create python
  Time (mean ± σ):      1.637 s ±  0.024 s    [User: 0.501 s, System: 0.679 s]
  Range (min … max):    1.597 s …  1.675 s    10 runs

Benchmark 2: target/release/rattler.exe create python
  Time (mean ± σ):      1.740 s ±  0.034 s    [User: 0.521 s, System: 0.768 s]
  Range (min … max):    1.694 s …  1.787 s    10 runs

Summary
  target/release/rattler_fast.exe create python ran
    1.06 ± 0.03 times faster than target/release/rattler.exe create python

[ruben-arts]

On my windows machine with wsl

~/development/rattler> hyperfine --shell nu --prepare "rm -rf .prefix" "target/release/rattler create python" "target/release/rattler_slow create python"
Benchmark 1: target/release/rattler create python
  Time (mean ± σ):     846.2 ms ±  24.8 ms    [User: 926.7 ms, System: 967.0 ms]
  Range (min … max):   821.7 ms … 894.6 ms    10 runs

Benchmark 2: target/release/rattler_slow create python
  Time (mean ± σ):     859.0 ms ±  50.4 ms    [User: 968.9 ms, System: 1332.6 ms]
  Range (min … max):   812.5 ms … 952.1 ms    10 runs

Summary
  target/release/rattler create python ran
    1.02 ± 0.07 times faster than target/release/rattler_slow create python

[ruben-arts]

And for completeness on my linux beast:

➜ hyperfine --prepare "rm -rf .prefix" "target/release/rattler create python" "target/release/rattler_slow create python"
Benchmark 1: target/release/rattler create python
  Time (mean ± σ):     917.9 ms ±  44.7 ms    [User: 1107.3 ms, System: 828.9 ms]
  Range (min … max):   878.7 ms … 1008.0 ms    10 runs
 
Benchmark 2: target/release/rattler_slow create python
  Time (mean ± σ):     946.0 ms ±  35.1 ms    [User: 1191.0 ms, System: 914.6 ms]
  Range (min … max):   905.1 ms … 1008.5 ms    10 runs
 
Summary
  target/release/rattler create python ran
    1.03 ± 0.06 times faster than target/release/rattler_slow create python