Vox has the prospects to become the best compiler backend

[rempas]

Of course before we even start, I just want to tell you that I'm closely wanting the progress Vox is making and I'm very happy that you keep doing the extremely hard task. As always, I hope you are enjoying it as much as possible. Now for the actual topic:

From the way I see it, even for people that will not want to use Vox as a straight up compiler and language, it can be the best compiler backend to be used for code generation (of course excluding compiler specific ones)! Let me explain me view behind this idea.

You see, a lot of (non-toy) compilers these days will either use an IR like LLVM, GCC's IR or either compile to another language (mostly C) as their backend. You will rarely see a compiler that outputs either clean assembly or even better, directly create the final executable file (like Vox or TCC). Of course these methods mean that your job to create the backend will be much much easier that if you had to go with the alternative route (which is why I think you are a genius for what you do!) and on top of that, these backends will also do most (if not all) of the optimizations for you so not only you'll get a backend that has an easy "API" but your code will also run very fast. That's awesome right?

Well, this comes with a great cost. Compilation times, the ancient problem! The compilation time of your compiler will always be x + n. Where 'n' is the time it takes your compiler to parse and output and 'x' is the time it takes for the "final" compiler to give use the final output. Of course, that's a problem! Most compiler are slow (or bearable at best). So having to parse another language on top of that will make things terrible! Every language that outputs to C is the proof of that. Languages like V, Zig, Nim just to name a few.

Well, at this point, you will want to stop me and tell me that TCC exists, right? A fully non-toy C compiler that implements C11 and a couple of GCC extensions. So we can use that right? Yeah, not exactly.... You see, TCC is truly fast. Like.... SUPER fast! So fast that is even 3-4 (or even more) times faster that even if you wrote clean assembly and used an assembler and linker directly. However, there is a problem and that is that TCC doesn't make any optimizations. This includes low level optimizations like the right usage of registers or loop unrolling (and probably a lot more but I'm a n00b in this topic so I don't know) that the high level language cannot make. Or If we could make, It means that we would have to manually write inline assembly and that will be a terrible experience and defeats the whole purpose of using a high level language to make our lives easier and in that case, it would be better to just output assembly directly.

But wait, Vox exists! Vox is compiler that is close to TCC when it comes to how fast it complies code! I tried to compare GCC, LDC and TCC with Vox to create a program that just prints the string: "hello, word!" and exist (using inline assembly for the other 3 compilers) and the results were:

GCC: 0.033s
LDC: 0.066s
TCC: 0.002s
VOX: 0.003s

Of course, I know that this is the smallest possible program and the results would be different in a bigger real life program but still, this is enough to prove my point!

And the difference with TCC is that you want to actually make optimizations (from what you have told me) and I suppose that you will make the important ones that I mentioned so there should be no problem! Vox can become the best compile backend out there! So good that it will be worth for the current languages to change their current backend (especially if it is the ugly LLVM) to Vox despite the fact that this will be a time consuming and hard task to do!

Now, I'm not just saying this because I'm bored and I want to start a conversation. I'm actually saying it because I really think that you should actually start promoting that! You have explained to me that Vox was created mainly for Game dev and if you don't plan to make Vox into a language with modern features that languages that can compete against language like Rust, Python, Java, and D have then I think that you should also start advertising the power than the Vox compiler has itself and how it can be used for compiler developers so people can create whatever they want. I know because I'm actually trying to make one and not having to learn assembly and having the power of a language as a backend can seven me tons of time on actually focusing to the language itself rather than on how to do some complicated stuff like learning how to properly parse expressions and re-order them (just to name what I'm currently trying to make).

I also think having a shared backend makes more sense for compiler developers as this means that people that want to improve it will contribute to the project so everyone wins for every improvement that the backend will get (the same idea with LLVM and why so many people choose it). So it will also help the compiler attract more contributors and get better!

What do you think about all that? I'm I just dreaming too much or are my points valid?

[MrSmith33]

You are correct on all points and this is already in my plans. The main difficulty with that is that I need to separate backend so it is standalone and has C API/ABI available. This is non-trivial and requires some serious motivation. Multiple people expressed the desire to use such backend if it was available, but I need someone who is more invested in trying out Vox backend to give me proper motivation.

[rempas]

Nice! I'm in! Do you need anything specific for now?

[MrSmith33]

Are you writing your frontend in C?

[rempas]

I'm writing it in D using "-betterC".

[MrSmith33]

The way I see Vox being used from languages is by linking either with static lib or with dynamic lib. In order to actually use it, that language should define bindings. I think I can try doing this with -betterC example D program, that wouldn't import vox files, but will use extern(C) API + lib.

[rempas]

Good morning! Are you talking about the shared object thing I mentioned? In that case, yeah! Just like D does it with "extern(C)" but without the "ImportC" which means that we will have to manually create bindings. That's not something tragic however, it's how most languages do it anyway. In in the future, we can create a program that will do this for us by parsing C header files. But for now, just making the shared object linking work on ELF files will be enough imo. I also tried to convert the code for my frontend from D to Vox yesterday but I remembered that I cannot link with libc as I was doing that. So here is a practical example ;)

Also, please! Give the ability to declare/call a symbol under a custom name. This will allow us to use libraries for other languages that have their own mangle without requiring Vox to know the mangle of these languages. Just have that in mind, it's not an urgently.

Static libraries (.a) will be awesome but if you ask me, first focus on making shared objects work and then you can make static libraries work. I'm saying that because shared objects are more common from what I know (and this is also true for dynamic libraries on Windows too). And some libraries like "libc" (at least on GNU/Linux distros) only offer a shared version.

P.S. Thank you for updating the spec!!! ❤️