gccrs .rox and rustc .rmeta

[lu-zero]

It seems that they have the same purpose, would be worthy to try to converge the two in a single one?

[CohenArthur]

Do you mean that they should share the same format? If so, this is impossible for a few reasons (at least for now)

1. They are both a representation of each compiler's internal structures. Because the types are not exactly the same, that representation differs
2. Because of this, that also means that our and rustc's representation are unstable (it's unstable for gccrs for sure - I think that representation is also unstable for rustc but would need some confirmation). As soon as we add a new field to one of our AST classes, this would change the representation.

We haven't made plans for any stable representation or anything like that, and I think we'd like to keep it unstable for now as it helps speed up development. I'm also not sure if it's worth exploring this venue, as there are plenty of other places where code compiled under gccrs will not be compatible with code compiled with rustc.

I might have misunderstood your question so feel free to correct me @lu-zero !

[lu-zero]

You answered perfectly :) Only once the language reaches ABI stability I guess it might get interesting.

[CohenArthur]

I think ABI stability won't be enough sadly! This is about internal compiler exports, basically creating the equivalent of a header file so that crates can access the same public functions/macros/traits.

For example, in my crate foo I'll define the following function and type:

// foo/src/lib.rs

pub struct Foo<T, U, V>(pub T, pub U, pub V);

pub fn foo_id<T>(arg0: T) -> T { arg0 }

In my crate/program bar, I'll use the foo crate. So I want to be able to do the following:

// bar/src/main.rs

fn main() {
    let foo = Foo(15i32, "hey", vec![1, 2, 3, 4]);
    let foo_f = generic_id(foo.0);
}

Compilation is gonna happen in two parts: First, compiling the foo crate and then compiling the bar one. Thus, rustc (or gccrs) needs to export information about the public functions/macros/types/traits/impls of the foo crate. rustc does it by exporting some binary representation of the AST node equivalent to a function declaration and a struct declaration in the .rmeta/.rlib file, but they could very well choose to instead export a JSON or YAML representation using Serde. In fact, we currently export "rust code" to our .rox files, but we might switch over to JSON as well. But nothing prevents us or rustc from exporting the public foo items in the following format:

function "generic_id":
    type_parameters: [T]
    parameters: ["arg0", T]
    return_type: T
    body: [
        variable ("arg0")
    ]

And because this is long and takes a lot of space, we could compress it something like that:

f"generic_id":[T]["arg0", T]T:[v"arg0"]

or even a completely different format. And since this happens during compilation and not execution, and that this representation is internal to the compiler and mainly used when bringing code from other crates (not even at the link phase - at the compilation phase), there's no purpose or incentive to stabilize it. In fact, the above format is bad because it misses a lot of information such as attributes, constness, unsafeness, asyncness, or other keywords/concepts that might be introduced later in the language, and it still takes a long time to generate and print out to the associated metadata files. So we probably want to optimize it further!

So even if Rust gets a stable ABI, I think this is unlikely to be stabilized, and it is unlikely that we'd like to stabilize it as well :) I'm definitely not opposed to the idea! But to me the cons outweigh the pros.

[bjorn3]

but we might switch over to JSON as well.

Please don't! You will be getting garbage performance if you aren't able to do partial decodes of just those parts that are necessary. Rustc even wrote a hash map implementation (https://crates.io/crates/odht) that is able to access elements without decoding the entire hash map from the crate metadata to improve performance with respect to metadata.

[CohenArthur]

Very interesting, thanks a lot!! Didn't know about that crate :DD