How should we build pixi build?

[ruben-arts]

He all, we would like to discuss an idea that has been on our radar for a long long time!

pixi build

We want to be able to build conda packages directly from a pixi project.

There are however lots of unanswered questions and design decisions that still need to be made. Of course we want to use our rattler-build application to build the actual package but lets go over a few use cases we would like to support:

• Build C++ packages against multiple compilers on multiple platforms
• Build noarch python packages
• Build interpreter specific python packages, variants?
• Build a pixi project into a conda-forge feedstock. (translate to recipe.yaml?)
• Use the pixi build logic to be able to depend on pixi projects as source dependencies. (Like python sdists but for pixi project with conda packages)
• Have a workflow like cargo or pip to depend on local versions of a package while developing it where we use pixi build to build the package on install.

There are no concrete ideas from the pixi team yet so please share your ideas and use-cases, that would really help us develop the right integration! What can we learn from the other ecosystems and existing tools?

[majidaldo]

How would this be different from conda-build or rattlter-build? Is this related to mutienv?

From my experience, I never really cared for a difference between build, dev, and prod, as there were just variations of some base functionality. Therefore, from this point of view, it all can be handled from one language/spec.

[ruben-arts]

It should be really close to rattler-build as we're going to use that as the library to execute the building but we need a way to specify the build in the pixi toml to avoid having a recipe.yaml next to your pixi.toml as we want to have pixi be one integrated tool.

[majidaldo]

...sooo just embed the recipe.yaml as toml in pixi.toml?

[jacksonwb]

Getting the developer workflow right, and provide something easy akin to pip's editable install would really be cool, and move the whole conda experience forward from my perspective.

[pavelzw]

This will be solved by #879

[olivier-lacroix]

It's bit self-serving, but I'd love to be able to build python packages directly with pixi build @ruben-arts !

[ruben-arts]

To stay true to the current idea, we could possibly add pixi build --pypi.

[mbestipa]

I am all for only merging recipe.yaml into pixi.toml.

I mentioned this in the discord:
I've been using conda to build packages as well as for building python environments for a while now. At my current role in an ML team, most of our repos build a library (with a meta.yaml file), as well as keep track of an environment for development (environment.yml). We start with building jupyter notebooks, try things out and take things we want to include into python scripts.

People on the team will start by adding a dependency to the environment.yml file so they can try new things out, but then forget to add the dependency to library itself. I currently have some scripts to ensure that the dependency gets added to both files, but having a flag in pixi build where you can add a dependency to the environment but also specify to add it to the recipe file could be helpful

[anjos]

This looks like an important feature. I'd have two questions:

Would the same also work using pyproject.toml files?

From another perspective, what is the current delta between the schema of pixi.toml and what can go in a recipe.yaml?

[ruben-arts]

Yes if we support it in pixi.toml it should be supported in pyproject.toml

There is currently a significant delta e.g.

• Tasks might be mapped to hard-coded build commands, but it might also be confusing.
• The dependencies need to be split up into host and build dependencies.
• A lot of fields are missing, test build output

[anjos]

One could also think about features as build variants (the -m flag in conda-build or rattler-build), in certain conditions.