More control over docker-compose content

[icemagno]

I'm facing a situation where I'm already have a IPFS private network and a private permissioned blockchain.

My goal is to not allow FF to create its own blockchain node and use the one I have instead in native way (I mean in replacement of its own, not using connectors - I don't want an extra node in same machine )

Another thing is to use a customized IPFS node to connect in to my own network.

Some things I must do like empty the Bootsrap list of the IPFS config, add my own bootstrap, add my swarm file instead pass it from the compose (I may change it without to re run all the compose again - who knows?)

In the same way I need to make some adjusts in the Blockchain node to fit my needs.

In short, I must run some scripts before these components come up.

What I suggest is to allow to remove these pieces from the compose and run they "by hand". I don't know if I can do it using the override compose. By the way... why can't we just edit the compose instead using an override file? Is that possible to remove things from compose using the override file?

OR ....

Maybe you guys want to publish in some documents the step-by-step on how to build all the stack "by hand" ... from the docker pull to contract publishing ...all these things hidden by the FF-CLI so we may do more "granularity" ...

[nguyer]

Hi @icemagno, thanks for the suggestions. If you're at the point of needing fine grained control of your configuration and needing to integrate FireFly with existing external systems, it sounds like you're ready to move beyond a local development environment built with the FireFly CLI.

The goal of the FireFly CLI is to give developers a complete local stack so they can build apps that use FireFly's API. It's not intended to be an end-all-be-all orchestration tool for FireFly. It's also not our intent to "hide" anything behind the FireFly CLI. The source is open so you can read exactly what it does to set everything up. You can also turn on --verbose logging and see exactly what commands its running to set up and configure the containers.

If you're looking for a way to set up a more long lived environment I would suggest using Kubernetes for orchestration. We also provide helm charts as an example of how you might deploy FireFly in a Kubernetes environment: https://github.com/hyperledger/firefly-helm-charts

Note: these charts are a bit difficult to use (in my opinion) at the moment and I'm working on adding some significant new functionality to them. Should have some updated charts here soon.

[icemagno]

Uh.. many thanks. Sorry if my "hidding" sounds like something bad or proposital. I mean in sense of facility.

I will try the verbose to check if I can reproduce the process.

The CLI is a great tool. Unfortunately we do not have ( from FF ) a documented way to deploy the stack piece by piece in a multi server environment with some pieces already running ( well.. I can't find any ).

But I think you will give much more power to the CLI if you allow us to change ( including deleting ) things in the compose without using the override file.

[nguyer]

No problem 🙂

The local docker set up is a bit complex because part of the config needs to be written, processes started up, the a contract gets deployed, and the address of that contract has to be added to the config, and then have processes restart to pick up the change.

The docker-compose.yml file is generated and re-generated at one or two points during the first start which is why it can't be edited directly, because it would just get overwritten again.

[icemagno]

Hum... complex indeed. Now I see. Well I'll see what I can do. Thanks again. I think nothing can be done right now so you can close if you want.

[icemagno]

@nguyer But... can't we even remove some components like, besu or ipfs from compose?
It will be helpful if FF CLI can make assumptions if these elements are already there.
Obviously we MUST respect container names and ports to the evmconnector, signer and core be sure they are there....
I mean, if I already have an IPFS node up and running, the compose just ignore it own and use mine.
All I must do is respect same address and port expected by FF CLI...

And things like gas to publish contracts must be kept in mind. Mine is gas free so I may have no problem with this.

I think this may help a lot.

My blockchain is permissioned and I having a lot of trouble to adjust these things.