In a Polkadot hackaton dubbed as Buildathon: India, my Opsumo team aims to take part in the category of "Blockchain Using Substrate, Open Hack ( Dev Tooling / Block Explorer / Block Visualiser etc.)"
A Cloud Native Computing Foundation compliant DevOps culture solution.
At its simplest but profoundly elegant way, we intend to demonstrate a CNCF compliant DevOps culture solution. We have resolved to provide a way to automatically spin-up test infrastructures for hosting Polkadot blockchains, components, networks and tools into hybrid/multiple clouds. In order to attain CNCF compliance we believe that the best cloud-native platform that can host the target objectives is to use Kubernetes. We however further argue (highly opinionated in some quarters), that in order to achieve a uniformly written automation script, we have to leverage the most mature Kubernetes platform provider in the market, i.e Redhat Openshift. As secondary objective, we also intend to implement a demo of an automated deployment pipelines that can effectively trigger Continuous Integration and Continuous Deployment (CICD) of a desired Blockchain configuration. And in addition, we also have to provide, on a per need or on-demand basis, a way to provision and deploy a corresponding set of monitoring tools. CICD is the heart of DevOps, and fully realizing its implementation and gain from its benefits is the quintiessential ideals of the culture.
Autotmated DevOps Tooling is what we aim to accomplish in this Buildathon: India 2021.
First off, is the why? Why it is essential?
-
We recognize that for a would be layer-1 or 2 app development like Parachains, Parathread, Bridge, Validator, Collator, Smart Contracts, NFT, DeFi, etc starting from scratch is hard. So what if, there exist pre-baked set of initial Infra as Code provisioner, Cluster, Network and Security setup that a would-be team could leverage and use to scaffold their own use-cases as a foundational building block? Imagine the win-win benefit it could give for the Polkadot Network community. Speed of development could certain attain greater velocities, and with it, a more lively and active community. For the teams using it, again, the value is speed to market.
-
Like in the first, what if we offer a service for a substrate-node-template to be readily provisioned in a cloud infra and all the she-bangs needed to speed-up development of other custom blockchain that will address a use case, such as DeFi, smart-contract or another cryto-tech.
Secondly is the what? While Docker it seems still is the status quo, and we can see that it simply works, we can't argue with that! But why do we have to introduce Kubernetes, and even more so, Redhat Openshift?! For so many reasons we don't have to repeat saying, like the explanations from the following articles:
- K8 vs Docker primer
- K8 vs Docker by SumoLogic
- K8 vs Docker by NewRelic
- K8 vs Docker by IBM
- K8 vs Docker by Microsoft
However, few of the most compelling DevOps arguments namely networking, security, automated scheduling, scaling and high-availability in production environments are the key features that Docker can never achieve on its own. Not even with Docker Swarm -- until finally Docker, Inc themselves have raised the white flag, and recognized the supremacy that Kubernetes can provide, so they have to adapt to focus on tooling. Try running docker images each time in a cluster and configure all these each time, like snowflakes, it easily crumbles and is quite a painful and a stressful process to remediate or fix. Multiply these challenges when you try to deploy hundreds to thousands of these in multiple clouds, and an ocean of differences will be more evident, and cycle of re-inventing the wheel commences.
Remember, the DevOps culture aims to address the "joy" of being in either of the role as Developer and consequently do Operations or vice versa (as SRE), i.e. to function dually effectively and reliably. By far, only Kubernetes have nearly been able to achieve the DevOps/SRE ideals, i.e. of enabling a platform for automated integration and deployment, at scale, securely and covering as much of the many architectural ilities concerns. Albeit the benefits, we also acknowledge the fact that it is really hard to learn it at first.
But why Openshift, again? Well, like Linux, Kubernetes is likened to a kernel, it needs a distribution with a specific set of tools, and by far, Redhat's Kubernetes tooling is the most superior among the many that are out there. Take the case of CRC (Code Reader Containers), a desktop app that allows us to quickly use Openshift flavored Kubernetes in our laptops, i.e. for those with well-endowed resources. Furthermore compelling are the following explainers:
How?
Upon introspection from the initial attempt to successfully execute Phase 0, to do a quick and dirty semi-automatic stateless deployment of a ParityTech released Polkadot Image with a UI viewer, we have arrived to a conclusion that the following Phases and targets could also be achieved, even though not necessarily within the Buildathon timeline.
The following other important Phases (can be interchanged also as a Priority) are also soft-targeted for implementation during the Buildathon:
- Phase 1: Deploy2K8 Test Polkadot Relay Chain + (Validators x n + Parachain + Collators x m) x l + CICD
- Phase 2: Deploy2K8 Test (Custom Chain x n) + UI + CICD, via Substrate
- Phase 3: Deploy2K8 Monitoring Tools
- Phase 4: Deploy2K8 Secure Validators + CICD, Deploy2K8 Secure Collators + CICD, Deploy2K8 Secure Parachain + CICD, Deploy2K8 Secure Fisherman + CICD
- Phase 5: Deploy2K8 Generic Test (Custom Chain(with own tokenomics) x n) + UI + CICD
- Phase 6: Deploy2K8 Smart Contract Chain + CICD
A couple or several of the Phases enumerated above can be implemented in parallel. E.g., P1 and P2 can be targetted for implementation during the Buildathon from April 29 to May 30. While the other phases could be implemented at other timelines, ParityTech and W3 permitting.
Setup the github secrets, do the clone and checkout like below, update the ./.github/workflows/openshift.yml, then git push in develop branch
git clone https://github.com/opsumo/polkadot-cn-devops-priv.git
git checkout -b develop
vi README.md #Do some change, then save :wp
git commit -a -m "any good context"
git push origin develop