I’ve been thing about a way to integrate devops into a radicle workflow and wanted to share my thoughts and see what other people are thinking, or may have already done.
The immediate use case under consideration are actions initiated by code commits. Such events are useful for running tests, building artifacts, and performing updates.
This line of thinking leads to 2 areas, change detection and action result reporting.
The first seems possible by following a user’s Device ID with some automated upstream client (or remote-follow-agent). This remote-follow-agent can monitor a project for changes and run tasks described in the project itself. With this subscription model, many actors can follow the same project for redundancy and there is no need for central coordination of who gets what events. You could imagine different remote-follow-agents performing different actions based on evaluation of what has changed with a project and their particular configuration. Maybe they ignore certain types of changes, act differently on others, etc.
The second is more complicated is a decentralized scenario where actors may be malicious. How do you verify the job was done correctly? How are results recorded and referenced?
These are just some ideas I’ve had, and have heard discussed and I am really curious if anyone else is thinking about them.
We’ve been thinking about this for some time. I think the approach you suggest is the right one: on push to the rad remote, either some remote node can pick up the change and run a job, or your local node can do so, and call a remote service to trigger a job.
In terms of trust, I think the simplest model is to define which service you trust (by public key) and have that service sign the job results. The client can then verify these results locally, and if the signature is valid, display the result to the user.
There are more expensive and decentralized variations of this, for eg. by dispatching the job to three workers and checking that they produce the same output, but in terms of bang for the buck, I think the approach above is more effective.
I’ve got an idea how to practically do the trustless part for build verify game: outsource it to https://truebit.io/ which is a protocoll for challange/verify type games with a running wasm vm on ethereum mainnet. An assembly emulator, running linux, could be compiled to wasm which executes all build tasks and pushes results to ipfs.
I’ve written a small POC for this use case here. Its a RISC-V VM emulator running buildroot linux compiled to wasm and running on truebit. Although it has verry limited capabilities (running bash is all you can do for now) and to slow to be used on real world use cases (truebit wasm interpreter is ~600x slower then nodejs native wasm interpreter, also a similar slowdown is likely running native code vs. running it in a VM emulator with nodejs wasm), it demonstrates how this overall aproach is possible.
Currently truebit-wasm interpreter is not written with performance in mind so that there is a lot of room for improvement.
What do you mean by that exactly?
The builds and tests can’t depend on external sources comming from the network or user input: all dependencies and inputs must be specified before running the entire vm. But any nondeterminism in tests or builds (e.g. tests depending on local time or randomness or concurrency) will be deterministinc since the VM emulator is deterministic: providing mock functions for randomness and time, also linearization of multiple threads is done deterministically so all builds and tests will provide the same outputs for the same inputs, with no special requirements from the users.
I just raised this topic in the general channel on Element.
I think it’s feasible if you introduce a consensus mechanism for build artifacts that are stored in the network.
I see it as a full node optionally enabling CI/CD services. Producing the build artifact, getting a calculated checksum when published to the network. Then the UI would show a relative trust score based on the checksums from other nodes. And could be repurposed by users to install their own full node, and using the feature to produce their own builds of code from the network.
I’m seeing this feature’s primary role is as a Radicle focused package management tool. Like Cargo, Apt, or Maven. Which I’d find incredibly valuable. Paid DevOps services can be explored later by those that are interested.