abbey
October 31, 2019, 11:00pm
1
Hello from the Radicle team
After a short hiatus, we’re excited to get back to our Radicle development updates. We’ll be sending these out bi-weekly to keep you all updated on development, releases, and next steps.
opened 03:16PM - 31 Oct 19 UTC
At Monadic, we've taken a short hiatus on Radicle to think about our internal st… rategy and reflect on how we're allocating our energy and resources. Here's an update on what came out of that:
Radicle, as conceived, is much more general than code collaboration: it allows one to give (almost) arbitrary semantics to deterministic replicated state machines. While this is interesting in itself, we don't think it is worthwhile _for us_ to pursue a vision of general-purpose distributed computation. Instead, what we’re interested in is to make code collaboration work in a decentralised setting. This does include state machines modelling the lifecycles of collaborative interactions (issues, pull requests, etc.), yet it also includes sharing code and changes to it, which we think we haven’t captured too well with the current architecture.
**So what does this mean?**
Our biggest issues with our current stack have been, unfortunately, with IPFS. It is conceptually quite close to what we wanted (which is why we chose it in the first place), but turned out to be a bit unwieldy in practice:
* It is a very heavy dependency, has a number of rough edges, and works like a
monolithic black box from our point of view.
* The current architecture of Radicle requires a daemon to be reachable on the
network in order to receive updates from collaborators. For that, we relied on
IPNS and IPFS PubSub, both of which are not particularly performant. In fact,
we had to create our own IPFS network to make it work _at all_.
* Lastly, replicating `git` repos peer-to-peer _on the storage layer_ doesn't
leave much choice but to use ["loose" `git` objects](https://git-scm.com/book/en/v2/Git-Internals-Git-Objects) in order to preserve
content-addressability. That means, however, that we lose the main
optimisation which makes `git` ~~fast~~usable for interactive use: [packfiles](https://git-scm.com/book/en/v2/Git-Internals-Packfiles).
Note that this is not specific to IPFS: "feed" based systems like Dat and SSB
have the same issue (GitTorrent does a lot better, but being tied to the
BitTorrent network has its own drawbacks).
If we want a usable system for _code_ collaboration, the last point is crucial. In our current approach, the source code is a bit of a 'second-class citizen': there are no strong guarantees that the repo is available, and if you're working with a large history you're better off hosting your code elsewhere. We've been embedding patches (as per `git format-patch`) in the Radicle state machine to be able to talk about contributions without the notion of a repo in Radicle itself. This works in practice (just as well as exchanging patches over E-mail), but the ergonomics quickly become questionable if you consider non-linear histories, and workflows where the submitter builds on top of patch series not yet applied to the published base. Also, we can't recover the repo from the Radicle state machine.
**So what now?**
The most obvious thought when reflecting on the above is: why not turn the system around, and use `git` itself to distribute data (it is _designed_ to be used in a distributed fashion, isn't it?). Storing _collaboration_ artifacts (issues, pull requests, review comments, ...) in `git` has been done before also, and the data structure of a commit history satisfies all storage needs for a state machine log. We will need to build our own networking layer, but we think that this is exactly what we need more control over, based on our experience with IPFS.
We’re still working on figuring out the details of this approach and plan to share a first draft of a protocol spec soon. Over the coming months, we’ll be focusing on implementing this lower level of the stack.
As for the code we already have, we want to:
* Release `radicle-lang` separately
We think that the "pure core" can and should be its own thing. The language
may evolve on its own, and although it isn't going to be our main focus for
the time being, a deterministic language with controllable effects is a
good asset to have -- being "hackable" has always been a nice feature of
Radicle which we don't want to give up on.
Work already went into this (#684), but there's probably more to do until it
could "graduate" into its own project. For example, we haven't made up our
minds yet what to do with the existing `rad` code in lieu of a native
packaging system. If you would want to help out on this, please reach out!
* EOL the IPFS backend
This would mainly mean that we shut down the seed nodes for our own IPFS
network, as we don't have the bandwidth to properly operate them.
We will be moving away from IPFS, and most likely also from the client-server
architecture (which is why there hasn't been much progress on #670 for quite a
while). We would be open to transitioning this stack to the community, if there is
interest. Sunsetting the seed nodes is planned to happen once we have
something new to check out.
We hope this gives you an idea of where we’re at. Please let us know of any questions or concerns here or #radicle on freenode.
@fintohaps @kim are our dev leads. Hit them up for any and all questions regarding Radicle
We’ve been working on what’s next for Radicle. Check out our initial thoughts 
@fintohaps @kim are our dev leads. Hit them up for any and all questions regarding Radicle
Monadic is hiring full-time Radicle contributors. get in touch in you’re interested and check out the hiring thread for more info.