[#radicle-biweekly] 15.11.19

abbey · November 14, 2019, 11:00pm

We just pushed a working draft for our new protocol spec (WIP) on our new radicle-link repo. Discussion, feedback, and questions are welcome.

If you haven’t read our previous update, be sure to check it out for an overview of where we’re going with Radicle.

github.com/radicle-dev/radicle-alpha

Roadmap

opened 03:16PM - 31 Oct 19 UTC

kim

At Monadic, we've taken a short hiatus on Radicle to think about our internal st…rategy and reflect on how we're allocating our energy and resources. Here's an update on what came out of that: Radicle, as conceived, is much more general than code collaboration: it allows one to give (almost) arbitrary semantics to deterministic replicated state machines. While this is interesting in itself, we don't think it is worthwhile _for us_ to pursue a vision of general-purpose distributed computation. Instead, what we’re interested in is to make code collaboration work in a decentralised setting. This does include state machines modelling the lifecycles of collaborative interactions (issues, pull requests, etc.), yet it also includes sharing code and changes to it, which we think we haven’t captured too well with the current architecture. **So what does this mean?** Our biggest issues with our current stack have been, unfortunately, with IPFS. It is conceptually quite close to what we wanted (which is why we chose it in the first place), but turned out to be a bit unwieldy in practice: * It is a very heavy dependency, has a number of rough edges, and works like a monolithic black box from our point of view. * The current architecture of Radicle requires a daemon to be reachable on the network in order to receive updates from collaborators. For that, we relied on IPNS and IPFS PubSub, both of which are not particularly performant. In fact, we had to create our own IPFS network to make it work _at all_. * Lastly, replicating `git` repos peer-to-peer _on the storage layer_ doesn't leave much choice but to use ["loose" `git` objects](https://git-scm.com/book/en/v2/Git-Internals-Git-Objects) in order to preserve content-addressability. That means, however, that we lose the main optimisation which makes `git` ~~fast~~usable for interactive use: [packfiles](https://git-scm.com/book/en/v2/Git-Internals-Packfiles). Note that this is not specific to IPFS: "feed" based systems like Dat and SSB have the same issue (GitTorrent does a lot better, but being tied to the BitTorrent network has its own drawbacks). If we want a usable system for _code_ collaboration, the last point is crucial. In our current approach, the source code is a bit of a 'second-class citizen': there are no strong guarantees that the repo is available, and if you're working with a large history you're better off hosting your code elsewhere. We've been embedding patches (as per `git format-patch`) in the Radicle state machine to be able to talk about contributions without the notion of a repo in Radicle itself. This works in practice (just as well as exchanging patches over E-mail), but the ergonomics quickly become questionable if you consider non-linear histories, and workflows where the submitter builds on top of patch series not yet applied to the published base. Also, we can't recover the repo from the Radicle state machine. **So what now?** The most obvious thought when reflecting on the above is: why not turn the system around, and use `git` itself to distribute data (it is _designed_ to be used in a distributed fashion, isn't it?). Storing _collaboration_ artifacts (issues, pull requests, review comments, ...) in `git` has been done before also, and the data structure of a commit history satisfies all storage needs for a state machine log. We will need to build our own networking layer, but we think that this is exactly what we need more control over, based on our experience with IPFS. We’re still working on figuring out the details of this approach and plan to share a first draft of a protocol spec soon. Over the coming months, we’ll be focusing on implementing this lower level of the stack. As for the code we already have, we want to: * Release `radicle-lang` separately We think that the "pure core" can and should be its own thing. The language may evolve on its own, and although it isn't going to be our main focus for the time being, a deterministic language with controllable effects is a good asset to have -- being "hackable" has always been a nice feature of Radicle which we don't want to give up on. Work already went into this (#684), but there's probably more to do until it could "graduate" into its own project. For example, we haven't made up our minds yet what to do with the existing `rad` code in lieu of a native packaging system. If you would want to help out on this, please reach out! * EOL the IPFS backend This would mainly mean that we shut down the seed nodes for our own IPFS network, as we don't have the bandwidth to properly operate them. We will be moving away from IPFS, and most likely also from the client-server architecture (which is why there hasn't been much progress on #670 for quite a while). We would be open to transitioning this stack to the community, if there is interest. Sunsetting the seed nodes is planned to happen once we have something new to check out. We hope this gives you an idea of where we’re at. Please let us know of any questions or concerns here or #radicle on freenode.