Radicle Developer Tooling // Jan-Mar 2024

Radicle Developer Tooling // Jan-Mar 2024

Over time, our team has taken on work on different parts of the Radicle ecosystem, helping build more tooling around the decentralized code collaboration solution.

This Radworks Grant application requests funding to continue ongoing work in the following areas, to cover the period between 15 Jan - 31 March 2024:

  • CI/CD, IDE and Zulip integrations
  • Migration Tooling
  • Planning Boards Tool

For each of these areas, we outline the specific work items we will focus on during this time. For each project, we outline:

a. the high-level focus areas,
b. how long we will invest working on these areas.

In the background, we are also trying to figure out a longer-term plan to better integrate our team into a more permanent role within the Radworks ecosystem, but that is beyond the scope of this proposal. Watch out for other discussions in this forum on that topic.

1. Integrations with Continuous Integration / Continuous Delivery (CI/CD) Tools

Full-time equivalent (FTE) Capacity: 89.5 FTE days

Ahead of a “1.0” launch for our decentralized code collaboration solution, it is important to be able to show integrations with existing CI/CD systems, so that users that would onboard to Radicle don’t have to worry about also migrating their pipelines. By integrating with their existing tools, we will have one less hurdle for adoption.

GitHub Actions

The GitHub Actions integration is already functional, at the proof-of-concept level.

Our focus areas here include:

  • Working examples: Enable this integration on at least 3 projects (jetbrains plugin, vs code extension, github adapter), to offer examples that new users who want to migrate from GitHub → Radicle can follow to set up this integration on their own projects.
  • Better reporting: Enrich the GitHub Actions adapter with more details for each workflow, enabling detailed results in the comment reported back to the patch about which specific workflow failed (as opposed to a simple “success / failure” comment)

Jenkins

As Jenkins is perhaps the most widely adopted self-hosted CI server, we should implement a first version of the plugin that allows integration with Radicle.

On the Radicle side, no extra implementation will be required: we will use the existing outgoing webhooks.

Our focus areas here include:

  • Implement a new Jenkins plugin that will:
    • process Radicle webhooks and trigger Jenkins jobs
    • provide a pipeline step that allows pipelines to send results back to patches (as comments, for the time being)
  • Publish documentation about how to deploy a Jenkins instance, install the Radicle Plugin and configure it with the Radicle Broker, Webhooks Adapter and HTTP API.
  • Working example: set up the CI pipeline of at least one project on Jenkins, on our public node.

Outgoing Webhooks

We are currently in a position to send basic outgoing HTTP webhooks to 3rd party systems. In this second iteration of work on the webhooks, we want to improve and extend the existing functionality.

Our focus areas include:

  • persist webhook results: One missing feature with the existing outgoing webhooks is that the result of the outgoing HTTP request isn’t currently persisted, so there is no detailed record of which webhooks have been successfully sent to the 3rd party system we are integrating with. We want to persist all webhook requests, so that it is also possible to retry failed requests, on par with what other outgoing webhook integrations offer in other Forges.
  • improve reporting: report detailed status per webhook to broker, as opposed to the current “all or nothing” approach we have today.
  • improve documentation: provide further docs & tooling on enabling webhooks, making them easier to setup, less of a chore, etc.
  • extend webhook events: add some new event types that are important for Radicle users.
    • “comment”: many users want to know when a new comment has been left on their patch. This is also used in integrations with 3rd party systems, to support “ChatOps” - style interaction with patches, such as for example test / re-test / approve comments that can trigger CI pipelines.
    • “review”: a patch review (and whether it has been approved or not) is one of the most important events to be notified about as a contributor.

Woodpecker

The Radicle Forge integration for Woodpecker exists as a proof-of-concept and has been shown to work.

In order to make it generally available however, it now needs to undergo a small rewrite as a “Woodpecker addon”. The addon system was added specifically to support additional Forges in Woodpecker, especially non-established ones like Radicle. As such, we need to port our current implementation to a separate addon, so that any Woodpecker user can choose to deploy it.

With that in mind, our focus areas here include:

  • Port current implementation to “Woodpecker Addon”
  • Complete documentation and create a guide explaining how to enable Woodpecker CI on Radicle projects.
  • Provide at least one example of a project that has a CI pipeline running on Woodpecker.

Concourse

The integration with Concourse CI was implemented a while ago, on the previous incarnation of the Radicle Broker. It does not currently work with the new Broker - Adapter architecture.

Our focus areas for Concourse include:

  • Migrate the existing business logic to a separate adapter, so that it is available for users who want to use Concourse CI and also serves as an example of how to implement similar integrations with other CI systems that offer an API.
  • provide documentation about how to deploy Concourse with Radicle and
  • publish at least one example project that has a pipeline on Concourse CI.

Explore Zulip Integration

With several teams (Heartwood, radicle-interface and our own) now relying on Zulip to fill in the missing functionality around working with Patches and Issues, we want to explore if we can use the Radicle Broker + Outgoing Webhooks Adapter to integrate some notifications in Zulip.

This integration can result in real-time notifications, updates, and alerts within the Zulip interface, ensuring that team members are promptly informed about relevant activities, discussions, and changes related to their projects. This aims to streamline the workflow associated with Radicle Patches and Issues, providing a centralized communication hub for all teams involved.

It should be noted, that we are not yet sure how much is possible here and what level of notifications we can have right now. We will spend some time exploring and hopefully be able to provide some kind of alpha version of a Radicle Zulip Integration.

2. Radicle in Integrated Development Environment (IDE) Tools

Full-time equivalent (FTE) Capacity: 67.75 FTE days

Jetbrains IDE Plugin

IntelliJ and other Jetbrains IDE users are the first Radicle users that can already properly review Patches by leaving general and inline comments (allowing in-place discussions about specific code changes to come to life) and creating/merging Patches directly from the IDE.

In our next release, we will focus on:

  • the last missing piece of the puzzle: Revision Reviews. This will be the only new feature.
  • The rest of the work will cover bugfixes, User eXperience (UX) and Development eXperience (DX) improvements and general “quality of life” improvements, while also ensuring the plugin stays up-to-date with the breaking changes in Heartwood and the radicle-httpd.

In more detail, here is the current plan (which is very likely to change, as work progresses) for the scope of the v0.9 Milestone.

VS Code Extension

In a nutshell, the plan for the upcoming v0.5 version is to support code reviews in VS Code and to complete the Patch workflow, by allowing users to both create, but also to merge patches. A lot of this work was targeted for v0.4, but has been pushed to v0.5 due to the need for important additional infrastructure groundwork in v0.4 - i.e. Webviews and refactoring the app’s state management into a reactive global store.

More specifically, you can see the currently planned functionality (which may change over time) on the v0.5 Milestone.

3. Migration Tooling

Full-time equivalent (FTE) Capacity: 14 FTE days

Migrating an entire project from another forge (GitHub, GitLab, Gitea) introduces a significant hurdle. By creating migration tools, we can lower the overhead and make it that much easier for teams to migrate their projects. Without this, the task is daunting and may be too high a hurdle to convince teams to switch tooling to Radicle.

Our team has already delivered tooling for Migrating GitHub Issues from GitHub → Radicle, as part of this Radworks Grant.

Our focus areas here include:

  • Allow users to migrate GitHub Wikis → Radicle
  • Allow users to migrate GitLab Issues → Radicle
  • Allow users to migrate Gitea Issues → Radicle
  • Extend the current implementation so that it can function as an always-on mirroring tool for GitHub Issues → Radicle. This will allow the tool to pick up new changes on GitHub issues and update the already created, corresponding Radicle issues, rather than creating new issues on every run.

4. Planning Boards Tool

Full-time equivalent (FTE) Capacity: 24.5 FTE days

One of the missing features that has been highlighted by every single team that is has moved their daily workflow to Radicle is some kind of board view of their backlog items, so the team can better plan their work and roadmap.

Based on stakeholder feedback on zulip, we have developed an early proof-of-concept, retroactively funded by a Radworks Grant.

In this Grant application, we would like to fund a little bit of further work in this area, so we can move beyond the proof-of-concept and ship a first alpha version of the application.

Our work areas here include:

  • Align with Heartwood team on what route to follow for persisting the necessary information for the boards
  • Align with the Web team on how to integrate the Planning Boards web app into the existing Web UI for radicle (radicle-interface)
  • Take codebase from “proof-of-concept” to “alpha” quality status: a lot of corners had been cut as part of the first PoC phase, that we now need to come back to, as we move towards making this actually usable.
  • Ship a hosted version of the planning boards in our own public node: this way there is no additional overhead on the Web team in terms of maintaining or operating this application.
  • New features (as time will permit):
    • filtering: to allow finding issues faster, or displaying only certain relevant features on the board (e.g. ones that have a specific label)
    • improve overall look & feel, also taking into account some suggestions from the Web team (thanks Daniel! :pray:)
    • editable issues: allow editing some issue fields directly from the UI.

Budget

  • Total Duration: 11 calendar weeks (Jan 15 - Mar 31 2024)
  • Full-Τime Εquivalent (FTE) Capacity: 195.75 FTE days
  • Total Budget: 82 500 USDC + 35 000 RAD
  • Recipient Wallet: 0x445717316388f1d1fb1730D3f6f9Bf59e0b03f4f

For the first time, we would like to request part of our payment in Radworks tokens (RAD), as both a testament of our belief in this project, and an indication of our long-term commitment to the Radworks community.

Note: The RAD amount has been estimated using the average price of the last 6 months (1.53 USD), based on historical data from coingecko, to account for volatily / instability.

Drips-Powered Payments

We would like to continue “drinking our own champagne” with Radworks product stack and would propose to use Drips (which has worked great so far!).

Legal Structure

  • Registered Legal Entity: Cytech Ltd.
  • Registered Address: Science & Technology Park of Crete, Heraklion, Greece

Team :busts_in_silhouette:

Team members

The team is comprised of 8 team members, all currently contributing to Radicle in a part-time capacity:

  • Yorgos Saslis
  • Ioannis Christodoulou
  • Stelios Mavrommatakis
  • Kostis Maninakis
  • Michalis Zampetakis
  • Zacharias Fragkiadakis
  • Themistoklis Dakanalis
  • Vaggelis Antoniadis

This is the same team :busts_in_silhouette: that’s delivered the other Radworks Grant-funded work. See previous grants for more info.

2 Likes

Hi @yorgos

Thanks a bunch for this.

I’m really excited about every item on this list. It reads like an essential list of ad-ons leading into v1.0.

I won’t leave too much other comments as most of this content has already gone through the wringer via the Org Proposal process over the last few months. So I’m quite familiar with it. Looks awesome, awesome, awesome!

Regarding Funding with RAD

First off, funding makes sense to me.

For reference to anyone else, the Grants Org itself used the same type of 6-month analysis to assess how much RAD to request previously

1 Like

Hello everyone!

A little bit of a late update here, but there sure is a lot to report on! :sweat_smile:

First and foremost, our team has finally graduated from the Radworks Grants
program into a new Radworks-funded Org: the Radicle Integrations & Tooling
Org
! :tada:

A lot of work has gone into the creation of this new Org, these past couple of months that involved exploring the most suitable legal entity setup, understanding all the requirements for new Radworks Orgs and, of course, finding the right positioning for our Org alongside the other Orgs in the ecosystem.

But here is what else we’ve been working on, while our new baby Org was incubating!

1. Integrations with Continuous Integration / Continuous Delivery (CI/CD) Tools

GitHub Actions Integration

  • Working examples: We have this working successfully on several repositories:
    • the Radicle Jetbrains IDE plugin repository,
    • the GitHub Actions Adapter itself and
    • the Radicle Core team’s radicle-interface repository (i.e. the Radicle Web UI) - coming soon.
  • Better reporting:
    • each message now reports more details for each workflow, in the comment reported back to the patch, allowing developers to know which specific workflow failed (as opposed to an overall “success / failure”
      comment).
    • we received feedback that the patch comment could also contain links to the artifacts produced by the GitHub Actions workflow. This feature is implemented and will be released soon - watch this space! :wink:
    • we have started working on a better UX for this, where, instead of multiple comments being sent to each Patch, a single comment will be updated as the GitHub Actions workflows progress.
  • Documentation & How-to Guides:
    • We have put together some new guides for Project Maintainers and Seed Node Operators, with more details on how they can set up CI Integrations with Radicle.

Jenkins Integration

We have put together a quick guide about how a Jenkins pipeline can be
triggered by HTTP webhook events (when the Radicle Webhooks Integration is
enabled):

We are planning to tidy this all up in a Jenkins shared library, in the weeks
ahead.

Outgoing Webhooks Integration

The latest version (v0.5.0) of the radicle-webhooks-adapter
comes with important improvements:

  • webhook invocations and results are now persisted in a local database (sqlite), so there is a detailed record of which webhooks have been successfully sent to the 3rd party system we are integrating with.

  • Much better docs!!! We have :new: CI Integration guides:

  • A CLI (Command Line Interface) tool that makes it easier for project maintainers to manage Radicle Webhooks in their repos!

    • To add some webhooks, run radicle-webhooks add --help.

The work to extend the adapter to emit more webhook events (such as “comment” or “review”) is left as future work, as it also requires changes on the Radicle CI Broker that we need to coordinate with the Radicle Core team.

Woodpecker CI Integration

Not much of an update on the Woodpecker front, as we have been waiting for a rework on their addons architecture (coming up in their 2.5.0 release), so we can port our implementation of the Radicle Forge integration to that new architecture.

Concourse CI Integration

The Concourse CI integration is fully functional (from v0.5.0 onwards) and is what our own team uses to self-host our own CI/CD workflows for our own projects, using a self-hosted Concourse CI Instance, as we are dogfooding the integration itself working towards bringing it to a more stable place:

Zulip Integration

No time left to explore that during the last couple of months.

Kraken CI

In collaboration with the Kraken CI maintainer, we have built support for Radicle in Kraken and we now have a detailed guide on how a self-hosted Kraken CI instance can run jobs from your Radicle repositories, powered by the Radicle CI Broker.

2. Radicle in Integrated Development Environment (IDE) Tools

Jetbrains IDE Plugin

For Jetbrains IDE we have focused primarily on keeping the plugin stable and compatible
with both the latest Radicle and Jetbrains IDE releases, focusing on just one new feature: Revision Reviews.

  • Patch versions 0.8.1, 0.8.2, 0.8.3 and 0.8.4 mainly included bugfixes, but also the ability to open the browser on a specific line from your IDE, as can be seen on the short video in the release announcement on Zulip.
  • v0.9.0 allows adding a review along with comments and also adds the reviews in the timeline view, alongside other fixes.

VS Code Extension

In VS Code, the v0.4.0 release was a feature-packed released with a huge amount of work in it, focused primarily around the overall Patch Detail view. The changelog provides a detailed list of all the important changes and features introduced with this release.

If you’re a VS Code user, grab your favourite beverage and take some time to review - there’s a lot to unpack there!!

3. Migration Tooling

As Radicle is getting closer and closer to its “1.0” stable release, we have been putting more work towards making migration of projects from other open source forges less of a problem for potential users.

The v0.5.0 and v0.6.0 releases of the Radicle
Migration Tool now make it possible to:

  • :white_check_mark: migrate GitHub Wikis → Radicle
  • :white_check_mark: migrate GitLab Issues → Radicle
  • :infinity: use this tool as an always-on mirroring tool for GitHub and GitLab Issues → Radicle. The tool picks up new changes on GitHub/GitLab issues and updates the already created, corresponding Radicle issues, rather than creating new issues on every run.

If you would like to try migrating your project from GitHub or GitLab to Radicle, please feel free to create a new topic and ask us about that on #integrations on zulip! :wink:

4. Planning Boards Tool

Plan… your Patches! (a.k.a. “who needs issues anyway?”)

Apart from issues, the board can now also display patches. This unlocks workflows where you don’t need to create an issue AND a patch for every single little task, but can instead keep the information in the patch itself and organize it alongside your issues.

There is a toggle on the board to switch between displaying issues, patches, or both.

Persistent card order

Card order within columns is now persisted in the issue/patch labels. This means that if you move a card to a new position in a column, it will stay there even after you refresh the page. This should make it easier to keep track of your tasks and priorities.

Column re-ordering

You can now re-order columns on the board as you see fit. This should make it easier to organize
your workflows and keep things tidy.

As of now, column order is persisted locally within the browser. Other users will not see the
changes you make to the column order.

Optimistic updates

When we added support for patches, as the information we needed to fetch increased, we started to see some performance issues when quickly moving cards around the board. To address that, we’ve introduced optimistic updates when moving cards around the board. This makes the experience seamless.

Miscellaneous

  • The labels of an issue/patch labels are now displayed on its card
  • Added the ability to filter out “done” tasks older than 2 weeks from the board
  • Tweaked the design of the board to bring it more inline with radicle-interface
  • Added an about link pointing to the radicle repo
  • Added a loading spinner to the board while it’s initially loading data

You can view some short videos of all these new features on the zulip announcement.

The latest version of the planning boards is also available on app.radicle.at, where you can see all these changes in action!

Conclusion

With this being the final grant from the Radworks Grants Org, and our team moving to a separate Radworks Org, I want to officially thank @bordumb , @abbey and all the other Radworks Grants Org team members who have helped and supported our team along the way. It is no exaggeration to say that, without them, we really wouldn’t be here today. Thank you!! :pray: