Grant Application - Package Manager

šŸ’ø Grants šŸ¤ Completed Grants
1

Radicle Grant Application - Package Manager

  • Project Name: Package Manager Research
  • Team Name: Chris Troutner
  • Payment Address: 0x4fdcEcf844B578d0190AFe267c498B2a0253A084
  • Level: :seedling:-Seed

Project Overview :page_facing_up:

The scope of this project is to conduct research into package management systems. The goal is to evaluate the work necessary to create an embedded package registry system within Radicle, similar to the GitHub Packages.

Radicle has done a great job decentralizing the features of GitHub, this grant helps Radicle take another step by decentralizing a package management system. Many programming languages have package registries, the JavaScript npm registry and Docker Hub being two of the most famous. Package registries are where code dependencies are stored and downloaded from. GitHub has decided that creating a package registry to compete with npm, Docker Hub, and other registries is a natural fit for their business. Itā€™s also a natural fit for Radicle: in addition to storing code, Radicle could also be used to store and fetch the dependencies needed by that same code.

This grant started from a conversation between Bordumb (Radicle Community) and myself. I founded the Permissionless Software Foundation DAO, and I wrote this research article encouraging other members of the PSF to back up their code to Radicle. I received a retroactive grant as a result of my efforts. The PSF is a consortium of entrepreures and JavaScript developers, and we are painfully aware of how dependent we are on the centralized npm service. Decentralizing our code with Radicle was a natural first step, and now weā€™d like to take the next step with Radicle by decentralizing npm dependencies.

Building a package reposity that could replace npm is mearly a step down a longer road. If that is accomplished, the package repository could be expanded to additional programming languages. It could even be integrated with operating-system-level package managers like brew and tea, which would allow Radicle to securely distribute binary files.

The scope of this project is not to do any of this work, but to simply research and develop a roadmap. Through discussions with Radicle community devs, weā€™ve created a list of several open-ended questions. This grant would pay for the reseach needed to answer these questions, and perhaps build a simple proof-of-concept.

Team :busts_in_silhouette:

Team members

  • Chris Troutner

Contact

Legal Structure

  • Registered Address: 1004 Commercial Ave #458, Anacortes WA 98221
  • Registered Legal Entity: Chris Troutner (Sole Proprieter)

Teamā€™s experience

Team Code Repos

Team LinkedIn Profiles

Project Description :page_facing_up:

The scope of this project is to conduct up to 100 hours of research and prototyping. This grant will be broken up into phases:

  1. The first phase is concerned with assessing the feasibility of using Radicle as a package repository, similar to npm or GitHub Packages. Initial research will focus on:
  • Generating an npm-compatible package from source code obtained from Radicle.
  • Signing packages with Radicle keys.
  • Uploading the package to a decentralized hosting site such as Radicle, Filecoin, or Arweave
  • Adapting Verdaccio to serve these Radicle packages, and fall back to npm if the package can not be found.
  1. Once the first phase is complete, a second phase of research may be conducted to evaluate the feasibility of serving binary files from Radicle, and the ability to integrate those packages into operating-system-level package manager such as brew or tea.

Deliverables :nut_and_bolt:

In the spirit of Radicle open source, the research will be published in blog-style format, and a final report will summarize the results of the research for posterity. If the research takes less than 100 hours, then funds will be returned to Radicle DAO. There is a good chance that this research will spawn another grant.

  • Total Estimated Duration: (up to) 100 hours
  • Full-time equivalent (FTE): 12.5 FTE days
  • Total Costs: $10,000

Milestone 1 - npm Compatibility

  • Estimated Duration: 60 hours
  • FTE: 7.5 FTE days
  • Costs: $6,000
Number Deliverable Specification
1. Package code Generate an npm-compatible package from source code obtained from Radicle
2. Package signing Sign packages with Radicle keys
3. Package upload Research best way to upload packages
4. Verdaccio integration Research the best way to integrate with Verdaccio

At the end of the first milestone, we should have a very clear picture of the feasibility for using Radicle as a package repository. Verdaccio is an npm proxy. The goal is to have it preferentially serve packages from Radicle, and then fall back to npm if it canā€™t find the package. In this way, the existing user experience is unchanged, which is good UX. The users can simply ā€˜opt inā€™ to using decentralized infrastructure without changing their workflow at all.

Verdaccio however is not our only option, just the most apparent low-hanging fruit. As new information comes to light during this research, the plan will adapt.

Milestone 2 - Operating System Packages & Binaries

  • Estimated Duration: 20 hours
  • FTE: 2.5 FTE days
  • Costs: $2,000
Number Deliverable Specification
1. Brew & Tea Integration Research integration with brew and tea
2. Binary signing & delivery Research modifications needed to signing and upload for binary files vs npm packages

After the first milestone is complete, and if there is still budget left, the research will be extended to operating-system-level package managers. This is a ā€˜level upā€™ from code package managers. The goal of this step is to be able to distribute arbitrary binaries, and sign them with Radicle keys, in a format that can be integrated with brew or tea.

Milestone 3 - Final Report

  • Estimated Duration: 20 hours
  • FTE: 2.5 FTE days
  • Costs: $2,000
Number Deliverable Specification
1. Final Report Summarize the research and recommend next steps

The final deliverable for this grant will be a report. It will summarize the results of the research and recommend paths forward for Radicle. It will discuss the technical trade-offs between the different paths forward.

Future Plans

GitHub Packages provides an excellent blueprint for Radicle to improve upon, in order to implement their own package registry. It is highly likely that this research project will result in a second grant, to build out the repository features into Radicle, including documentation.

As a developer who loves JavaScript and decentralized tech, Iā€™m extremely excited at the possibility to have a decentralized replacement for npm. The developers who are a part of the Permissionless Software Foundation will eagerly dogfood this technology. We work hard to promote censorship-resistent software.

3 Likes
2

This all sounds good to me :+1: :seedling:

I think something to keep an eye out for is your research may end up inspiring one or more RIPs in the case of finding improvements for the protocol.

Something Iā€™d be interested in seeing in a deliverable is less so a report, and more a document outlining what the implementation might look like ā€“ akin to an RFC. I think that would make it easier to evaluate the research and show how feasible it is to implement :slight_smile:

2 Likes
3

Thatā€™s a good point. ā€˜Reportā€™ in the context of the grant is intentionally generic. Iā€™ll keep your suggestions in mind, and try to incorporate them into the report. I agree that it should contain information on feasibility and recommend changes (if needed) to the Radicle protocol. It may very well result in a new RIP.

1 Like
4

Considering we already have mature package repositories like Nexus or Artifactory, both of which are open source software and therefore available to be self-hosted, it is not entirely clear to me why it makes sense to invest in some new package repositoryā€¦ (Especially considering those already support many more packaging formats than NPM.) What would be the Unique Selling Proposition (USP) and competitive advantages for such a package repository over the existing solutions?

GitHub packages may be a centralized offering, but there are already options for self-hosting package repositories out there and if each org were to host their own packages (signed with their org keys) on an org-hosted instance of an existing solution, it sounds to me like this is a pretty decentralized setup when looked at from a distanceā€¦

Perhaps Iā€™m missing something, so can you please help me understand how the proposed research here would help us improve on a setup similar to the above ?

3 Likes
5

This is an excellent objection, and itā€™s an opportunity to clarify the scope of the grant.

The software world is full of mature package managers. It doesnā€™t make sense for Radicle to go head-to-head with them in a competitive spirit.

Let me paraphrase the conversation that inspired this grant proposal:

ā€œIf Radicle can store and serve code, it shouldnā€™t be that hard to store and serve packages.ā€

Thatā€™s what makes this research a natural next step for Radicle. The scope of the research isnā€™t to evaluate how Radicle could compete with package managers, but instead, itā€™s to evaluate how Radicle could integrate with existing package managers.

This is why Verdaccio makes a good use-case: By integrating Radicle with it, users can simply opt-in to using a decentralized package repository (Radicle) without changing their workflow. Throughout the course of this research, I will be looking for similar opportunities to integrate with package managers for other languages.

The example you give of organizations self-hosting their own package repositories is a good one. If that org has made the decision to integrate Radicle, it would be wonderful if that integration could cover both code and packages. And it would be even better if there was an easy way to integrate Radicle (on the back end) with their ā€œorg-hosted instance of an existing solutionā€.

1 Like
6

@christroutner

Regarding this point:

ā€¦I will be looking for similar opportunities to integrate with package managers for other languages.

Yeah, depending on what you find here, I could imagine this essentially helping to automate a lot of that integration.

Itā€™s a bit of a handy wavey concept at the moment, but I definitely think this is something worth at least doing some due diligence on to see what, if any, interesting comes out of it.

7

Thanks for clarifying this ! :+1:

How this integration would look like does sound like an interesting area to explore further, yes!

You will find already quite a lot of discussion that I think is related to this, in this forum topic: Software Releases on Radicle - #27 by kim

We have spent quite some time there, discussing what ā€œReleasesā€ on Radicle would look like. I think thatā€™s relevant because ā€œReleasesā€ are the entities that typically carry information about packages stored in package repositories. I think it is also particularly important to take into account the discussion around ā€œwhat constitutes a canonical release in a decentralized settingā€.

Having said that, there are 2 areas we explicitly excluded from that discussion:

  • ā€œhow do we produce these binariesā€ and
  • ā€œwhere/how do we actually host these binariesā€

Considering this research is about integration with the package repositories, it sounds to me like those would be worthwhile areas to explore.

I hope that sounds in line with what you also had in mind and you find this helpful rather than confusing things furtherā€¦ But please do let me know if youā€™d like any further clarifications on the above or if youā€™re thinking about taking this in some different direction! I would love to hear more on this!

2 Likes
8

Hey @christroutner!

Thanks for the application :seedling: Iā€™m generally excited to see what this becomes of this research! As @yorgos mentioned, the topic has been raised on the forum previously but it seems like this grant could be a good continuation of that discussion.

Understanding artifact management for Radicle is a super interesting topic that I know the Clients team has been discussing recently. It seems like you understand Radicle well (great posts!) so as long as milestones are reviewed with the right people (Iā€™m thinking @fintohaps @cloudhead from Clients, and @yorgos) Iā€™m supportive of funding this grant!

Looking forward to seeing the first deliverables :+1:

2 Likes
9

@christroutner

Weā€™ve started the transaction for Milestone 1.
This covers the initial 20% of the 6,000 USDC, so 1,200 USDC to start.
The remaining 4,800 will be paid upon review and approval of the delivery.

It may take several days to reach quorum and you can track the progress here:

Note: any taxes will be the responsibility of the grantee according to their local jurisdiction.

1 Like
10

Here is a summary report that covers the first part of the first milestone:

11

Hello Radicle community and Grant reviewers.

The research for this grant is complete, and the final report is ready for review. Here are some handy links:

During the research, limitations were discovered which hindered the ability to to directly integrate Radicle with package mangers like npm, brew, and tea. As a result, Milestone 2 was dropped from the scope of the research. The final report recommends some next steps to overcome the limitations that were discovered.

However, for motivated JavaScript developers, the research lays out some workarounds in order to use Radicle as a package repository for working with npm packages.