Grant Application - Decentralized CI/CD Pipeline

Radicle Grant Application

  • Project Name: Decentralized CI/CD Pipeline
  • Team Name: BuidL DAO
  • Payment Address: 0xab8A95f711f9B85898BDd1D405F2ffF9ce8CeC64
  • Category: Core Infra OR Radicle Adoption

Project Overview :page_facing_up:

This project aims to build a decentralized continuous integration and continuous delivery (CI/CD) pipeline for open-source software development. The platform will use peer-to-peer networking and cryptography to ensure the security and privacy of the build process and will leverage existing decentralized infrastructure, such as IPFS and Ethereum, to distribute builds and artifacts. The platform will be designed to be modular and extensible, allowing developers to use a wide range of programming languages and build tools. My team has a clear plan for how it will use the Radicle grant to build a proof-of-concept implementation of the platform, and improve it, including adding support for more build tools and integrating with other decentralized development tools.

Overview

Please provide the following:

  • Decentralized CI/CD Pipeline.

  • The project is focused on building a decentralized continuous integration and continuous delivery (CI/CD) pipeline for open-source software development. CI/CD pipelines are an essential part of the software development process, allowing developers to automate the testing, building, and deployment of their code.

  • This project aligns with Radicle’s focus on building decentralized infrastructure for open-source software development. By providing a decentralized CI/CD pipeline, this project would make it easier for developers to build and deploy software in a secure and scalable manner, while also promoting the values of privacy and user control that Radicle prioritizes.

  • My team is interested in creating this project for several reasons, including:

Innovation: Building a decentralized CI/CD pipeline is an innovative and forward-thinking project that can help your team stay at the forefront of modern software development practices.

Security: Decentralization can increase the security and resilience of your software builds, ensuring that your team’s software is not subject to single points of failure, data breaches, or other security risks.

Collaboration: The decentralized approach to CI/CD enables more collaboration and coordination between different stakeholders, including developers, testers, and operations teams.

Flexibility: A decentralized CI/CD pipeline can be highly flexible and customizable, allowing your team to build a platform that meets your specific needs and requirements.

Community: The open-source and decentralized nature of the platform can help build a community around the project, which can provide opportunities for collaboration, feedback, and contributions.

Cost savings: By using decentralized resources, your team can potentially save on infrastructure costs, while still benefiting from a scalable and resilient CI/CD pipeline.

Overall, creating a decentralized CI/CD pipeline project can provide numerous benefits to both Radicle and our team, from increased security and resilience to greater collaboration and innovation.

Team :busts_in_silhouette:

Team members

  • Decent Okeh
  • Jordi Piñana
  • Jordi Moraleda
  • others

Contact

Legal Structure

  • Registered Address: Not a registered legal entity
  • Registered Legal Entity: Not a registered legal entity

Team’s experience

Please describe the team’s relevant experience.

We have a distributed team of mixed frontend/backend/full-stack engineers in total and they will be working on this project. the team has 25+ years of experience and has worked with some of the largest software engineering organizations in the world.

Decent Okeh (tigthor): I have spent the last six years developing my skills as a web3 developer, where I have won several performance awards and been promoted twice. I love managing teams and solving customer problems. I’ve contributed to and managed numerous projects, and used my experience to make some.

Jordi Piñana: Software developer with 4 years of experience in Blockchain programming. He is also teaching at the Polytechnic University of Catalonia blockchain master’s degree program.

Jordi Moraleda: Senior full-stack engineer. He is heavily involved in the architecture, research, and integration of the components that make things possible. Integrated and validated zk-SNARKs on mobile devices to generate proofs within a matter of seconds. He is also in charge of the UI and smart contract components of the platform.

Team Code Repos

Team LinkedIn Profiles (if available)

Project Description :page_facing_up:

The project is focused on building a decentralized continuous integration and continuous delivery (CI/CD) pipeline for open-source software development. CI/CD pipelines are an essential part of the software development process, allowing developers to automate the testing, building, and deployment of their code. However, many existing CI/CD pipelines rely on centralized servers and infrastructure, which can be vulnerable to attacks and can limit the privacy and control of developers.

To address these issues, we are building a decentralized CI/CD pipeline that leverages existing decentralized infrastructure, such as IPFS and Ethereum. This means that instead of relying on a centralized server to build and deploy code, the pipeline will use a distributed network of nodes to share resources and execute builds in a secure and scalable manner.

The platform will use cryptography to ensure the security and privacy of the build process. For example, the build artifacts will be encrypted and stored on IPFS, a decentralized storage network, to prevent unauthorized access. Additionally, the build process will be secured using cryptographic signatures and other techniques to prevent tampering or unauthorized modifications.

The decentralized CI/CD pipeline will also be designed to be modular and extensible, allowing developers to use a wide range of programming languages and build tools. The pipeline will support popular build tools like Gradle, Maven, and NPM, as well as emerging technologies like WebAssembly. The pipeline will also be designed to be customizable, allowing developers to specify their build and test scripts and integrate them with other decentralized development tools.

Our team recognizes that there are still many challenges to overcome, such as ensuring the scalability and reliability of the platform and integrating it with other decentralized development tools.

To address these challenges, we plan to use the Radicle grant to improve the platform in several ways. For example, we plan to add support for more build tools and testing frameworks, as well as develop a more sophisticated system for distributing build resources across the network. We also plan to integrate the platform with other decentralized development tools, such as version control systems and code review tools, to create a more seamless and integrated development experience.

Overall, the decentralized CI/CD pipeline project is an innovative and exciting example of how decentralized infrastructure can be used to improve the software development process. By providing a secure, scalable, and modular platform for building and deploying software, this project will help to promote the values of privacy and user control that are important to the Radicle community, while also enabling developers to be more productive and efficient in their work.

Deliverables :nut_and_bolt:

Please list the deliverables of the project in as much detail as possible. Please also estimate the amount of work required and try to divide the project into meaningful milestones.

  • Total Estimated Duration: 4 months
  • Full-time equivalent (FTE): 66 days
  • Total Costs: $50,000

Milestone 1

Please add additional milestones in the same way:

  • Estimated Duration: 3 months
  • FTE: 40 days
  • Costs: $35,000
Number Deliverable Specification
1. Build scheduler - Design documentation for the build scheduler
- Basic scheduler implementation in Python
- Integration with Docker and Kubernetes
2. Build runners - Design documentation for the build runners
- Build runner implementation in Rust or Go
- Integration with GitHub Actions or Jenkins
3. Build artifacts repository - Design documentation for the build artifacts repository
- Artifact storage and retrieval functionality in JavaScript
- Integration with decentralized storage networks like IPFS or Filecoin
4. Decentralized control plane - Design documentation for the decentralized control plane
- Implementation of a decentralized consensus protocol like IPFS Pubsub or libp2p
- Integration with the build scheduler and runners

Milestone 2

Please add additional milestones in the same way:

  • Estimated Duration: 1 month
  • FTE: 26 days
  • Costs: $15,000
Number Deliverable Specification
1. User interface - Design documentation for the user interface
- Implementation of a web-based dashboard for the platform
- Integration with other Radicle tools like Radicle Link
2. Testing and optimization - Test suite for the entire platform
- Performance optimization of all components
- Security audit and vulnerability assessment
3. Launch - Public launch of the platform
- Documentation and user guides
- Ongoing support and maintenance of the platform

Future Plans

Please include the following:

There are several general strategies that we’ll use to sustain the future of the decentralized CI/CD pipeline project in the long term:

  • Community building: To sustain the project, we’ll focus on building a strong and engaged community around the decentralized CI/CD pipeline. This could involve creating documentation, tutorials, and other resources to help developers learn how to use the platform, as well as fostering a culture of collaboration and contribution through events, forums, and social media.

  • Business model: we’ll explore various business models to sustain the project financially. For example, we’ll offer paid support, consulting, and training services to companies that use the platform, we’ll also monetize the platform by charging a fee for premium features or hosting services. Lastly, we’ll use a crowdfunding model, such as a community-supported development (CSD) model, to fund the ongoing development and maintenance of the platform.

  • Partnerships: we’ll also explore partnerships with other companies or organizations in the decentralized development space to help sustain the project. For example, we’ll partner with companies that provide decentralized storage or hosting services to integrate those services with the CI/CD pipeline platform, we could also partner with other decentralized development tool providers to create a more integrated and comprehensive platform.

  • Open-source collaboration: Finally, we’ll leverage the power of open-source collaboration to sustain the project. By making the platform open source, we are able to tap into a global community of developers and contributors who can help improve the platform, fix bugs, and add new features. This can help ensure that the platform remains up-to-date and relevant, while also fostering a sense of ownership and participation among the developer community.

Overall, sustaining a decentralized development project like the CI/CD pipeline platform requires a combination of technical expertise, community building, business acumen, and strategic partnerships. By leveraging these strategies and remaining flexible and adaptable, we’ll help ensure that the project continues to thrive and meet the evolving needs of the decentralized development community.

Additional Information :heavy_plus_sign:

How did you hear about the Grants Program? Medium.

In addition to the information provided earlier, there are a few other key considerations that are relevant to the development of the decentralized CI/CD pipeline project:

  1. Scalability: As the project grows and more developers use the platform, it is important to ensure that the decentralized infrastructure can scale to meet the increased demand. This may involve optimizing the use of resources, improving the distribution of build nodes, or leveraging techniques like sharding to ensure that the system can handle a high volume of builds.
  2. User experience: To ensure that developers find the platform easy to use and intuitive, it is important to focus on the user experience (UX) of the platform. This may involve conducting user testing and feedback sessions, designing clear and intuitive user interfaces, and providing helpful documentation and tutorials to guide users through the process of using the platform.
  3. Security and privacy: Since the platform involves sensitive development processes and code artifacts, it is crucial to prioritize security and privacy in the design of the platform. This may involve using strong encryption techniques, ensuring the integrity of the build process through digital signatures, and implementing access controls to prevent unauthorized access to code artifacts.
  4. Integration with other tools: To create a seamless and integrated development experience, it is important to design the platform to integrate well with other decentralized development tools, such as version control systems and code review tools. This may involve building integrations with existing tools or creating new tools that are specifically designed to work with the CI/CD pipeline platform.

By considering these factors and balancing the priorities of security, usability, scalability, and integration, we’ll be building a decentralized CI/CD pipeline platform that meets the needs of developers and helps advance the goals of the Radicle community.

If the decentralized CI/CD pipeline project is built using Radicle’s platform and infrastructure, it could be adopted by Radicle as an official project within its ecosystem. This would involve integrating the project with other Radicle projects and tools, and providing support and resources to help the project grow and thrive.

There are several ways that this project could be of benefit to the Radicle ecosystem:

  1. Improved developer productivity: By providing a decentralized CI/CD pipeline platform, developers would be able to more easily and efficiently build, test, and deploy their code. This would help to improve their productivity and enable them to quickly iterate on their projects more.
  2. Increased platform usage: As more developers begin using the decentralized CI/CD pipeline platform, it could help to drive increased usage of other Radicle tools and projects. This would help to grow the Radicle ecosystem and make it more vibrant and sustainable.
  3. Strengthened community: By fostering a culture of collaboration and contribution through events and forums, the decentralized CI/CD pipeline project could help to strengthen the community of users and contributors within the Radicle ecosystem. This would help to create a more supportive and engaging environment for decentralized development.
  4. Advancement of Radicle’s goals: The decentralized CI/CD pipeline project would help to advance Radicle’s overall mission of promoting decentralized development and creating a more open and collaborative ecosystem for software development. By enabling developers to more easily and securely collaborate and share code, the project would help to drive innovation and progress within the software development industry.

Overall, the adoption of the decentralized CI/CD pipeline project by Radicle could help to advance the goals of both the project team and the Radicle ecosystem as a whole. By working together, they could create a more vibrant and sustainable ecosystem for decentralized development that benefits developers, users, and the broader software development community.

FAQ

  1. How does the decentralized CI/CD pipeline project differ from traditional CI/CD pipelines?
  2. What are the advantages of using a decentralized approach for CI/CD?
  3. What kind of users or organizations would benefit from using this platform?
  4. How does the platform integrate with existing development tools and workflows?
  5. How does the platform ensure the security and integrity of build artifacts?
  6. What kind of decentralized storage networks are supported by the platform, and how are they integrated?
  7. How does the platform ensure reliable and consistent builds across different environments and platforms?
  8. What kind of consensus algorithm is used by the decentralized control plane, and how does it ensure correctness and fault tolerance?
  9. How do the platform handle scaling and high availability?
  10. What kind of support and maintenance will be available for users of the platform?

Answer

  1. The decentralized CI/CD pipeline project differs from traditional CI/CD pipelines in that it uses a decentralized network of resources to manage builds and artifacts. This allows for more resilience, scalability, and collaboration than traditional centralized approaches.
  2. The advantages of using a decentralized approach for CI/CD include increased resilience, security, and scalability. Because the platform is distributed across many nodes, there is no single point of failure, and builds can be run in parallel, reducing build times. Additionally, because the platform is built on open standards and decentralized protocols, it is more secure and resistant to censorship than centralized alternatives.
  3. Users or organizations that would benefit from using this platform include those working on decentralized applications or projects, those concerned with security and resilience in their build processes, and those looking for a more collaborative approach to build management.
  4. The platform integrates with existing development tools and workflows through APIs and integrations with popular tools like GitHub Actions or Jenkins. This makes it easy for users to incorporate the decentralized platform into their existing processes.
  5. The platform ensures the security and integrity of build artifacts through the use of cryptographic hashes and signatures, as well as the integration with decentralized storage networks like IPFS or Filecoin.
  6. The platform supports a variety of decentralized storage networks, including IPFS and Filecoin. These networks are integrated through standardized protocols and APIs, allowing for easy integration and retrieval of build artifacts.
  7. The platform ensures reliable and consistent builds across different environments and platforms through the use of containerization and virtualization technologies. Builds can be run in isolated environments that mimic production environments, ensuring consistent and reliable builds.
  8. The consensus algorithm used by the decentralized control plane could vary depending on the specific implementation, but generally, it would be a fault-tolerant, Byzantine fault-tolerant, or leader-based consensus algorithms like Raft or PBFT.
  9. The platform handles scaling and high availability through the use of distributed architectures and load-balancing techniques. This ensures that the platform can scale to handle large volumes of builds and traffic while maintaining high availability and performance.
  10. Support and maintenance for the platform could be provided through a variety of mechanisms, including user forums, documentation, and professional services. Additionally, because the platform is open-source and built on decentralized protocols, users can also contribute to the development and maintenance of the platform themselves.

Hello Administrators, my project is still awaiting review. @bordumb

Hello administrators, my project is still awaiting review. @bordumb @shelb_ee @abbey @yorgos. your approval is highly anticipated.

Hi @Dee

It sounds like the some combination of the Core Team (Radicle Protocol) and @yorgos 's team may be starting work on this.

In an effort to not duplicate work, I recommend the following:

  1. We don’t go ahead with this particular grant
  2. You get in touch with the Core Team/Yorgos to see how CI/CD is progressing and look into whether it makes sense to contribute to their work

Please see the Discord below and let me know if you need an intro/help navigating

Hello @Dee ,

First of all, please let me clarify that at this point I’m only a fellow Grantee - not an administrator. :wink:

With that said, I should also share that a decentralized CI system has also been on the cards for a long time and the topic has come up several times in the radicle discord server.

We will start working on one such system as part of the new Radicle Code Collab org in the months ahead, but first we have a much more important topic to address: integrations of the Radicle Code Collab solution with existing Continuous Integration / Continuous Delivery (CI/CD) systems - both hosted solutions (e.g. GitHub actions / CircleCI / etc. etc.) and open source solutions (e.g. jenkins / tekton / etc. etc.). This is much more important in allowing teams to start moving their projects to Radicle - keeping their existing CI and offering a smoother migration path to Radicle.

*Note: At this point, it has to be said that open source CI solutions already cover a big part of the “decentralized CI” value proposition, considering that any org can run their own existing open source CI server and not have to rely on any centralized solution, but we will address that issue *

@bordumb With this context in mind, I don’t think it’s worth funding a separate effort towards such a decentralized CI system. My general feedback for this particular proposal is that it primarily doesn’t sufficiently address the “why” and it is rather too vague and too big in scope. I would also propose that we don’t move ahead with this particular proposal - but…

At the same time, I think there is still some potential for Radicle Grants on the CI front: once we first the have couple of integrations to existing CI systems in place, we could perhaps open up a Request for Proposals (RFP) that invites team to submit Radicle Grant Proposals for further integrations to additional CI systems. We would only implement a couple inside the Code Collab team and this would be paving the way to external teams implementing integrations that we don’t have the capacity to cover internally.

Hope that helps!

@yorgos

Totally agree with everything you said here. I especially like the thinking around smooth migration to start → further decentralization later.

In summary it sounds like this:

  1. Foundation: Radicle Code Collab builds initial integrations. These become the blueprint for further integrations.
  2. Additional Integrations: Rely on RFPs and Grants to cover additional integrations. I imagine this could be a mix of proactive (i.e. we create RFPs) and retroactive funding (i.e. we send money whenever community members finish building a useful integration on their own)

Thanks for the input!

@Dee in light of what Yorgos has shared, I think it makes sense that we don’t continue with this grant in its current form. Let’s wait until initial integrations are built. Then if there is a very specific CI/CD integration your team would like to build, we can chat at that time.