Opening Upstream via links on the web

📜 Archive 🌱 Development
,
1

Hey folks :wave:,

I was looking for the next bigger thing to tackle and both @Julien & @abbey suggested I should look into deep linking, because it was suggested by the community on multiple occasions (1, 2, 3).

Before we shipped our first public beta we also had a discussion around linking to entities within Upstream, but due to time constraints we decided to postpone figuring out what the structure for deep linking should be. Now would be a great time to revisit it again.

What I’d like to get out of the discussion here is a better definition of what the user experience should be, I think this would help to properly scope and break down this feature for implementation.

So, I talked to @abbey regarding her feature request, and if I understood correctly, then her goal is to reduce the steps that are necessary to look at a project’s source code while browsing the web and not having Upstream open, or in other words the higher-level goal is to improve project discoverability.

To achieve this goal, we essentially want a shareable URL which is rendered as a clickable link by services like Twitter/GitHub/forums/etc. A caveat, which at first I didn’t realise is, that these services only render URLs with http:// and https:// protocols as clickable links. In other words, custom protocols like rad:, rad:// or upstream:// would be rendered as plain-text by these services and would not be clickable, thus a user would have to still select, copy and paste them somewhere for this feature to work, which is not ideal.

This is why other apps (like Telegram, for example) link to a website which calls their custom protocol to open the app on the user’s computer. There are also other benefits of having this intermediate step:

  • we can provide instructions to download and install the app if a user hasn’t installed it yet
  • generate link previews for Twitter and other services
  • provide a URL shortening service
  • we have an additional abstraction between the links shared on the web and an application that handles them, giving us the flexibility to redirect to something else in the future

To illustrate things further, here are some screenshots:

The other end of this feature is about how to handle these external requests inside of Upstream.

Let’s assume that for the initial version we only want to handle sharing projects via their Radicle IDs. In this case, once a user clicks on a project’s link on the web this link leads to the redirect webpage, which in turn opens Upstream and forwards it the Radicle ID. Now, if the user already has the project locally, we can simply display that. But if the project hasn’t been followed and replicated yet, we could show the search modal with the Radicle ID pre-filled in the search field.

macOS proof-of-concept

Linux proof-of-concept

@xla already proposed a scheme which could work for opening Upstream via a custom protocol.

Next steps:

  • discuss in this thread whether the feature as I laid it out here makes sense
  • once we agree on the feature, break it down into smaller steps, eg.:
    • define what the URL scheme of the redirect webpage should look like and what domain to use
      • something to consider here is that .xyz domains often get blocked
    • design the UI of the redirect website
    • figure out what stack to use and how to deploy the redirect website
    • formalise @xla’s proposed custom Upstream protocol scheme
      • should we use rad: (URN form), rad:// (URL form) or upstream:// as the custom protocol, i.e. should the custom protocol be usable by many different apps or should it be Upstream specific?
    • make a proof-of-concept implementation to validate that the custom protocol works on Linux and macOS: 1652
11 Likes
2

Great write-up and work so far, @rudolfs :heart:

One thought on the custom protocol:

IMO, using upstream is sub-optimal since it strongly ties the URIs that identify protocol identities to a specific client of said protocol. Ideally, shall there be more clients in the future, I would like the URIs to be supported by those clients and have them work interchangeably between clients.

The same way that we use http/https and not safari/chrome/etc.

Now, I can come up with a deep link that would be upstream-specific, like a deep link that takes a user to a specific section of the settings. However, I don’t think that such use case - which we don’t have - should impact the choice here. We could always introduce upstream-specific URIs, shall there be a need for that in the future.

So in sum, since the URIs will be pointing to entities of the protocol, I’d rather have the custom protocol be the protocol/product name and not the name of the (current) client.

Edit: fixed some typos

3

It would be nice to build a radicle badge generator via https://shields.io using this feature for helping people linking existed GitHub repos to their radicle addresses.

4

It is interesting that nobody seems to object to the privacy and security implications of such a feature.

Keep in mind that a. rad:// is already a semi-public scheme as it is used to dispatch to the git remote helper, and b. if any compliant (heh) radicle application is supposed to be able handle such deep links, it would effectively become a public API, which may preclude some upstream-specific uses.

1 Like
5

Thanks for raising this @kim, it’s a good point, I hadn’t thought about that.

Do I understand correctly that the problem is having part of the user interaction going through a central service, which, if compromised, could leak information about who clicks on which links? Another attack vector I can think of is if a potential attacker gets hold of that service, then they could redirect users to some malicious service.

Am I missing any other privacy or security implications?

My personal take on this is that while it’s unfortunate that services like Twitter don’t properly render custom protocols forcing users to use workarounds thereby exposing themselves to all kinds of tracking, we could still offer it as an opt-in feature.

I’d argue that if I’m already using Twitter & co, then I am aware of such threats and might as well use the convenience provided by a redirect service and security/privacy conscious users can share plain-text URNs like they did before.

And on other websites/forums which do render custom protocols as links, users could use our custom protocol which would give them both convenience and privacy/security.

I’m curious to hear what others think? \cc @cloudhead @abbey

1 Like
6

I think there are at least three potential issues:

  1. Custom protocol handlers are by design a loophole to escape the browser sandbox.

    There are many ways to exploit them, ranging from buffer overruns in parameter handling, over remote code execution when the parameters are not properly escaped, to simply DoS because the receiving application is overwhelmed with requests. Yes, maybe one can implement this correctly, and yes, I can also choose to not install the handler (which I would). It’s just that the flows for installing one are typically such that users aren’t made aware that they’re making a very fundamental trust decision.

  2. A redirecting web service is an excellent and cheap opportunity to sneak in some tracking.

    Sure, we’re the good people, so we would even turn off request logging (we would, right? right?), so it’s perhaps a little more obvious to folks that they’re making a decision to trust the domain of said service. Since we’re all about OSS and alternative implementations, how do you teach people that they should really not trust any other domain?

  3. Any webpage can send a custom-protocol redirect (composes with 1.).

    Ya, well, just make the browser ask me every time. But oh well, convenience…

2 Likes
7

I think this redirect system is a little out of scope of the original feature, and as @kim points out, introduces a trust problem (sneakily). Not only do I have to trust twitter now, I also have to trust the redirect service to direct me to the right place.

To me, it would make more sense to bundle this “feature” with a read-only web-view of radicle, which is something for the future.

3 Likes
8

We had a discussion within our team, to recap:

  • we settled for using radicle:// as the custom protocol, because rad:// is already used by radicle-link and upstream:// would be too app specific
  • we won’t be doing a redirect website due to security/privacy concerns
  • TBD: we need to write a formal specification for the custom protocol URL scheme, so that other clients can implement it as well
5 Likes
9

While I’m here reading I’ll link to the formal spec proposal :slight_smile: Add Radicle client protocol spec by rudolfs · Pull Request #10 · radicle-dev/radicle-decisions · GitHub

2 Likes
10

And now it’s official.

4 Likes
11

I would love to be able to have edit buttons on a static site (of documentation, for instance) which point to objects in upstream and automatically bring up an editor through upstream (maybe according to a preference setting of whether to use a native editor or one of choice). This is already common practice in documentation deployed from github and the like, even sometimes defaulting to online editors like prose.io.

One thing I wonder though is whether it might be possible or desirable to bundle a repo into a static site (such as, but not limited to the source of that site) and have a client spin up right from the site and right in the browser (similar to WebTorrent–likely making use of WebRTC?) whenever any part of the repo is accessed to help ensure that those links always point to something. Alternatively, if a bundle were not made available (or if the bundle were behind a specified origin, perhaps), some links could function like magnet links, whether handled in the browser or externally by upstream.

This idea of bundling and an embedded client is probably well outside of the scope of this proposal, but I suspect it could offer a relatively decentralized approach to the issue of linking repos externally by allowing users to self-host the repo and its contents under their own domain with link previews, etc…

12

Not sure if a bug but links don’t open anything on Linux

Upstream just does nothing… it doesn’t open any repos (v 0.2.8)

13

@v1rtl I assume you integrated Upstream on linux as outlined in: https://docs.radicle.xyz/docs/understanding-radicle/faq#is-it-possible-to-launch-upstream-via-links-on-the-web?

Does Upstream itself launch when you click on a link?

14

@rudolfs, yes Upstream launches but it doesn’t open a repository

I even tried with xdg-open

it just opens the app but not the repo

15

I tried it on mac, seems to be working on that platform still. Could you please open an issue with repro steps on Issues · radicle-dev/radicle-upstream · GitHub?

16

@rudolfs, done issue is here: radicle:// links don't open the repo on Linux · Issue #2166 · radicle-dev/radicle-upstream · GitHub

1 Like