We could mitigate this by requiring the acked CID to be an ancestor of a known CID.
What is a "known" CID? If it is a CID that has already been processed
(by the usual IPNS lookup) then this doesn't help at all?
The downside of this approach is that anybody could forge an ack and force an update of the machine’s input
log.
This seems too severe for me. Shall we just make signing acks with the
same key as the IPNS machine name a priority?
···
On Thu, Mar 7, 2019 at 1:59 PM Thomas Scholtes <thomas@monadic.xyz> wrote:
Can’t we get rid of pending inputs? If a daemon receives an ack for a submitted input it takes the CID in the
ack and updates the in-memory state of the machine to the given CID.
This approach has the advantage that apps do not need to know about pending inputs and we update the
state more quickly.
The downside of this approach is that anybody could forge an ack and force an update of the machine’s input
log. (We could mitigate this by requiring the acked CID to be an ancestor of a known CID.)
On Thu, Mar 7, 2019 at 1:26 PM James Haydon <james@monadic.xyz> wrote:
We have received some feedback on read-your-writes consistency when
using Radicle, e.g.:
> delay between locally created issues and seeing it when it lists is jarring
Here is a proposed solution:
- When an owner daemon `ACK`s an input via pubsub, it includes the CID of the
input as written to the IPFS linked list.
- Follower daemons keeps track of inputs that have been sent by the user that
have been `ACK`'d via pubsub. These are "pending inputs" to the machine. When
the daemon encounters the relevant CID when updating the cached machine, it
marks the pending input as no longer pending.
- The `query` endpoint returns the list of currently pending inputs
along with the result of the query, e.g.:
{
"result": [
"{:number 42 :author ... :title ...}",
...
],
"pending_inputs": [
"{:author ... :title \"something very wrong\"}"
]
}
That way apps can inform the user that the query may be somewhat out of date
with respect to their own inputs. E.g.:
$ rad issue list
state # title author updated
open 2 title 8989 James Henri Haydon 2019-03-05T10:43:30Z
open 1 very bad James Henri Haydon 2019-03-05T10:41:22Z
open 0 foo is bar James Henri Haydon 2019-03-05T10:10:47Z
Pending:
open - something very wrong James Henri Haydon 2019-03-05T10:45:32Z
James
Digging into how to extract private keys, I also came across a comment
that signed pubsub messages might also be on the roadmap.
···
On Thu, Mar 7, 2019 at 3:08 PM Julian Arni <julian@monadic.xyz> wrote:
I'd also suggest doing more investigation of IPNS over pubsub before implementing this. Merle and I did some yesterday, but we did not try out the version that was just released (0.4.19) or really delve into why it didn't seem to be working. In theory the updates would then be fast enough so that this wouldn't happen.
On Thu, Mar 7, 2019 at 3:25 PM James Haydon <james@monadic.xyz> wrote:
> My recommendation would be to implement signed acks.
Agreed!
On Thu, Mar 7, 2019 at 2:20 PM Thomas Scholtes <thomas@monadic.xyz> wrote:
>
> Am Donnerstag, den 07.03.2019, 14:05 +0000 schrieb James Haydon:
> > > We could mitigate this by requiring the acked CID to be an ancestor of a known CID.
> >
> > What is a "known" CID? If it is a CID that has already been processed
> > (by the usual IPNS lookup) then this doesn't help at all?
>
> I do mean a CID that has already been processed, i.e. a CID that points to an entry in the
> input log that is known to the daemon. And I think it does help—my use of “ancestor” was
> probably confusing. I meant that the input log that unfolds from the head specified by the
> CID in the ack is an extension of the known input log.
>
> > > The downside of this approach is that anybody could forge an ack and force an update of
> > > the machine’s input
> > log.
> >
> > This seems too severe for me. Shall we just make signing acks with the
> > same key as the IPNS machine name a priority?
> This problem is indeed severe. My thinking regarding what to do about this problem in
> general goes as following:
>
> The best solution is probably signed acks that carry the CID pointing to the latest input
> log head. This would solve the problem where users do not see the submissions although the
> command line accepted them. The hardest part about this is implementing signing.
>
> Exposing the concept of pending inputs is not as good as signed acks because it complicates
> the interaction with the user and between the daemon and the CLI and it does not fix the
> underlying problem. It only makes sense to choose this solution if it takes significantly
> less time to implement pending inputs then it takes to implement signed acks.
>
> I personally don’t think that implementing pending inputs it will require less time, just
> because of the fact that it involves changes across the whole stack.
>
> My recommendation would be to implement signed acks.
>
> - Thomas
>
>
I’d also suggest doing more investigation of IPNS over pubsub before implementing this. Merle and I did some yesterday, but we did not try out the version that was just released (0.4.19) or really delve into why it didn’t seem to be working. In theory the updates would then be fast enough so that this wouldn’t happen.
···
On Thu, Mar 7, 2019 at 3:25 PM James Haydon james@monadic.xyz wrote:
My recommendation would be to implement signed acks.
Agreed!
On Thu, Mar 7, 2019 at 2:20 PM Thomas Scholtes thomas@monadic.xyz wrote:
Am Donnerstag, den 07.03.2019, 14:05 +0000 schrieb James Haydon:
We could mitigate this by requiring the acked CID to be an ancestor of a known CID.
What is a “known” CID? If it is a CID that has already been processed
(by the usual IPNS lookup) then this doesn’t help at all?
I do mean a CID that has already been processed, i.e. a CID that points to an entry in the
input log that is known to the daemon. And I think it does help—my use of “ancestor” was
probably confusing. I meant that the input log that unfolds from the head specified by the
CID in the ack is an extension of the known input log.
The downside of this approach is that anybody could forge an ack and force an update of
the machine’s input
log.
This seems too severe for me. Shall we just make signing acks with the
same key as the IPNS machine name a priority?
This problem is indeed severe. My thinking regarding what to do about this problem in
general goes as following:
The best solution is probably signed acks that carry the CID pointing to the latest input
log head. This would solve the problem where users do not see the submissions although the
command line accepted them. The hardest part about this is implementing signing.
Exposing the concept of pending inputs is not as good as signed acks because it complicates
the interaction with the user and between the daemon and the CLI and it does not fix the
underlying problem. It only makes sense to choose this solution if it takes significantly
less time to implement pending inputs then it takes to implement signed acks.
I personally don’t think that implementing pending inputs it will require less time, just
because of the fact that it involves changes across the whole stack.
My recommendation would be to implement signed acks.
> We could mitigate this by requiring the acked CID to be an ancestor of a known CID.
What is a "known" CID? If it is a CID that has already been processed
(by the usual IPNS lookup) then this doesn't help at all?
I do mean a CID that has already been processed, i.e. a CID that points to an entry in the
input log that is known to the daemon. And I think it does help—my use of “ancestor” was
probably confusing. I meant that the input log that unfolds from the head specified by the
CID in the ack is an extension of the known input log.
> The downside of this approach is that anybody could forge an ack and force an update of
> the machine’s input
log.
This seems too severe for me. Shall we just make signing acks with the
same key as the IPNS machine name a priority?
This problem is indeed severe. My thinking regarding what to do about this problem in
general goes as following:
The best solution is probably signed acks that carry the CID pointing to the latest input
log head. This would solve the problem where users do not see the submissions although the
command line accepted them. The hardest part about this is implementing signing.
Exposing the concept of pending inputs is not as good as signed acks because it complicates
the interaction with the user and between the daemon and the CLI and it does not fix the
underlying problem. It only makes sense to choose this solution if it takes significantly
less time to implement pending inputs then it takes to implement signed acks.
I personally don’t think that implementing pending inputs it will require less time, just
because of the fact that it involves changes across the whole stack.
My recommendation would be to implement signed acks.
- Thomas
···
Am Donnerstag, den 07.03.2019, 14:05 +0000 schrieb James Haydon:
Can’t we get rid of pending inputs? If a daemon receives an ack for a submitted input it takes the CID in the
ack and updates the in-memory state of the machine to the given CID.
This approach has the advantage that apps do not need to know about pending inputs and we update the
state more quickly.
The downside of this approach is that anybody could forge an ack and force an update of the machine’s input
log. (We could mitigate this by requiring the acked CID to be an ancestor of a known CID.)
···
On Thu, Mar 7, 2019 at 1:26 PM James Haydon james@monadic.xyz wrote:
We have received some feedback on read-your-writes consistency when
using Radicle, e.g.:
delay between locally created issues and seeing it when it lists is jarring
Here is a proposed solution:
-
When an owner daemon ACKs an input via pubsub, it includes the CID of the
input as written to the IPFS linked list.
-
Follower daemons keeps track of inputs that have been sent by the user that
have been ACK’d via pubsub. These are “pending inputs” to the machine. When
the daemon encounters the relevant CID when updating the cached machine, it
marks the pending input as no longer pending.
-
The query endpoint returns the list of currently pending inputs
along with the result of the query, e.g.:
{
"result": [
"{:number 42 :author ... :title ...}",
...
],
"pending_inputs": [
"{:author ... :title \"something very wrong\"}"
]
}
That way apps can inform the user that the query may be somewhat out of date
with respect to their own inputs. E.g.:
$ rad issue list
state # title author updated
open 2 title 8989 James Henri Haydon 2019-03-05T10:43:30Z
open 1 very bad James Henri Haydon 2019-03-05T10:41:22Z
open 0 foo is bar James Henri Haydon 2019-03-05T10:10:47Z
Pending:
open - something very wrong James Henri Haydon 2019-03-05T10:45:32Z
James
We have received some feedback on read-your-writes consistency when
using Radicle, e.g.:
delay between locally created issues and seeing it when it lists is jarring
Here is a proposed solution:
- When an owner daemon `ACK`s an input via pubsub, it includes the CID of the
input as written to the IPFS linked list.
- Follower daemons keeps track of inputs that have been sent by the user that
have been `ACK`'d via pubsub. These are "pending inputs" to the machine. When
the daemon encounters the relevant CID when updating the cached machine, it
marks the pending input as no longer pending.
- The `query` endpoint returns the list of currently pending inputs
along with the result of the query, e.g.:
{
"result": [
"{:number 42 :author ... :title ...}",
...
],
"pending_inputs": [
"{:author ... :title \"something very wrong\"}"
]
}
That way apps can inform the user that the query may be somewhat out of date
with respect to their own inputs. E.g.:
$ rad issue list
state # title author updated
open 2 title 8989 James Henri Haydon 2019-03-05T10:43:30Z
open 1 very bad James Henri Haydon 2019-03-05T10:41:22Z
open 0 foo is bar James Henri Haydon 2019-03-05T10:10:47Z
Pending:
open - something very wrong James Henri Haydon 2019-03-05T10:45:32Z
James