Praefect: Update voter state on failed node RPC
Currently when a secondary node RPC fails the error is ignored and the
transaction continues waiting for additional votes even if there are not
enough outstanding votes to reach quorum. If quorum becomes impossible,
due to the failed node, the transaction hangs until the context gets
canceled. This is not desirable as ideally once it has been established
that there is not enough outstanding votes to reach the required
threshold specified by the transaction, the transaction should be
canceled. This change adapts the ErrHandler
function of the secondary
nodes to cancel the voter in the transaction associated with the failed
RPC. In this process the voter's result state is updated to
VoteCanceled
and the subtransaction is checked to see if quorum can
still be achieved. If quorum is impossible the vote is failed and the
voters are unblocked. If quorum is still possible the voters remain
blocked waiting for further votes to decide the outcome.
Closes: #4088 (closed)