raft: Implement networking handlers for Raft

etc/raft is a minimalist, composable library for Raft. The core of the library is concerned with the Raft state only. It leaves the Transport component to the application. This issue aims to implement gRPC handlers for Raft. The handlers act as thin forwarders. They should not handle messages directly. Instead, they'll need to forward messages to storage package so that the package can act accordingly.

Unlike dragonboat, etcd/raft doesn't have a built-in multi-raft feature. A raft member is identified as a unique ID within the group. In the plan, we would like to structure one partition = one Raft group. Thus, the storage name and partition ID could be used as a unique identity to forward messages to the right partition.

A viable approach is to create a wrapper message:

message RaftMessageV1 {
  uint64 ID = 1;
  string cluster_id = 2;
  string authority_name = 3;
  uint64 parittion_id = 4;
  LogEntry log_entry = 5;

  oneof log_data {
    ReferencedLogData referenced = 6;
    PackedLogData packed = 7;
  }
}

In detail, this issue consists of:

• Implement gRPC handlers for Raft. This handler redirects requests of a partition to the corresponding Raft manager. It needs to follow a fast track and bypasses unnecessary middleware such as WAL.
• Handle cross-node authentication. Unlike Rails, Gitaly servers have an internal authentication for cross-node communication.
• Implement a Transport interface in raftmgr package. This transport packs the log data at the sender side and unpacks data at the receiver side. The raft manager receives a reference to the data.

In the long term, gRPC handlers could be extended for the sake of performance optimization.

• Batch health-check only requests by destination. We are not too sure if it's still necessary after enabling quiesced. The idea is that instead of letting Raft groups spam heath-check requests to the same destination, the sender implements a debounce layer that batches such requests. On the destination side, the handler broadcasts health-check requests to corresponding Raft groups. raft: Use raw node for better multi-raft support (#6468)
• Streaming heavy data via Sidechannel. Data of log entries are embedded into raftpb.Message. Unlike traditional Raft systems, the data transfer between Gitaly nodes (say Packfiles) is potentially huge. Gitaly has a bad history of sending large data via gRPC. Thus, we could enable Sidechannel for such transmission. The log entry is transferred as a whole where Raft + Log Entry comes first and Packfiles come later. An optimization to detach them is being explored here: raft: Detach log entry committing and packfile ... (#6466)

References:

• Raft Message proto: https://github.com/etcd-io/raft/blob/main/raftpb/raft.proto
• HTTP implementation reference: https://pkg.go.dev/go.etcd.io/etcd/server/v3/etcdserver/api/rafthttp