Integrate Reviewer Recommender via the CI template for gitaly
Pre-requisites: Reviewer Recommender (aka UnReview) is an ML-based code reviewer recommendation system acquired by GitLab. By integrating a beta version of Reviewer Recommender into the Gitaly team, our Applied ML group would like to collect feedback to improve the quality of recommendations and help the team control the review workload.
Integration: This MR allows getting code reviewer recommendations when running the reviewers-recommender
job included via the CI template. The reviewers-recommender
job generates an artifact file with top-n
recommendations, which can be used as a suggestion for assigning code reviewers. After implementing reviewer-recommender-bot
v0.1.0, Reviewer-Recommender now puts recommendations into an MR note similar to Dangerbot. At each CI job run, Reviewer-Recommender searches for its previous comment within the first 10 notes. If the comment is found, then RR updates it. If not, then RR creates a new comment. Multiple RR comments have a very low probability of appearing in newly created MRs. In general, this logic is implemented to limit the number of heavy API GitLab calls and will be updated after unblocking https://gitlab.com/gitlab-org/modelops/applied-ml/review-recommender/gcp-postgres/-/issues/3.
Configuration:
- The
reviewers-recommender
job doesn't change the existing review workflow. - Reviewer Recommender periodically collects the merge request history and trains the model.
- The
RECOMMENDER_TOP_N
env variable sets thetop-n
option (default to 5). - The Reviewer Recommender server authenticates requests using the
CI_JOB_TOKEN
passed by thereviewers-recommender
job automatically. - Our Applied ML group will automatically track the generated artifacts files and existing assignments to improve the quality of recommendations.
- The
reviewers-Recommender
job is configured with theallow_failure: true
parameter to prevent breaking the project pipeline. Invite @GitLabReviewerRecommenderBot to the project with theReporter
permissions in order to get recommendations.
Known limitations:
- The Reviewer Recommender may provide outdated recommendations. We are already working on this issue in gitlab#340459.
- Our Applied ML group work to improve the way we present recommendations to MR authors.
You can find an example of how Reviewer Recommender works in this MR.
Related issue: https://gitlab.com/gitlab-org/gitlab/-/issues/342778, gitlab-org/modelops/applied-ml/review-recommender/recommender-bot-service#5