feat: AI Assisted Code Suggestions: Telemetry: Only count suggestions actually shown to users
Context
In the telemetry the VS Code sends to the model gateway, we send the following counts:
- Requests
- Accepted
- Errors
The number of requests is including the suggestions that never get shown to the user because they moved the cursor before the response was received.
Goal
Improve the precision of the counts we send for telemetry by allowing us to quantify Acceptances vs Shown suggestions.
This results in a better awareness of the quality and not include the suggestions never shown in the assessment of quality.
Expected scenario
This is what I would expect:
- If the user sees a suggestion and decides not accept it, it is considered a rejection.
- If the user does not see a suggestion because the user moves the cursor before the suggestion is displayed to the user, the suggestion is not displayed. In this case, it should not be considered a rejection. (It is something else...perhaps a timeout...which we don't track separately but perhaps should in the future)
- If the the request times out even if the user doesn't move the cursor, same as #2 (closed) above. It is a timeout and should not be considered a user rejectoin. in slack
Further context
While the telemetry is done in the VS Code, in a near future it will be moved to our [Beta] Language Server for Code Suggestions (&10746 - closed)
Edited by André Luís