test(integration): reduce test flakiness by replacing globalState

This blows my mind. After hours of investigation, I found out that our tests are failing because sometimes the extension `globalState doesn't contain the access token that should have been stored there. I think that the state gets either overridden during some internal VS Code store update, there is a bug in the store implementation, or we somehow don't use the store correctly.

The issue happens because the token is missing even after we store it:

1. We set a dummy API access token before running the integration test:

   await tokenService.setToken(GITLAB_URL, 'abcd-secret');

2. Under the hood, the token service calls:

   const tokenMap = this.context.globalState.get<Record<string, string>>('glTokens', {});
   tokenMap[instanceUrl] = token;
   await this.context.globalState.update('glTokens', tokenMap);

   • This updates the VS Code globalState for our extension.
3. Then our API services gets the token by calling:

   tokenService.getToken(instanceUrl);

4. Which again under the hood calls:

   const tokenMap = this.context.globalState.get<Record<string, string>>('glTokens', {});
   return tokenMap[instanceUrl];

The last block of code returns correct token most of the time but sometimes it returns undefined. This shouldn't happen, I don't have a logical explanation because the state was successfully updated before we read it.

This can be reproduced when running the extension tests. It is not easy to reproduce in the debugger and I wasn't able to reproduce this behaviour in isolation. I tried to create an extension where I'd reproduce this behaviour in much simpler code. If that worked I'd create a bug in the VS Code project. But after several hours I'm giving up. I'm not sure where this behaviour comes from. AFAIK it doesn't happen in the extension when used normally (not in tests).

Solution

Since I'm unable to find the root cause, the next-best-thing is to use an in-memory stub of the globalState for storing tokens. For the purpose of tests, it doesn't have any drawbacks. The only drawback is that I'm now really curious about what is the root cause. I just can't afford to burn any more time on it.

Unfortunately, this hasn't been the only cause of flakiness and so the integration tests still occasionally fail. But when running the tests locally, the success rate improved significantly (instead of failing half the time now the tests fail maybe 1/10 cases).

Related to #299 (closed)