Skip to content

[MR Widget Eng] - Add ability to poll nested expanded content

Context

This issue builds on #361286 (closed)

Using two different endpoints for collapsed/uncollapsed state

This is done for performance optimizations. The idea being that we need much less data to provide a collapsed summary. For some widgets the collapsed/uncollapsed state use the endpoint.

For others, that are optimized like License Compliance, we have 2 endpoints. All of the ~"group::secure" widgets suffer from the 204 - No Content problem as described in #361286 (closed)

Problem

The Widget extension has polling only for the collapsed endpoint. The assumption being that if the data is available in the collapsed state, then we should have data ready for the expanded state.

The problem with ~"group::secure" widgets is that we parse report artifacts after a successful pipeline run. The parsed results are cached. The caching layer works at the endpoint layer. By having two different endpoints, the collapsed endpoint will eventually show a 200 - OK with data populated. This assumes we solve #361286 (closed). That result is cached.

When we expand the widget, we hit the fullData endpoint. Because it's a different endpoint, the results are a cache miss, and the reports are then re-generated. We then end up with the 204 - No Content #361286 (closed) problem all over again.

Problem Workflow

Initial page load -> initial widget load -> 204 - No Content Response - Continue to Poll until 200 OK -> then show widget summary -> Expand widget -> Attempt to fetch fullData endpoint -> Another 204 - No Content -> Another exception caught because we have no data -> Widget errors out

Screen_Shot_2022-05-03_at_8.09.09_PM

Screen_Shot_2022-05-03_at_8.08.45_PM

Solution

Allow for something like enablePollingFullData so we can implement the same polling logic as the enablePolling flag for the collapsed endpoints. Handle 204/200 or any other successful status code

Gotchas

  • We need to solve how we want to handle polling in for 204 - No Content #361286 (closed). It blocks this issue

Implementation Plan

Based off the work in !87107 (merged)

In: app/assets/javascripts/vue_merge_request_widget/components/extensions/base.vue

  1. Leverage the existing enablePolling: true setting.
  2. Update
    loadAllData() {
      if (this.hasFullData) return;

      this.loadingState = LOADING_STATES.expandedLoading;

      this.fetchFullData(this.$props)
        .then((data) => {
          this.loadingState = null;
          this.fullData = data.map((x, i) => ({ id: i, ...x }));
        })
        .catch((e) => {
          this.loadingState = LOADING_STATES.expandedError;

          Sentry.captureException(e);
        });
    },

to something like

    loadAllData() {
      if (this.hasFullData) return;

      this.loadingState = LOADING_STATES.expandedLoading;

      if (this.$options.enablePolling) {
        if (this.fetchFullMultiData) {
          this.initExtensionFullDataMultiPolling();
        } else {
          this.initExtensionFullDataPolling();
        }
      } else {
        this.fetchFullData(this.$props)
          .then((data) => {
             this.loadingState = null;
             this.fullData = data.map((x, i) => ({ id: i, ...x }));
          })
          .catch((e) => {
             this.loadingState = LOADING_STATES.expandedError;
             Sentry.captureException(e);
          });
      }
    },
  1. Implement initExtensionFullDataMultiPolling and initExtensionFullDataPolling as something like:
   initExtensionFullDataMultiPolling() {
      const allData = [];
      const requests = this.fetchMultiData();

      requests.forEach((request) => {
        const poll = new Poll({
          resource: {
            fetchData: () => request(this.$props),
          },
          method: 'fetchData',
          successCallback: (response) => {
            const headers = normalizeHeaders(response.headers);

            if (typeof headers['POLL-INTERVAL'] === 'undefined') {
              poll.stop();
              allData.push(response.data);
            }

            if (allData.length === requests.length) {
              this.setFullData(allData);
            }
          },
          errorCallback: (e) => {
            poll.stop();
            this.loadingState = LOADING_STATES.expandedError;
          },
        });

        poll.makeRequest();
      });
    },
    initExtensionFulLDataPolling() {
      const poll = new Poll({
        resource: {
          fetchData: () => this.fetchFullData(this.$props),
        },
        method: 'fetchData',
        successCallback: ({ data }) => {
          if (Object.keys(data).length > 0) {
            poll.stop();
            this.setFullData(data);
          }
        },
        errorCallback: (e) => {
          poll.stop();

          this.loadingState = LOADING_STATES.expandedError;
        },
      });

      poll.makeRequest();
    },
  1. Implement setFullData as something like:
    setFullData(data) {
       this.loadingState = null;
       this.fullData = data.map((x, i) => ({ id: i, ...x }));
    },

  1. Attempt to refactor and generalize the initExtensionFullDataMultiPolling, initExtensionFullDataMultiPolling, initExtensionFulLDataPolling, initExtensionPolling functions to reduce code duplication

  2. Update existing components with enablePolling: true behave correctly with the changes in the implementation plan by passing the full response object instead of the data only. Update unit tests as well

app/assets/javascripts/vue_merge_request_widget/extensions/accessibility/index.js

app/assets/javascripts/vue_merge_request_widget/extensions/test_report/index.js

ee/app/assets/javascripts/vue_merge_request_widget/extensions/metrics/index.js

Edited by -