Add LRU-like behaviour to incremental compilation

What does this MR do?

Add LRU-like behaviour to incremental compilation

This adds least-recently-used-cache-like behaviour to the incremental webpack compiler. The DEV_SERVER_INCREMENTAL_TTL environment variable now determines the number of days that page bundles are considered "recent", and should be eagerly compiled. This number represents the trade-off between lazy/eager compilation versus low/high memory consumption of the webpack development server. A higher number means fewer pages needing to be compiled on demand, at the cost of higher memory consumption. A lower number means lower memory consumption, at the cost of more pages being compiled on demand. A value of 0 means that all pages in your history, regardless of how long ago you visited them, are eagerly compiled.

This also makes the compiler record a history of visited pages even when disabled, so that if and when it is enabled, that history can still be used to inform the LRU cache.

The history-recording function is explicitly disabled in the case that webpack is running in CI.

Finally, if there is no existing history file, it seeds the history file with a reasonable set of common entry points. This is to reduce friction for fresh installs, or for those who hadn't already explicitly enabled the incremental compiler.

gitlab-development-kit!2146 (merged) will make the incremental compiler enabled by default, and also make it possible to configure the TTL via gdk.yml.

Part of #300412 (closed).

Screenshots or Screencasts (strongly suggested)

n/a

How to setup and validate locally (strongly suggested)

The CI case (no history recording)

1. Add export CI=true in env.runit
2. Run gdk restart webpack
3. Observe using gdk tail webpack that IncrementalWebpackCompiler: Status – disabled is logged
4. When running the GDK, visit a page you've never visited before (you could temporarily rename tmp/cache/incremental-webpack-compiler-history.json to pretend your history is empty), and observe that it loads without an "compilation" overlay.

The history-recording case

1. Set webpack.incremental: false in your gdk.yml
2. Run make Procfile in your GDK directory
3. Observe using gdk tail webpack that IncrementalWebpackCompiler: Status – history-only is logged
4. When running the GDK, visit a page you've never visited before (you could temporarily rename tmp/cache/incremental-webpack-compiler-history.json to pretend your history is empty), and observe that it loads without a "compilation" overlay.
5. Observe that the tmp/cache/incremental-webpack-compiler-history.json file has been updated to include that page's entry point(s).

The LRU incremental compiler case

With no TTL set

1. Set webpack.incremental: true in your gdk.yml
2. Run make Procfile in your GDK directory
3. Observe using gdk tail webpack that IncrementalWebpackCompiler: Status – enabled, ttl=Infinity is logged
4. When running the GDK, visit a page you've never visited before (you could temporarily rename tmp/cache/incremental-webpack-compiler-history.json to pretend your history is empty), and observe that it loads with a "compilation" overlay.
5. Visit a page you've visited before, and observe that it loads without a "compilation" overlay.
6. Observe that the tmp/cache/incremental-webpack-compiler-history.json file has been updated to include that page's entry point(s).

With a TTL

1. Set webpack.incremental: true in your gdk.yml
2. Run make Procfile in your GDK directory
3. Add export DEV_SERVER_INCREMENTAL_TTL=7 in env.runit
4. Observe using gdk tail webpack that IncrementalWebpackCompiler: Status – enabled, ttl=7 is logged
5. When running the GDK, visit a page you've never visited before (you could temporarily rename tmp/cache/incremental-webpack-compiler-history.json to pretend your history is empty), and observe that it loads with a "compilation" overlay.
6. Visit a page you've visited within the last 7 days, and observe that it loads without a "compilation" overlay.
7. Visit a page you last visited more than 7 days ago, and observe that it loads with a "compilation" overlay.
8. Observe that the tmp/cache/incremental-webpack-compiler-history.json file has been updated to include that page's entry point(s).

Does this MR meet the acceptance criteria?

Conformity

• [-] I have included changelog trailers, or none are needed. (Does this MR need a changelog?)
• I have added/updated documentation, or it's not needed. (Is documentation required?)
• I have properly separated EE content from FOSS, or this MR is FOSS only. (Where should EE code go?)
• [-] I have added information for database reviewers in the MR description, or it's not needed. (Does this MR have database related changes?)
• I have self-reviewed this MR per code review guidelines.
• This MR does not harm performance, or I have asked a reviewer to help assess the performance impact. (Merge request performance guidelines)
• I have followed the style guides.
• This change is backwards compatible across updates, or this does not apply.

Availability and Testing

• [-] I have added/updated tests following the Testing Guide, or it's not needed. (Consider all test levels. See the Test Planning Process.)
• [-] I have tested this MR in all supported browsers, or it's not needed.
• [-] I have informed the Infrastructure department of a default or new setting change per definition of done, or it's not needed.