Endpoint for symbol file download

What does this MR do and why?

In Add service to index nuget symbol files (!131567 - merged), we started indexing the symbol PDB files in the NuGet Repository. To simplify, indexing can be understood as making those files available for consumption or download.

In this MR, we introduce the needed endpoint to make downloading/consuming those PDB files possible.

Technical details:

The endpoint doesn't follow the usual authentication approaches because debuggers don't support attaching a PAT or any other token with the request. Alternatively, we authenticate using the params we receive in the request. Those params are:

• the file signature
• the file name
• the file sha256 digest

However, we put the endpoint behind a feature flag and it's disabled by default. Because the endpoint doesn't follow how we authenticate the other private NuGet Repository endpoints, we will give the choice to the users to enable/disable this endpoint. Using a project/namespace setting (in a subsequent MR), users can have the liberty to consume their symbol files knowing that the endpoint doesn't follow the normal authentication.

From where are those params coming? And how reliable are they?

Let's start from the beginning. A developer has a work-in-progress .net project on his local machine. during the development process, the developer decides that the project needs to include a NuGet package to satisfy a specific functionality. So the following steps can be followed:

• A local .net project is opened in Visual Studio or Visual Studio Code.
• Installing the needed NuGet package in the project. The package should reside in GitLab NuGet Repository. This step will include a bunch of the package files into the project. What we are most interested in here is the executable file. A file that has the .dll extension and holds the information the debugger needs to be able to download the symbol debugging PDB files.
• When the developer wants to debug the code in the project that uses the installed NuGet package, the debugger (Visual Studio or Visual Studio Code in this case) would look into the executable file .dll for the installed package, and from this executable, it extracts three pieces of information:
  • the file signature
  • the file name
  • the file sha256 digest
• If the endpoint of the symbol server was added to the debugger's settings, a request holding the three extracted pieces of information (filename, signature & sha256) is fired to that endpoint, asking for the debugging .pdb file.
• The endpoint in the symbol server (the endpoint we add in this MR) will receive this request. The server should search for the requested .pdb file using the received information (filename, signature & sha256).
• If the file is found, it should be returned. Otherwise, 404 is returned to the debugger.
• The information we rely on to find the right symbol file (filename, signature & sha256) is not retrievable unless the debugger has access to the executable .dll, and the access to the executable means the debugger was correctly authenticated to install the NuGet package from GitLab's NuGet Repository.

The endpoint we introduce has the following URL:

https://gitlab.example.com/api/v4/projects/<project_id>/packages/nuget/symbolfiles
-- or --
https://gitlab.example.com/api/v4/groups/<group_id>/-/packages/nuget/symbolfiles

This URL should be added to the debugger's settings as a symbol server URL. The debugger uses this URL and appends the needed params then fires the following request:

https://gitlab.example.com/api/v4/projects/<project_id>/packages/nuget/symbolfiles/<file_name>/<signature>/<file_name>
-- or --
https://gitlab.example.com/api/v4/groups/<group_id>/-/packages/nuget/symbolfiles/<file_name>/<signature>/<file_name>

and the sha256 digest is included in the request headers:

"Symbolchecksum"=>"SHA256:XXXXXXXXXXXXXXXXXXXXXXXXXXX, SHA256:XXXXXXXXXXXXXXXXXXXXXXXXXXX",

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

Before	After

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

1. We need to test with a NuGet package that has symbol files. We can test with this public package.
2. Download the .nupkg & .snupkg files.
3. Publish the package to your GDK following these steps. Ignore the feature flag step since it's enabled by default now.
4. Clone this .net testing project. It has the package Polly we pushed to GDK as a dependency.
5. Open the cloned project in VSCode, and make sure the C# & C# Dev Kit extensions are installed.
6. Open VSCode settings and search for csharp.debug.symbolOptions.searchPaths. Click on Add Item and enter the URL of the symbol server endpoint: http://gdk.test:3000/api/v4/projects/<project_id>/packages/nuget/symbolfiles
7. Search for the csharp.debug.justMyCode setting and uncheck it.
8. Open the Program.cs file then add a breakpoint on line 3 and press F5 to start a debugging session.
9. The debugger (VSCode) will send a request to the GDK symbol server endpoint to download the symbol file for the Polly package.
10. In the Debug Console, you should see that the symbol was successfully loaded:

Loaded '/Users/moazkhalifa/Downloads/dotnet-testing-project/bin/Debug/net7.0/Polly.dll'. Symbols loaded.

11. The PDB file is downloaded in Users/moazkhalifa/.dotnet/symbolcache path. If you want to re-test downloading the symbol, you can go to this path and then delete the polly.pdb folder, and rerun the debugging session in VSCode.
12. If you switched to the master branch, the symbol file won't be downloaded.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

• I have evaluated the MR acceptance checklist for this MR.

Related to #416178 (closed)