Re-orchestrate updating NuGet package flow from downloaded package file
What does this MR do and why?
In Add service to index nuget symbol files (!131567 - merged), a new service was added to create symbol files .pdb
records in the database. Those .pdb
files are embedded in the uploaded .snupkg
files. Thus, we need to download the .snupkg
file from the Object Storage first and then extract the .pdb
files from it.
On the other hand, the downloaded .snupkg
file is used to extract the metadata file nuspec
from it. That means we should download the .snupkg
only once and then use it to do two things:
- extract the metadata file
nuspec
- extract the symbol files
.pdb
In order to do so, the ProcessPackageFileService was introduced in !131567 (merged). It was responsible for downloading the package file and then passing it to the metadata extraction service & the symbol creation service.
So this was the flow of the services:
flowchart BT
Packages::Nuget::ExtractionWorker --> Packages::Nuget::UpdatePackageFromMetadataService
Packages::Nuget::UpdatePackageFromMetadataService --> Packages::Nuget::MetadataExtractionService
Packages::Nuget::MetadataExtractionService --> Packages::Nuget::ProcessPackageFileService
Packages::Nuget::ProcessPackageFileService --> Packages::Nuget::ExtractMetadataFileService
Packages::Nuget::ProcessPackageFileService --> Packages::Nuget::Symbols::CreateSymbolFilesService
The problem with this flow is that we need to use the downloaded file from Packages::Nuget::ProcessPackageFileService
inside Packages::Nuget::UpdatePackageFromMetadataService
& Packages::Nuget::MetadataExtractionService
. This is against the direction of the flow and made us unable to present the orchestration between the services in a correct way.
Also, we faced a problem as we were unable to pass the correct package to the Packages::Nuget::Symbols::CreateSymbolFilesService
when creating the symbol files. That's because we are detecting the package in Packages::Nuget::UpdatePackageFromMetadataService
while we are creating the symbol files after Packages::Nuget::ProcessPackageFileService
downloaded the package file.
To summarize, we needed to rearrange the flow and start with downloading the file and then passing it to whatever services would use it. The updated flow is as follows:
flowchart BT
Packages::Nuget::ExtractionWorker --> Packages::Nuget::ProcessPackageFileService
Packages::Nuget::ProcessPackageFileService --> Packages::Nuget::UpdatePackageFromMetadataService
Packages::Nuget::UpdatePackageFromMetadataService --> Packages::Nuget::MetadataExtractionService
Packages::Nuget::UpdatePackageFromMetadataService --> Packages::Nuget::Symbols::CreateSymbolFilesService
Packages::Nuget::MetadataExtractionService --> Packages::Nuget::ExtractMetadataFileService
By starting the flow from downloading the file, we can make sure that we are able to extract whatever file we need and maintain a predictable and one-directional orchestration between the services.
Screenshots or screen recordings
Not applicable.
How to set up and validate locally
Follow the same steps mentioned here. This is a refactoring MR so uploading a nuget package should work fine as before.
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Related to #426207 (closed)