Add file_metadata jsonb
What does this MR do and why?
- This MR adds
file_metadata
field of typejsonb
to store extra information related to the associatedpackage file
needed by frontend. - We are populating the
file_metadata
field asynchronously usingCorpusFileMetadataUpdateWorker
. This worker will be triggered inafter_create
callback. -
CorpusFileMetadataUpdateWorker
parses thezip
file and fetch the required metadata from the zip. Currently, we are only supporting thezip
file.
Steps to test
- Create a
package_file
using the following command. The command will also create a package.
curl --header "PRIVATE-TOKEN: uzsLZ6teL2oWJobTmTJE" \
--upload-file ~/Downloads/zip_2MB.zip \
"http://gitlab.localdev:3000/api/v4/projects/1/packages/generic/corpus8_package/0.0.1/zip_2MB.zip?status=hidden&select=package_file"
Note:
a. Please adjust the PRIVATE-TOKEN
and project_id
as per your setup.
b. You can use attachedzip_2MB.zip zip file or create your own.
- The above API will respond with the
package_id
. Use thepackage_id
to create a corpus.
AppSec::Fuzzing::Coverage::Corpus.create(project_id: 1, package_id: 33)
- The
create
will respond with emptyfile_metadata
and trigger a background job.
file_metadata: {}
- Reload the object and it should show the
file_metadata
as follows:
file_metadata: {"total_size"=>2036861, "total_files"=>3}
Note: Since this MR is introducing a new background job you may have to restart your sidekiq with command gdk restart rails-background-jobs
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR. -
Review from scalability team. -
Review from db team.
Related to #341318 (closed)
Edited by Aditya Tiwari