Skip to content

Add extra validation in corpus model

Aditya Tiwari requested to merge 345454-add-validation-for-corpus into master

What does this MR do and why?

Adds the following validations:

  1. Package with package file.
  2. Package file in supported zip format.
  3. Unique package for corpuses.

How to set up and validate locally

1. package_with_package_file


project = Project.last.id
package1 = Packages::Package.create!(project_id: project, name: 'package3', package_type: 'generic', version: '1')

AppSec::Fuzzing::Coverage::Corpus.create!(package: package1, project: package1.project)


pry(main)> AppSec::Fuzzing::Coverage::Corpus.create!(package: package1, project: package1.project)

  TRANSACTION (0.2ms)  BEGIN /*application:console,db_config_name:main,line:/ee/app/models/app_sec/fuzzing/coverage/corpus.rb:51:in `first_package_file'*/
  Packages::PackageFile Load (0.3ms)  SELECT "packages_package_files".* FROM "packages_package_files" WHERE "packages_package_files"."package_id" = 39 ORDER BY "packages_package_files"."id" ASC LIMIT 1 /*application:console,db_config_name:main,line:/ee/app/models/app_sec/fuzzing/coverage/corpus.rb:51:in `first_package_file'*/
  Packages::PackageFile Load (0.3ms)  SELECT "packages_package_files".* FROM "packages_package_files" WHERE "packages_package_files"."package_id" = 39 ORDER BY "packages_package_files"."id" ASC LIMIT 1 /*application:console,db_config_name:main,line:/ee/app/models/app_sec/fuzzing/coverage/corpus.rb:51:in `first_package_file'*/
  TRANSACTION (0.2ms)  ROLLBACK /*application:console,db_config_name:main,line:/lib/gitlab/database.rb:312:in `rollback'*/
ActiveRecord::RecordInvalid: Validation failed: Package should have an associated package file
from /Users/aditya-work/.asdf/installs/ruby/2.7.4/lib/ruby/gems/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/validations.rb:80:in `raise_validation_error'
[28] pry(main)>

2. validate_file_format

file = Packages::PackageFile.last
file.update!(file_name: 'test')

package1.package_files << file

pry(main)> AppSec::Fuzzing::Coverage::Corpus.create!(package: package1, project: package1.project)
ActiveRecord::RecordInvalid: Validation failed: Package format is not supported



3. unique package_id
package1.package_files.reload
file = Packages::PackageFile.last
file.update!(file_name: 'test.zip')

package1.package_files << file

[73] pry(main)> AppSec::Fuzzing::Coverage::Corpus.create!(package: package1, project: package1.project)
  TRANSACTION (0.2ms)  BEGIN /*application:console,db_config_name:main,line:(pry):85:in `__pry__'*/
  AppSec::Fuzzing::Coverage::Corpus Create (0.5ms)  INSERT INTO "coverage_fuzzing_corpuses" ("project_id", "package_id", "created_at", "updated_at") VALUES (20, 39, '2021-11-15 10:34:00.598980', '2021-11-15 10:34:00.598980') RETURNING "id" /*application:console,db_config_name:main,line:(pry):85:in `__pry__'*/
  TRANSACTION (0.2ms)  ROLLBACK /*application:console,db_config_name:main,line:/lib/gitlab/database.rb:312:in `rollback'*/
ActiveRecord::RecordNotUnique: PG::UniqueViolation: ERROR:  duplicate key value violates unique constraint "index_coverage_fuzzing_corpuses_on_package_id"
DETAIL:  Key (package_id)=(39) already exists.

Numbered steps to set up and validate the change are strongly suggested.

Database


╰─>$ rails db:migrate:down VERSION=20211112073413
== 20211112073413 ChangePackageIndexOnCorpus: reverting =======================
-- transaction_open?()
   -> 0.0000s
-- index_exists?(:coverage_fuzzing_corpuses, :package_id, {:name=>"index_coverage_fuzzing_corpuses_on_package_id", :algorithm=>:concurrently})
   -> 0.0063s
-- execute("SET statement_timeout TO 0")
   -> 0.0005s
-- remove_index(:coverage_fuzzing_corpuses, {:name=>"index_coverage_fuzzing_corpuses_on_package_id", :algorithm=>:concurrently, :column=>:package_id})
   -> 0.0103s
-- execute("RESET statement_timeout")
   -> 0.0007s
-- transaction_open?()
   -> 0.0000s
-- index_exists?(:coverage_fuzzing_corpuses, :package_id, {:name=>"index_coverage_fuzzing_corpuses_on_package_id", :algorithm=>:concurrently})
   -> 0.0016s
-- add_index(:coverage_fuzzing_corpuses, :package_id, {:name=>"index_coverage_fuzzing_corpuses_on_package_id", :algorithm=>:concurrently})
   -> 0.0069s
== 20211112073413 ChangePackageIndexOnCorpus: reverted (0.0345s) ==============


╰─>$ rails db:migrate
== 20211112073413 ChangePackageIndexOnCorpus: migrating =======================
-- execute("DELETE FROM coverage_fuzzing_corpuses")
   -> 0.0045s
-- index_exists?(:coverage_fuzzing_corpuses, :package_id, {:name=>"index_coverage_fuzzing_corpuses_on_package_id"})
   -> 0.0041s
-- transaction_open?()
   -> 0.0000s
-- index_exists?(:coverage_fuzzing_corpuses, :package_id, {:name=>"index_coverage_fuzzing_corpuses_on_package_id", :algorithm=>:concurrently})
   -> 0.0019s
-- execute("SET statement_timeout TO 0")
   -> 0.0006s
-- remove_index(:coverage_fuzzing_corpuses, {:name=>"index_coverage_fuzzing_corpuses_on_package_id", :algorithm=>:concurrently, :column=>:package_id})
   -> 0.0045s
-- execute("RESET statement_timeout")
   -> 0.0005s
-- transaction_open?()
   -> 0.0000s
-- index_exists?(:coverage_fuzzing_corpuses, :package_id, {:unique=>true, :name=>"index_coverage_fuzzing_corpuses_on_package_id", :algorithm=>:concurrently})
   -> 0.0013s
-- add_index(:coverage_fuzzing_corpuses, :package_id, {:unique=>true, :name=>"index_coverage_fuzzing_corpuses_on_package_id", :algorithm=>:concurrently})
   -> 0.0026s
== 20211112073413 ChangePackageIndexOnCorpus: migrated (0.0230s) ==============

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #345454 (closed)

Edited by Aditya Tiwari

Merge request reports

Loading