Skip to content

Make horizontal data generation more robust

Nailia Iskhakova (OOO) requested to merge 440-gpt-data-generator-horizontal-data into master

After investigation of customer's logs in #440 (closed), we identified what should be done to make data generation more robust #440 (comment 564723054).

  • Pin groups thread iterations to pool size
  • Add a wait after change_env_settings call so that the change is propagated
    • Was able to replicate scheduled group deletion problem that we see in the logs. Turned out that the delete request was called too quickly after disable_soft_delete_settings, so that the env setting hasn't changed yet. Adding a wait resolved the issue.
  • Use subgroup number as a base for child projects.
    • Previously we could have gpt-subgroup-1 with projects gpt-project-(20...25). Now we use subgroup number for child projects name, so that we will always have gpt-subgroup-1 with gpt-project-(1...5).

Closes #440 (closed)

Merge request reports