Skip to content

Add node.concurrency to internalAPI

Dmitry Gruzd requested to merge add-concurrency-to-internal-api into main

What does this MR do and why?

This MR adds a new node.concurrency argument to the indexer, which will be helpful for determining the number of jobs we want to give to the indexer.

Most likely we'll set it to params['node.concurrency'] - params['node.task_count'] on the Rails side.

How to set up and validate locally

  1. Export GDK_DIR 
    export GDK_DIR=<PATH_TO_GDK>
  2. Execute Zoekt indexer in GDK mode 
    make gdk
  3. Note the concurrency in the log message 
    2023/12/02 08:38:40 Starting taskRequest for 'm1.local' { index_url: 'http://localhost:6080', search_url: 'http://localhost:6090', gitlab_url: 'http://localhost:3000', concurrency: 10 }
  4. Tail log/development.log
  5. Verify that you see node.concurrency as of the parameters for /api/v4/internal/search/zoekt/<UUID>/tasks. It should be set to the numbers of cores you have (or to GOMAXPROCS if you have it overridden)
Edited by Dmitry Gruzd

Merge request reports