Change Zoekt to use Gitaly for fetching code

What does this MR do and why?

This MR adds support for the new indexer (https://gitlab.com/gitlab-org/gitlab-zoekt-indexer) in GitLab. The change is behind a feature flag use_new_zoekt_indexer so that we can gradually roll this out.

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

Before	After

How to set up and validate locally

1. Add these lines to your gdk.yml

zoekt:
  enabled: true

2. Execute gdk reconfigure
3. Run these commands in you rails console to add the flightjs namespace to Zoekt

::Feature.enable(:index_code_with_zoekt)
::Feature.enable(:search_code_with_zoekt)
zoekt_shard = ::Zoekt::Shard.find_or_create_by!(index_base_url: 'http://127.0.0.1:6080/', search_base_url: 'http://127.0.0.1:6090/')
namespace = Namespace.find_by_full_path("flightjs") # Some namespace you want to enable
::Zoekt::IndexedNamespace.find_or_create_by!(shard: zoekt_shard, namespace: namespace.root_ancestor)

4. Change the visibility of the flightjs project to public
5. Run project = Project.find_by_full_path('flightjs/Flight'); project.repository.update_zoekt_index!. Ensure that it returns {"Success"=>true}
6. Perform search in a project inside the flightjs namespace. Ensure that it returns results
7. Run Repository.truncate_zoekt_index!(Zoekt::Shard.first)
8. Enable the feature flag: Feature.enable(:use_new_zoekt_indexer)
9. Set your GDK_DIR: export GDK_DIR=/full/path/to/gdk
10. Install gitlab-zoekt-indexer: GOBIN=$GDK_DIR/zoekt/bin go install gitlab.com/gitlab-org/search-team/gitlab-zoekt-indexer/cmd/gitlab-zoekt-indexer@latest
11. Run cd $GDK_DIR && gdk stop zoekt-dynamic-indexserver-development && ./zoekt/bin/gitlab-zoekt-indexer -index_dir zoekt-data/development/index -listen :6080
12. Run project = Project.find_by_full_path('flightjs/Flight'); project.repository.update_zoekt_index!. Ensure that it returns {"Success"=>true}
13. Perform search in a project inside the flightjs namespace. Ensure that it returns results

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

• I have evaluated the MR acceptance checklist for this MR.

Related to #402984 (closed)