Skip to content

GitLab Chat: Use moderation API

Dmitry Gruzd requested to merge gitlab-chat-use-moderated-argument into master

What does this MR do and why?

We've just merged !119050 (merged), which introduces moderation API usage. For GitLab chat, the defaults don't make sense. We only need to use it for the initial question that's turned into embeddings.

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

How to set up and validate locally

  1. enable gdk for pgvector: gdk config set pgvector.enabled true
  2. enable the embedding database: gdk config set gitlab.rails.databases.embedding.enabled true
  3. gdk reconfigure
  4. open rails console: rails c
  5. Enable checks: Feature.enable(:openai_experimentation), Feature.enable(:tanuki_bot), Feature.enable(:openai_moderation)
  6. Add openAI api key: Gitlab::CurrentSettings.update(openai_api_key: "<openai-api-token>")
  7. Execute the script to bootstrap the database
  8. Execute Gitlab::Llm::TanukiBot.execute(current_user: User.first, question: 'What is advanced search?')
  9. Ensure that it returns a valid response
  10. Replace the question with an example from https://platform.openai.com/docs/api-reference/moderations/create
  11. It should raise an exeption Gitlab::Llm::OpenAi::Client::InputModerationError: Provided input violates OpenAI's Content Policy

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Dmitry Gruzd

Merge request reports