AI Framework Test Suite - Integration of Existing Chat RSpec Tests into CI/CD Pipeline
Problem Statement
Currently, our GitLab Duo Chat lacks a robust testing mechanism that can ensure the quality and reliability of its responses. The existing RSpec tests are not integrated into our CI/CD pipeline, which makes it challenging to catch issues early and ensure the Chat is functioning as expected. This lack of integration also hinders our ability to monitor the Chat's performance over time and make data-driven improvements.
Exit Criteria
- CI/CD Integration: The existing RSpec tests for the chatbot system are fully integrated into the CI/CD pipeline. Any changes to the Chat's code should trigger these tests.
- Test Execution: The RSpec tests should run successfully in the CI/CD pipeline without any errors. Any failures should be properly logged and reported.
Considerations
- A concern about the flakiness of the test but I think the ability to run the rspec in CI still makes it easier to run a test.
Edited by David O'Regan