Skip to content

Log all GraphQL requests

Tan Le requested to merge log-every-graphql-request into main

What does this merge request do and why?

This MR logs all GraphQL requests. We would like to understand why we reach an abnormal amount of requests in the RCA eval run in staging.

Screenshot_2024-08-08_at_3.34.29_PM

(source)

How to set up and validate locally

  1. Ensure GCP environment variables are setup.
  2. Check out to this merge request's branch.
  3. Run the follow command to kick off the pipeline.
    $ poetry run promptlib root-cause-analysis eval --config-file=data/config/root_cause_analysis/eval_slash_troubleshoot_experiment.json --test-run --sample-size 1
    Getting evaluation from claude-3-5-sonnet: 0it [00:00, ?it/s] INFO:promptlib.completion.graphql_util:Request payload digest e139e10a68d51c4b4d313609996ea4f3. Attempt 1
    INFO:promptlib.completion.graphql_util:Request payload digest 8411502fb122682a99fd128383e93e23. Attempt 1
    INFO:promptlib.completion.graphql_util:Request payload digest 8411502fb122682a99fd128383e93e23. Attempt 2
    INFO:promptlib.completion.graphql_util:Request payload digest 8411502fb122682a99fd128383e93e23. Attempt 3
    INFO:promptlib.completion.graphql_util:Request payload digest 8411502fb122682a99fd128383e93e23. Attempt 4
    Requesting answers from slash-troubleshoot: 1it [00:28, 28.87s/it]                              
    Getting evaluation from claude-3-5-sonnet: 1it [00:50, 50.39s/it]]                              
    INFO:promptlib.common.beam.io:Output written to CSV: data/output/experiment_slash_troubleshoot_20240809_172243__independent_llm_judge-00000-of-00001.csv

Merge request checklist

  • I've ran the affected pipeline(s) to validate that nothing is broken.
  • Tests added for new functionality. If not, please raise an issue to follow up.
  • Documentation added/updated, if needed.
Edited by Tan Le

Merge request reports

Loading