Skip to content

Better async requests

Hongtao Yang requested to merge better-async-requests into main

What does this merge request do and why?

This MR uses aiohttp to manage async requests. Compared to the current way, this approach 2 major benefits

  • Better performance.
  • Significantly simplify the ModelHandle trait implementation. All the common async logic are abstracted away into a separate function which uses aiohttp. This means that each concrete implantation becomes much simpler, we just need to specify the header, json payload and response parsing. Adding a new model becomes simpler now.

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

Merge request checklist

  • I've ran the affected pipeline(s) to validate that nothing is broken.
  • Tests added for new functionality. If not, please raise an issue to follow up.
  • Documentation added/updated, if needed.
Edited by Hongtao Yang

Merge request reports

Loading