Include Suffix in Trimming Logic

What does this merge request do and why?

This MR will change the current logic in _apply_model_limits to trim from the suffix (if exists) first before trimming the prefix.

This MR adds a _trim_and_decode method to the class ApplyPromptTransformations that handles trimming the prompt to a specified trim_length given tokens and a boolean is_prefix to indicate whether or not the trimming is happening from the top (for prefix) or the end of the string (for suffix).

The trimming logic is then:

1. Trim as much as possible from the suffix first.
2. If the prompt token count is still bigger than the token limit, then trim from the prefix.

How to set up and validate locally

The bigquery table that confirms these changes are not breaking the pipeline can be seen here. The table name is:

unreview-poc-390200e5.gl_gitlab_experiments.db_test_new_trim_logic_v2

Merge request checklist

• I've ran the affected pipeline(s) to validate that nothing is broken.
• Tests added for new functionality. If not, please raise an issue to follow up.
• Documentation added/updated, if needed.