Restructure codebase to allow easy addition of new usecases
Problem to solve
Currently different usecases that prompt-library supports has lots of duplication because they roughly follow the same logical flow and the same code structure. But they do have subtle differences that merge them all if not ideal.
Proposal
We should extract the common repeated logic/pattern into the common module to allow minimal duplication of new usecases while allow the flexibility for them to be different.
These are some areas where we see duplication:
-
write results -
config -
apply prompt templates -
batch request LLM -
generic judges -
prepare testcases -
inference models
Further details
Links / references
Edited by Hongtao Yang