Skip to content

Better Chunking

Hongtao Yang requested to merge better-sampling into main

This MR simplifies code chunkers and ensure that results from different chunkers are comparable.

  • Consolidate different chunking method into a single chunker.
  • Remove unnecessary ABC class
  • Different chunking methods takes in the same breakpoints (cursors) to ensure the results are comparable.

Merge request reports