Investigate a less memory-intensive approach for CSV export
Problem to solve
@ahegyi commented in an MR review:
(non-blocker): Just want to point out that the current way we write CSV file seems quite memory heavy:
- Write data in batches into a tempfile.
- Load the file into memory.
- Send it back to the client.
If you would pass a block to the
render
method, it would yield the tempfile object. With that, you might be able to use thesend_file
method (https://apidock.com/rails/ActionController/Streaming/send_file).
Proposal
Use send_file
to respond to CSV export requests, or alternatively pass the tempfile block to the render
method.
Edited by Dan Jensen