I'm the developer of SMS Import / Export, an Android Kotlin app that sequentially writes and reads large (multi GiB) files. A few months ago, I decided to add encryption to the app. I assumed that this would be a simple matter of finding an appropriate library / API that implemented the recommended modern standards, something in the AES or ChaCha20 families, and wiring it up.

Then I realized that I really should use authenticated encryption, and that's where the headaches began. Java / Kotlin offers CryptoOutputStream and CryptoInputStream, which can be used with AES-GCM or ChaChaPoly20-Poly1305 for authenticated encryption, so I initially tried them - but soon realized that they don't do streaming authenticated encryption and only authenticate the input stream after reading all of it, which means that I need to either buffer multiple GiBs of decrypted data in memory - not feasible on Android devices; store the decrypted data somewhere on disk - obviously insecure, or have my app process not-yet-authenticated data - obviously a bad idea.

I looked around for a ready-made streaming authenticated encryption solution, but the only one I could find was Google Tink's Streaming AEAD primitive. I tried using it, but I found Tink frustrating to work with, in large part due to its highly opinionated insistence on the use of its key management abstractions and its push toward the use of centralized key management systems.

So I wound up writing my own high level streaming authenticated wrapper around the libsodium SecretStream API, which I eventually split off into its own library, SecretStream For Kotlin:

Internally, SecretStreamOutputStream accumulates data passed in write() calls in a fixed size buffer; whenever this buffer fills up, the data is passed to the SecretStream API for encryption and the result is then written to the underlying OutputStream. (When the stream is closed, any remaining data is written in a final chunk, which may be smaller than the standard chunk size.) SecretStreamInputStream reads data in fixed size chunks, each of which is passed to the SecretStream API for decryption and authentication and the result stored in a fixed size buffer, from where it is provided to read() calls. No data is returned until it has been authenticated.

I'm not a cryptographer, and I was really hoping to find a ready-made solution and avoid reinventing any wheels or rolling my own crypto.

Questions: Is my approach sensible? Is there a better approach? Did I miss any useful facilities or libraries that I should have been using?

Not the first, will not be the last