Prefix Caching
Caches the processed state of a recurring
prompt prefix
so only the new suffix is processed.
Tip ☺
Put the
static
instructions and examples first and the user's input last, so the prefix stays identical across calls.
Note
The cache is evicted
least-recently-used
first; call
clearCaches()
to purge it.