Caching

By default, AI JSON caches all action outputs with a shelve file in a temporary directory.

Controlling caching

To disable caching, set the cache variable to False in your action:

from aijson import register_action

@register_action(
    # will never cache this result
    cache=False,
)
def my_action() -> str:
    return "Hello, world!"

Alternatively, to disable caching for a specific run of the action, subclass CacheControlOutputs on the output object:

from aijson import CacheControlOutputs, register_action


class Outputs(CacheControlOutputs):
    response: str


@register_action
def my_action(name: str) -> Outputs:
    if name == "world":
        # will not cache this result
        return Outputs(
            response="Hello, world!",
            _cache=False,
        )
    # will cache this result
    return Outputs(
        response="Hello, " + name + "!"
    )

For more information see Output Modifiers.

Providers

On disk

The default AI JSON behavior is to create a temporary file for storing cache entries.

Every flow will use a new temporary directory.

To share the directory among flows, or to persist it across runs, explicitly instantiate the ShelveCacheRepo, and pass it to the Flow constructor:

from aijson import Flow, ShelveCacheRepo

cache_repo = ShelveCacheRepo(
    temp_dir="my_cache_dir"
)

flow = Flow.from_file(
   "flow.ai.yaml",
   cache_repo=cache_repo,
)

With Redis

We also support Redis as a cache backend:

1. Run Redis locally or use a cloud provider.

2. Set the following environment variables:

• REDIS_HOST (required)
• REDIS_PASSWORD (required)
• REDIS_PORT (optional, defaults to 6379)
• REDIS_USERNAME (optional, defaults to empty string)

3. Override the default cache with:

from aijson import Flow, RedisCacheRepo

flow = Flow.from_file(
   "flow.ai.yaml",
   cache_repo=RedisCacheRepo,
)