Persistent Valkey for external data

Many integrations read from external systems that do not keep history for you: market feeds, partner APIs, SCADA snapshots, file drops, etc. If raclette only held that data in memory or on the default ephemeral Valkey, a restart or flush would leave dashboards and jobs empty until the next successful poll.

Valkey with RDB and/or AOF lets you keep a last-known snapshot on disk. This guide is about that use case. It is not specific to login, OAuth, or sessions — those are covered under Valkey instances and Authentication when you need to split ephemeral auth data from durable tokens.

What you are solving

Problem	Approach
Upstream API is volatile; you still want yesterday’s payload after restart	Persist snapshots in Valkey (rdb, aof, or rdb+aof)
Data is large, changes often, not worth MongoDB	Valkey JSON blobs + refresh job, not a new collection
Must not mix with short-lived framework cache	Use a dedicated Valkey (or dedicated key prefix + TTL policy), not “whatever fastify.cache already does”

MongoDB remains the place for data users create and edit in workbench. Valkey snapshots are integration cache: replaceable, refreshable, usually read-only in the UI.

Option A — One Valkey with persistence (common in customer projects)

Run a single services.cache and enable persistence on that container. Sessions, plugin cache, and external snapshots share one server; ops configures aof+rdb once.

import { defineRacletteConfig } from "@raclettejs/core"

export default defineRacletteConfig({
  services: {
    cache: {
      enabled: true,
      port: 6379,
      name: "raclette-cache",
      volume: "raclette-cache-data",
    },
  },
  backend: {
    cache: {
      persistence: "rdb+aof",
      RDB_OPTIONS: "3600 1 300 100 60 10000",
    },
  },
})

yarn raclette dev applies backend.cache.persistence to the generated Valkey service command (see raclette Config).

In plugins, use the existing fastify.cache API (get, set, cache, forceUpdate) with a stable key and a TTL that matches your refresh policy:

const SNAPSHOT_KEY = "upstream-feed"

export const createFeedService = (fastify: PluginFastifyInstance) => ({
  async getFeed() {
    const cached = await fastify.cache.get<FeedPayload>(SNAPSHOT_KEY)
    if (cached) return cached

    const fresh = await fetchFromUpstream()
    await fastify.cache.forceUpdate(SNAPSHOT_KEY, fresh, { ttl: -1 })
    return fresh
  },

  async refresh() {
    const fresh = await fetchFromUpstream()
    await fastify.cache.forceUpdate(SNAPSHOT_KEY, fresh, { ttl: -1 })
    return fresh
  },
})

Schedule refresh() with fastify.createIntervalTask. Keys are automatically prefixed with your plugin key, same as today.

Trade-off: Raclette sessions and login rate limits also live on this instance. That is acceptable when ops treats the whole Valkey as the persisted integration + session store. If you need sessions to disappear on restart while snapshots remain, use option B.

Option B — Two Valkey instances (ephemeral + persistent)

Split:

1. cache — persistence: none for sessions, OAuth CSRF state, rate limits, short TTL plugin cache.
2. A second Valkey — rdb+aof only for integration snapshots (and optionally other durable blobs).

Declare the persistent instance in raclette.config with kind: valkey. In dev, raclette generates the Compose service, volume, and backend env var. The backend registers named clients on fastify.valkeys when these env vars are set:

Env variable	Instance id
CACHE_URL	cache (always)
RACLETTE_VALKEY_<NAME>_URL	service key (e.g. RACLETTE_VALKEY_CACHE_PERSISTENT_URL → cachePersistent)

Example config (dev containers started automatically):

export default defineRacletteConfig({
  services: {
    cache: {
      enabled: true,
      port: 6379,
      volume: "raclette-cache",
    },
    cachePersistent: {
      kind: "valkey",
      enabled: true,
      port: 6380,
      name: "raclette-cache-persistent",
      volume: "raclette-cache-persistent",
      persistence: "rdb+aof",
    },
  },
  backend: {
    cache: { persistence: "none" },
  },
})

yarn raclette dev sets RACLETTE_VALKEY_CACHE_PERSISTENT_URL=redis://raclette-cache-persistent:6379 on the backend. For production or an existing external cluster, set that env var (or envVar override) in your deployment instead of relying on generated Compose.

In a plugin, build a second prefixed cache on that connection (same helper the framework uses internally):

import { createCacheService } from "@m/cache/cacheManager"

export const createSnapshotStore = (
  fastify: PluginFastifyInstance,
  pluginKey: string,
) => {
  const valkey = fastify.fastify.valkeys?.cachePersistent
  if (!valkey) {
    throw new Error("Persistent Valkey instance cachePersistent is not configured")
  }
  return createCacheService(valkey, { prefix: pluginKey, defaultTtl: -1 })
}

Use fastify.cache only for data that may be ephemeral; use the snapshot store only for upstream payloads.

Map auth concerns separately via auth.valkeyInstances if needed (Valkey instances) — that is independent of where you store feed data.

Refresh and TTL

Setting	When to use
defaultTtl: -1 / ttl: -1 on forceUpdate	Snapshot stays until the next successful refresh
Positive TTL (e.g. 3600)	Bound staleness if refresh fails
Interval task	Poll upstream every N minutes; cache() skips write if JSON unchanged

Persistence survives Valkey restart. It does not replace polling: when the external API changes, your job must still run.

Operations

• Mount a volume on /data for any Valkey that uses RDB/AOF.
• Monitor memory and disk; prune old keys if you version snapshots (feed:2025-05-29).
• Document whether snapshots are safe to delete (they usually are; MongoDB stays authoritative for user-owned data).

Related

• Migrate: ephemeral auth + persistent data — split an existing single persistent Valkey after a core upgrade
• Valkey instances — optional multi-instance overview (auth roles, env vars)
• Config boilerplate — example configs for option A and B
• raclette Config — backend.cache.persistence