diff --git a/architecture/gateway.md b/architecture/gateway.md index 0c221fb3a7..ff09c783f5 100644 --- a/architecture/gateway.md +++ b/architecture/gateway.md @@ -648,11 +648,28 @@ Gateway and Sandbox Protocol token responses follow the same convention: a present expiration timestamp carries the absolute deadline, while absence means the issued token does not expire. +On-disk SQLite databases run in WAL journal mode with `synchronous=FULL`. +The adapter switches the file to WAL on a single connection before the pool +opens, then applies both settings to every pooled connection. WAL lets readers +proceed while a writer commits and reduces each commit to one WAL `fsync`, +which matters because gateway hot paths such as SSH session issuance and +revocation are many small autocommit writes. `synchronous` stays at `FULL` +because some of those writes tighten authorization: under `NORMAL`, a power +loss could roll back an acknowledged SSH session revocation and make the token +valid again. Writes that are safe to lose, currently only SSH session issuance +through `Store::create_relaxed`, use a second single-connection pool with +`synchronous=NORMAL`. Losing a minted token only invalidates it, and because +both pools share one WAL, the next `FULL` commit also makes earlier relaxed +commits durable. Deployments that need multiple replicas use Postgres, where +`create_relaxed` is an ordinary durable insert. WAL requires a local filesystem with working shared +memory, so the SQLite file must not live on a network mount, and backups must +use `sqlite3 .backup` or `VACUUM INTO` rather than copying the main file alone. + The SQLite adapter tightens the on-disk database file to mode `0o600` on every connect so that provider API keys, SSH session tokens, and sandbox metadata are not readable by other local users on shared hosts. The same restriction is -reapplied to the `-wal` and `-shm` sidecars (created by SQLite's -default WAL journal mode), which mirror the same sensitive contents. +reapplied to the `-wal` and `-shm` sidecars that WAL mode creates, +which mirror the same sensitive contents. Persisted state includes sandboxes, providers, provider profiles, provider credential refresh state, SSH sessions, policy revisions, settings, deployment diff --git a/crates/openshell-server/src/grpc/sandbox.rs b/crates/openshell-server/src/grpc/sandbox.rs index 6170201db9..2d64d57031 100644 --- a/crates/openshell-server/src/grpc/sandbox.rs +++ b/crates/openshell-server/src/grpc/sandbox.rs @@ -2597,7 +2597,9 @@ pub(super) async fn handle_create_ssh_session( // Ensure metadata is valid (defense in depth - should always be true for server-constructed metadata) super::validation::validate_object_metadata(session.metadata.as_ref(), "ssh_session")?; - // Use MustCreate to atomically ensure the session token is unique + // `create_relaxed` fails if the token already exists, like MustCreate, but + // skips the per-commit fsync on SQLite. Losing a freshly minted token in a + // crash only makes it invalid; revocation stays on the durable `put_if`. let session_labels = session.object_labels(); let session_labels_json = if session_labels.as_ref().is_none_or(HashMap::is_empty) { None @@ -2609,14 +2611,13 @@ pub(super) async fn handle_create_ssh_session( }; state .store - .put_if( + .create_relaxed( SshSession::object_type(), &token, session.object_name(), session.object_workspace(), &session.encode_to_vec(), session_labels_json.as_deref(), - WriteCondition::MustCreate, ) .await .map_err(|e| Status::internal(format!("persist ssh session failed: {e}")))?; diff --git a/crates/openshell-server/src/persistence/mod.rs b/crates/openshell-server/src/persistence/mod.rs index 0306bdd312..c4f2d54f86 100644 --- a/crates/openshell-server/src/persistence/mod.rs +++ b/crates/openshell-server/src/persistence/mod.rs @@ -384,6 +384,38 @@ impl Store { )) } + /// Create an object that is safe to lose in a crash. + /// + /// Behaves like [`Self::put_if`] with [`WriteCondition::MustCreate`], but + /// the file-backed `SQLite` store commits it with `synchronous=NORMAL`, so + /// a power loss or kernel crash shortly after the call returns may roll + /// the insert back. Use it only for objects whose absence denies access, + /// such as newly minted SSH session tokens. Writes that revoke or tighten + /// anything must use [`Self::put_if`], which is always durable. + #[tracing::instrument( + name = "store", + skip_all, + fields(otel.name = "store.create_relaxed", otel.status_code = tracing::field::Empty, object_type = %object_type, object.id = %id, object.name = %name, workspace = %workspace) + )] + pub async fn create_relaxed( + &self, + object_type: &str, + id: &str, + name: &str, + workspace: &str, + payload: &[u8], + labels: Option<&str>, + ) -> PersistenceResult { + store_dispatch_traced!(self.create_relaxed( + object_type, + id, + name, + workspace, + payload, + labels + )) + } + /// Delete an object by id with compare-and-swap support. /// /// # Arguments diff --git a/crates/openshell-server/src/persistence/postgres.rs b/crates/openshell-server/src/persistence/postgres.rs index d79335b5f5..14bae6dfb1 100644 --- a/crates/openshell-server/src/persistence/postgres.rs +++ b/crates/openshell-server/src/persistence/postgres.rs @@ -180,6 +180,29 @@ ON CONFLICT (object_type, workspace, name) WHERE name IS NOT NULL DO UPDATE SET Ok(()) } + /// Create an object; Postgres commits are always durable, so this is + /// [`Self::put_if`] with [`WriteCondition::MustCreate`]. + pub async fn create_relaxed( + &self, + object_type: &str, + id: &str, + name: &str, + workspace: &str, + payload: &[u8], + labels: Option<&str>, + ) -> PersistenceResult { + self.put_if( + object_type, + id, + name, + workspace, + payload, + labels, + WriteCondition::MustCreate, + ) + .await + } + #[allow(clippy::too_many_arguments)] pub async fn put_if( &self, diff --git a/crates/openshell-server/src/persistence/sqlite.rs b/crates/openshell-server/src/persistence/sqlite.rs index d9f22bcbbe..a2a5e74f60 100644 --- a/crates/openshell-server/src/persistence/sqlite.rs +++ b/crates/openshell-server/src/persistence/sqlite.rs @@ -15,7 +15,9 @@ use openshell_core::SetResourceVersion; use openshell_core::paths::set_file_owner_only; use openshell_core::proto::Sandbox; use prost::Message; -use sqlx::sqlite::{SqliteConnectOptions, SqliteConnection, SqlitePoolOptions}; +use sqlx::sqlite::{ + SqliteConnectOptions, SqliteConnection, SqliteJournalMode, SqlitePoolOptions, SqliteSynchronous, +}; use sqlx::{Connection, QueryBuilder, Row, Sqlite, SqlitePool}; use std::path::{Path, PathBuf}; use std::str::FromStr; @@ -39,6 +41,10 @@ use super::{DELETE_MANY_BATCH_SIZE, DRAFT_CHUNK_OBJECT_TYPE, POLICY_OBJECT_TYPE} #[derive(Debug, Clone)] pub struct SqliteStore { pool: SqlitePool, + /// Pool for writes whose loss after a crash is harmless; see + /// [`SqliteStore::create_relaxed`]. On-disk stores open it with + /// `synchronous=NORMAL`; in-memory stores share `pool`. + relaxed_pool: SqlitePool, #[cfg_attr(not(any(test, feature = "test-support")), allow(dead_code))] in_memory_keepalive: Option>>>, } @@ -70,6 +76,124 @@ pub(super) async fn replace_pool_connection(store: &SqliteStore) -> PersistenceR connection.close().await.map_err(|e| map_db_error(&e)) } +/// Apply the on-disk journal settings and switch the database file to WAL +/// once, before the pool opens its connections. +/// +/// The gateway's hot paths (SSH-session tokens minted and revoked around every +/// forwarded connection, sandbox status updates) are many small autocommit +/// writes. `SQLite`'s default rollback journal makes each of those commits pay +/// several `fsync` calls and blocks readers while a writer holds the lock, so +/// under a burst of forwarded connections the whole store serializes on disk +/// latency. WAL mode removes the reader/writer exclusion and cuts each commit +/// to a single `fsync` of the WAL file. +/// +/// The main pool keeps `synchronous=FULL` rather than the usual WAL pairing +/// of `NORMAL`. Under `NORMAL` a power loss or kernel crash can roll back +/// transactions that were already acknowledged, and several of those writes +/// tighten authorization: an SSH session revoked just before the crash would +/// come back valid for the rest of its lifetime. `FULL` keeps every +/// acknowledged commit durable. Writes whose loss only ever denies access, +/// such as minting a new SSH session token, go through a separate +/// `synchronous=NORMAL` pool instead ([`SqliteStore::create_relaxed`]). Both +/// pools append to the same WAL file, so the next `FULL` commit's `fsync` also +/// makes every earlier relaxed commit durable, and a crash can never roll back +/// a `FULL` commit. +/// +/// `journal_mode=WAL` is persistent in the database file, but switching into +/// it needs exclusive access: if another connection holds the file open, the +/// switch waits out `busy_timeout` and then fails. Doing it up front on one +/// connection means the pool connections only ever re-apply the pragma to a +/// file that is already in WAL mode, which never blocks, and a failure +/// surfaces as a single clear connect error instead of a pool error later. +/// The first start after upgrading a rollback-journal database therefore needs +/// the file to be otherwise unopened. `synchronous` is a per-connection +/// setting and is applied through the options on every connection. +/// +/// In-memory databases are left on their defaults: WAL is meaningless there +/// and the shared-cache keepalive connection already provides their lifetime +/// guarantees. +async fn configure_on_disk_durability( + options: SqliteConnectOptions, +) -> PersistenceResult { + let options = options + .journal_mode(SqliteJournalMode::Wal) + .synchronous(SqliteSynchronous::Full); + let wal_error = |e: &sqlx::Error| { + PersistenceError::Database(format!( + "failed to switch SQLite database {} to WAL journal mode (the switch needs \ + exclusive access; close other connections to the file and retry): {}", + options.get_filename().display(), + map_db_error(e) + )) + }; + let connection = SqliteConnection::connect_with(&options) + .await + .map_err(|e| wal_error(&e))?; + connection.close().await.map_err(|e| wal_error(&e))?; + Ok(options) +} + +/// Insert a new object at resource version 1, failing if it already exists. +async fn insert_new_object( + pool: &SqlitePool, + object_type: &str, + id: &str, + name: &str, + workspace: &str, + payload: &[u8], + labels: Option<&str>, +) -> PersistenceResult { + let now_ms = current_time_ms(); + sqlx::query( + r#" +INSERT INTO "objects" ("object_type", "id", "name", "workspace", "payload", "created_at_ms", "updated_at_ms", "labels", "resource_version") +VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?6, ?7, 1) +"#, + ) + .bind(object_type) + .bind(id) + .bind(name) + .bind(workspace) + .bind(payload) + .bind(now_ms) + .bind(labels.unwrap_or("{}")) + .execute(pool) + .await + .map_err(|e| map_db_error(&e))?; + + Ok(WriteResult { + resource_version: 1, + created_at_ms: now_ms, + updated_at_ms: now_ms, + }) +} + +#[cfg(test)] +pub(super) async fn journal_settings(store: &SqliteStore) -> PersistenceResult<(String, i64)> { + pool_journal_settings(&store.pool).await +} + +#[cfg(test)] +pub(super) async fn relaxed_journal_settings( + store: &SqliteStore, +) -> PersistenceResult<(String, i64)> { + pool_journal_settings(&store.relaxed_pool).await +} + +#[cfg(test)] +async fn pool_journal_settings(pool: &SqlitePool) -> PersistenceResult<(String, i64)> { + let mut connection = pool.acquire().await.map_err(|e| map_db_error(&e))?; + let journal_mode: String = sqlx::query_scalar("PRAGMA journal_mode") + .fetch_one(&mut *connection) + .await + .map_err(|e| map_db_error(&e))?; + let synchronous: i64 = sqlx::query_scalar("PRAGMA synchronous") + .fetch_one(&mut *connection) + .await + .map_err(|e| map_db_error(&e))?; + Ok((journal_mode, synchronous)) +} + impl SqliteStore { /// Closes the connection pool. #[cfg(test)] @@ -107,6 +231,10 @@ impl SqliteStore { // so we can restrict the permissions after the database is connected. let db_path = (!is_in_memory).then(|| options.get_filename().to_path_buf()); + if !is_in_memory { + options = configure_on_disk_durability(options).await?; + } + let in_memory_keepalive = if is_in_memory { let connection = SqliteConnection::connect_with(&options) .await @@ -116,11 +244,25 @@ impl SqliteStore { None }; + let relaxed_options = + (!is_in_memory).then(|| options.clone().synchronous(SqliteSynchronous::Normal)); + let pool = pool_options .connect_with(options) .await .map_err(|e| map_db_error(&e))?; + // SQLite serializes writers, so one connection is enough for the + // relaxed pool. + let relaxed_pool = match relaxed_options { + Some(relaxed_options) => SqlitePoolOptions::new() + .max_connections(1) + .connect_with(relaxed_options) + .await + .map_err(|e| map_db_error(&e))?, + None => pool.clone(), + }; + // Tighten the permissions of the database file to owner-only access (0o600). if let Some(path) = db_path { restrict_db_file_permissions(&path)?; @@ -128,6 +270,7 @@ impl SqliteStore { Ok(Self { pool, + relaxed_pool, in_memory_keepalive, }) } @@ -182,6 +325,7 @@ impl SqliteStore { /// Do not call from runtime code; this tears down the active pool. #[cfg(any(test, feature = "test-support"))] pub async fn close(&self) { + self.relaxed_pool.close().await; self.pool.close().await; if let Some(keepalive) = &self.in_memory_keepalive { let connection = keepalive.lock().await.take(); @@ -225,6 +369,33 @@ ON CONFLICT ("object_type", "workspace", "name") WHERE "name" IS NOT NULL DO UPD Ok(()) } + /// Create an object with `synchronous=NORMAL` durability. + /// + /// Same semantics as [`Self::put_if`] with [`WriteCondition::MustCreate`], + /// except that a power loss or kernel crash shortly after the call returns + /// may roll the insert back. Use it only for objects whose absence denies + /// access, never for writes that revoke or tighten anything. + pub async fn create_relaxed( + &self, + object_type: &str, + id: &str, + name: &str, + workspace: &str, + payload: &[u8], + labels: Option<&str>, + ) -> PersistenceResult { + insert_new_object( + &self.relaxed_pool, + object_type, + id, + name, + workspace, + payload, + labels, + ) + .await + } + #[allow(clippy::too_many_arguments)] pub async fn put_if( &self, @@ -240,29 +411,16 @@ ON CONFLICT ("object_type", "workspace", "name") WHERE "name" IS NOT NULL DO UPD match condition { WriteCondition::MustCreate => { - // Insert only - fail if object exists - sqlx::query( - r#" -INSERT INTO "objects" ("object_type", "id", "name", "workspace", "payload", "created_at_ms", "updated_at_ms", "labels", "resource_version") -VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?6, ?7, 1) -"#, + insert_new_object( + &self.pool, + object_type, + id, + name, + workspace, + payload, + labels, ) - .bind(object_type) - .bind(id) - .bind(name) - .bind(workspace) - .bind(payload) - .bind(now_ms) - .bind(labels.unwrap_or("{}")) - .execute(&self.pool) .await - .map_err(|e| map_db_error(&e))?; - - Ok(WriteResult { - resource_version: 1, - created_at_ms: now_ms, - updated_at_ms: now_ms, - }) } WriteCondition::MatchResourceVersion(expected_version) => { // Update with version check diff --git a/crates/openshell-server/src/persistence/tests.rs b/crates/openshell-server/src/persistence/tests.rs index 49fa192094..3a27b7b568 100644 --- a/crates/openshell-server/src/persistence/tests.rs +++ b/crates/openshell-server/src/persistence/tests.rs @@ -324,14 +324,265 @@ async fn sqlite_connect_tightens_existing_db_file_permissions() { assert_eq!(mode, 0o600, "expected 0600, got {mode:04o}"); } +fn on_disk_store_url(db_path: &std::path::Path) -> String { + format!("sqlite:{}?mode=rwc", db_path.display()) +} + +async fn connect_on_disk_sqlite(url: &str) -> super::SqliteStore { + match Store::connect(url).await.expect("connect to sqlite") { + Store::Sqlite(store) => store, + Store::Postgres(_) => unreachable!("sqlite URL must select the SQLite store"), + } +} + +async fn file_journal_mode(db_path: &std::path::Path) -> String { + use sqlx::{Connection, SqliteConnection}; + + let mut connection = SqliteConnection::connect(&format!("sqlite:{}", db_path.display())) + .await + .expect("open database file directly"); + let journal_mode: String = sqlx::query_scalar("PRAGMA journal_mode") + .fetch_one(&mut connection) + .await + .expect("read journal_mode"); + connection.close().await.expect("close direct connection"); + journal_mode +} + +#[tokio::test] +async fn sqlite_connect_enables_wal_and_full_synchronous_on_disk() { + let tmp = tempfile::tempdir().expect("tempdir"); + let db_path = tmp.path().join("openshell.db"); + let url = on_disk_store_url(&db_path); + + let store = connect_on_disk_sqlite(&url).await; + + let (journal_mode, synchronous) = super::sqlite::journal_settings(&store) + .await + .expect("read journal settings through the pool"); + assert_eq!(journal_mode, "wal", "on-disk stores must run in WAL mode"); + // FULL (2), not NORMAL (1): acknowledged commits such as SSH session + // revocations must survive a power loss. + assert_eq!( + synchronous, 2, + "on-disk stores must run with synchronous=FULL (2), got {synchronous}" + ); + let (relaxed_journal_mode, relaxed_synchronous) = + super::sqlite::relaxed_journal_settings(&store) + .await + .expect("read journal settings through the relaxed pool"); + assert_eq!(relaxed_journal_mode, "wal"); + assert_eq!( + relaxed_synchronous, 1, + "the relaxed pool must run with synchronous=NORMAL (1), got {relaxed_synchronous}" + ); + + // Force a write so the WAL sidecars exist on disk, then confirm they are + // owner-only like the main file. + store + .put( + "sandbox", + "wal-probe", + "wal-probe", + "default", + b"payload", + None, + ) + .await + .expect("write through the store"); + let [wal_path, shm_path] = super::sqlite::sqlite_sidecar_paths(&db_path); + assert!(wal_path.exists(), "WAL sidecar should exist after a write"); + assert!(shm_path.exists(), "SHM sidecar should exist after a write"); + #[cfg(unix)] + { + use std::os::unix::fs::PermissionsExt; + for path in [&db_path, &wal_path, &shm_path] { + let mode = std::fs::metadata(path) + .expect("sidecar metadata") + .permissions() + .mode() + & 0o777; + assert_eq!( + mode, + 0o600, + "{}: expected 0600, got {mode:04o}", + path.display() + ); + } + } + + store.close().await; + assert_eq!( + file_journal_mode(&db_path).await, + "wal", + "WAL must persist in the database file after the store closes" + ); +} + +#[tokio::test] +async fn sqlite_create_relaxed_is_must_create_visible_to_durable_writes() { + let tmp = tempfile::tempdir().expect("tempdir"); + let db_path = tmp.path().join("openshell.db"); + let store = Store::connect(&on_disk_store_url(&db_path)) + .await + .expect("connect to sqlite"); + + let created = store + .create_relaxed("ssh_session", "tok", "tok", "default", b"issued", None) + .await + .expect("relaxed create"); + assert_eq!(created.resource_version, 1); + + let duplicate = store + .create_relaxed("ssh_session", "tok", "tok", "default", b"again", None) + .await + .expect_err("relaxed create must reject an existing object"); + assert!( + matches!(duplicate, PersistenceError::UniqueViolation { .. }), + "expected UniqueViolation, got {duplicate:?}" + ); + + // The durable pool sees the relaxed insert and can revoke it with CAS. + store + .put_if( + "ssh_session", + "tok", + "tok", + "default", + b"revoked", + None, + super::WriteCondition::MatchResourceVersion(1), + ) + .await + .expect("durable revoke of a relaxed insert"); + let record = store + .get("ssh_session", "tok") + .await + .expect("get") + .expect("record present"); + assert_eq!(record.payload, b"revoked"); + assert_eq!(record.resource_version, 2); +} + +#[tokio::test] +async fn sqlite_connect_switches_existing_rollback_journal_database_to_wal() { + use sqlx::sqlite::SqliteConnectOptions; + use sqlx::{Connection, SqliteConnection}; + use std::str::FromStr; + + let tmp = tempfile::tempdir().expect("tempdir"); + let db_path = tmp.path().join("openshell.db"); + let url = on_disk_store_url(&db_path); + + // A database created by an older gateway, or by any sqlx 0.8 default + // connection, is in rollback-journal (`delete`) mode. + { + let options = SqliteConnectOptions::from_str(&url) + .expect("parse url") + .create_if_missing(true); + let mut connection = SqliteConnection::connect_with(&options) + .await + .expect("create legacy database"); + sqlx::query("CREATE TABLE legacy_probe (x INTEGER)") + .execute(&mut connection) + .await + .expect("write legacy database"); + connection.close().await.expect("close legacy connection"); + } + assert_eq!(file_journal_mode(&db_path).await, "delete"); + + let store = connect_on_disk_sqlite(&url).await; + let (journal_mode, _) = super::sqlite::journal_settings(&store) + .await + .expect("read journal settings"); + assert_eq!( + journal_mode, "wal", + "connect must switch existing files to WAL" + ); + store.close().await; + assert_eq!(file_journal_mode(&db_path).await, "wal"); +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 4)] +async fn sqlite_file_backed_reads_and_writes_proceed_concurrently() { + let tmp = tempfile::tempdir().expect("tempdir"); + let db_path = tmp.path().join("openshell.db"); + let url = on_disk_store_url(&db_path); + let store = std::sync::Arc::new(Store::connect(&url).await.expect("connect to sqlite")); + + store + .put("sandbox", "seed", "seed", "default", b"seed", None) + .await + .expect("seed object"); + + // Mirror the forward-service pattern: many independent autocommit writes + // (a relaxed insert and a durable update per "connection", so the two + // pools contend for the writer lock) while readers keep fetching. Every operation must complete; no caller may observe a + // "database is locked" error even though the writes contend for the + // single SQLite writer. + let writers = (0..8).map(|writer| { + let store = store.clone(); + tokio::spawn(async move { + for index in 0..25 { + let id = format!("session-{writer}-{index}"); + store + .create_relaxed("ssh_session", &id, &id, "default", b"issued", None) + .await + .expect("insert session"); + store + .put_if( + "ssh_session", + &id, + &id, + "default", + b"revoked", + None, + super::WriteCondition::MatchResourceVersion(1), + ) + .await + .expect("revoke session"); + } + }) + }); + let readers = (0..4).map(|_| { + let store = store.clone(); + tokio::spawn(async move { + for _ in 0..100 { + let record = store + .get("sandbox", "seed") + .await + .expect("read while writers are active") + .expect("seed object present"); + assert_eq!(record.payload, b"seed"); + } + }) + }); + + for handle in writers.chain(readers) { + handle.await.expect("task completed"); + } + + let sessions = store + .list("ssh_session", "default", 1000, 0) + .await + .expect("list sessions"); + assert_eq!(sessions.len(), 200, "every session write must be durable"); + assert!( + sessions.iter().all(|record| record.payload == b"revoked"), + "every session must have been revoked by its follow-up write" + ); +} + // The next three tests cover `restrict_db_file_permissions` against the // WAL/SHM sidecars at increasing levels of fidelity: // // 1. `_tightens_main_and_wal_and_shm_files`: synthetic empty files, proves // the chmod loop walks all three paths. -// 2. `_skips_missing_sidecars`: proves the `exists()` guard, which is the -// actual production path today (sqlx 0.8 doesn't default to WAL and -// doesn't accept `journal_mode` as a URL parameter). +// 2. `_skips_missing_sidecars`: proves the `exists()` guard for databases +// that have not been written yet or were created before the adapter +// enabled WAL (sqlx 0.8 doesn't default to WAL and doesn't accept +// `journal_mode` as a URL parameter; `SqliteStore::connect` opts in +// through the builder API). // 3. `_handles_real_sqlite_wal_files`: opens a real sqlx pool with // `SqliteJournalMode::Wal` via the builder API so SQLite materializes // real `-wal` and `-shm` files, then checks the helper tightens them. diff --git a/deploy/helm/openshell/README.md b/deploy/helm/openshell/README.md index f0a96b143d..0eac39bb4b 100644 --- a/deploy/helm/openshell/README.md +++ b/deploy/helm/openshell/README.md @@ -281,7 +281,7 @@ discovery endpoint or its TLS CA. | server.credentialDrivers.vault.timeoutSecs | string | `""` | HTTP request timeout in seconds. Empty = driver default. | | server.credentialDrivers.vault.tokenPath | string | `""` | Mounted token file path when authMethod is token_file. | | server.credentialStorage.existingSecret | string | `""` | Name of a pre-existing Secret containing the key-encryption key. When set, the chart does NOT generate a new Secret; it references this one instead. The Secret must contain a key named "key-encryption-key" with a base64-encoded 32-byte value. Required for GitOps workflows that render manifests with `helm template` (where `lookup` is unavailable). | -| server.dbUrl | string | `"sqlite:/var/openshell/openshell.db"` | Gateway database URL (used for the default SQLite backend). | +| server.dbUrl | string | `"sqlite:/var/openshell/openshell.db"` | Gateway database URL (used for the default SQLite backend). SQLite runs in WAL mode and needs a local block-backed volume, not NFS or other network filesystems. | | server.defaultRuntimeClassName | string | `""` | Default Kubernetes runtimeClassName for sandbox pods. Applied when a CreateSandbox request does not specify one. Empty (default) = omit the field, using the cluster's default RuntimeClass. Set to a RuntimeClass name (e.g. "kata-containers", "nvidia") to apply it to all sandboxes that don't explicitly override it. | | server.disableTls | bool | `false` | Disable TLS entirely - the server listens on plaintext HTTP. Set to true when a reverse proxy / tunnel terminates TLS at the edge. | | server.drivers.kubernetes.operatorNamespaceFile | string | `""` | Path to a JSON file containing an array of namespace names allowed in operator mode. Hot-reloaded on change. | diff --git a/deploy/helm/openshell/values.yaml b/deploy/helm/openshell/values.yaml index 67d5587ec9..b141676113 100644 --- a/deploy/helm/openshell/values.yaml +++ b/deploy/helm/openshell/values.yaml @@ -206,7 +206,9 @@ server: # -- Namespace where sandbox pods are created. Defaults to the Helm release # namespace (.Release.Namespace) when left empty. sandboxNamespace: "" - # -- Gateway database URL (used for the default SQLite backend). + # -- Gateway database URL (used for the default SQLite backend). SQLite runs + # in WAL mode and needs a local block-backed volume, not NFS or other network + # filesystems. dbUrl: "sqlite:/var/openshell/openshell.db" # -- Name of a pre-existing Opaque Secret containing a PostgreSQL # connection URI (key: uri). When set, the gateway reads OPENSHELL_DB_URL diff --git a/deploy/rpm/CONFIGURATION.md b/deploy/rpm/CONFIGURATION.md index 7ae1f6f15e..44d9fc5b50 100644 --- a/deploy/rpm/CONFIGURATION.md +++ b/deploy/rpm/CONFIGURATION.md @@ -229,6 +229,10 @@ overrides that persist across package upgrades. The database URL is not accepted in TOML. When `OPENSHELL_DB_URL` is unset, the gateway uses `sqlite:$XDG_STATE_HOME/openshell/gateway/openshell.db`. +The SQLite database runs in WAL mode with `synchronous=FULL` (SSH session +issuance alone uses `NORMAL`), so +`openshell.db-wal` and `openshell.db-shm` sit next to it and must be kept +together with it; back it up with `sqlite3 openshell.db ".backup "`. ### Driver TOML settings diff --git a/docs/reference/gateway-config.mdx b/docs/reference/gateway-config.mdx index 5999176975..bd862ca03a 100644 --- a/docs/reference/gateway-config.mdx +++ b/docs/reference/gateway-config.mdx @@ -18,6 +18,8 @@ Gateway CLI flag > gateway OPENSHELL_* env var > TOML file > built-in defa `database_url` is env-only. The loader rejects it when it appears in the file. When `OPENSHELL_DB_URL` is unset, the gateway stores its SQLite database under `$XDG_STATE_HOME/openshell/gateway/openshell.db`. +On-disk SQLite databases run in WAL mode with `synchronous=FULL`: the gateway keeps `openshell.db-wal` and `openshell.db-shm` next to the database file, the file must live on a local filesystem, and a backup must use `sqlite3 openshell.db ".backup "` rather than copying `openshell.db` on its own. Every acknowledged write, including SSH session revocations, survives a power loss. The one exception is SSH session issuance, which skips the per-commit sync; a power loss can drop a just-issued session token, and the client must request a new one. + `name` assigns an operator-facing identity to the gateway installation. Set it with `[openshell.gateway].name`, `--name`, or `OPENSHELL_GATEWAY_NAME`. It defaults to `openshell`; the Helm chart defaults it to the chart fullname so all replicas in one installation share a name. Chart fullnames are only unique within their Kubernetes namespace, so set `server.name` explicitly when one collector receives telemetry from multiple namespaces or clusters. This identity is independent of client-side gateway aliases, TLS names, and `gateway_jwt.gateway_id`. ## Package-Managed Locations