The Sync Engine I: The Outbox

This is Part 6 of the Offline-First Flutter series — and the moment the sync engine comes alive. We have local writes (Part 4) and a trustworthy online signal (Part 5). Now we push local changes to the server, which is harder than it looks:

The naive version — "when online, POST each changed note" — breaks the instant reality intrudes: the request times out after the server saved it (did it save?), the app is killed mid-push, the user edits a note while it's being sent, two creates fire twice. The outbox pattern is how you push reliably and exactly once despite all of that.

This is real distributed-systems work, made tractable. Let's build it.

Stack: Drift + an abstract REST API, Flutter 3.38 / Dart 3.12. Builds on sync metadata and connectivity.

The analogy: the outbox tray

Analogy — the physical outbox tray. On an old office desk sits an outbox tray. You drop outgoing letters in it and carry on with your day — you don't wait by the mailbox. The mail carrier comes by whenever and takes whatever's in the tray. A letter stays in the tray until it's actually collected; if the carrier doesn't come (no signal), it just waits. And you write a clear address on each so even if a letter is somehow picked up twice, it still reaches exactly one recipient.

That tray is the outbox: a durable list of "changes that still need to reach the server." Writing to it is instant; draining it is the sync engine's job; items persist until confirmed; and idempotency keys stop duplicates.

Two outbox designs

There are two common ways to remember "what needs pushing," and it's worth knowing both:

| Design | What it stores | Pros | Cons | | --- | --- | --- | --- | | A. Dirty-flag | isDirty on each row (Part 2) | Dead simple; one flag | Loses history — only the current state, not the sequence of ops | | B. Operation log | A separate outbox table of operations (create/update/delete + payload, ordered) | Preserves order & intent; supports replay/audit | More moving parts; must dedupe |

Which to use? For state-sync of simple records (like FieldNotes), the dirty-flag approach is enough and elegant: "push the current state of every dirty row." For collaborative/event-sourced systems where the sequence of edits matters, an operation log is better. We'll use the dirty-flag as the spine and show the operation-log table for when you need it. Both are "outboxes" — durable records of pending work.

Design A: pushing dirty rows

The core push loop, in plain terms:

1. Read all rows where isDirty = true.
2. For each, send its current state to the server.
3. On success: mark it clean (isDirty = false) and store the server's version.
4. On failure: leave it dirty — it'll be retried next sync.

The API behind an abstract interface (so we're not tied to a vendor):

abstract interface class NoteApi {
  /// Idempotent upsert: server uses note.id (our UUID) as the key.
  /// Returns the authoritative server version after applying.
  Future<PushResult> pushNote(Note note);
}

class PushResult {
  final int serverVersion;
  final bool conflict;       // server had a newer version (handled in Part 8)
  const PushResult(this.serverVersion, {this.conflict = false});
}

The push, in the sync engine:

Future<void> pushDirtyNotes() async {
  final dirty = await (db.select(db.notes)..where((n) => n.isDirty.equals(true)))
      .get();

  for (final note in dirty) {
    try {
      final result = await api.pushNote(note);          // may throw on network error
      if (result.conflict) {
        await _handleConflict(note, result);            // Part 8
        continue;
      }
      // Success: clear dirty IF it hasn't changed since we read it (see race below).
      await _markCleanIfUnchanged(note, result.serverVersion);
    } catch (e) {
      // Network/server error: leave it dirty, try again next sync.
      _log.warning('push failed for ${note.id}, will retry: $e');
      // Optionally: increment a retry counter / backoff (below).
    }
  }
}

The key invariant: a row is marked clean only after the server acknowledges it. If anything goes wrong — timeout, crash, app killed — the row stays dirty, so the next sync simply tries again. Nothing is ever "lost in flight" because the source of truth (the dirty flag) only advances on confirmed success.

Exactly-once via idempotency keys

Here's the scenario that wrecks naive sync: you POST a note, the server saves it, but the response is lost (timeout). Your client thinks it failed and retries — now the server has two notes. Classic double-submit.

The fix is idempotency, and offline-first gives it to us for free:

Because every note has a client-generated UUID (Part 2), the server can treat push as an upsert keyed by that id. A retried push with the same id updates the same row instead of creating a new one. The UUID is the idempotency key. "Create note X" run twice = one note X.

// Server-side contract (conceptual): PUT /notes/{id} is idempotent.
// First call: inserts note with id. Retry: updates the same id. Never duplicates.
await api.pushNote(note); // safe to retry any number of times

This is why we insisted on UUIDs back in Part 2. It wasn't just for offline creation — it's the foundation of at-least-once delivery that's safe because the operation is idempotent, which together give effectively exactly-once. Design your endpoints as idempotent upserts (PUT /notes/{id}), not blind POSTs.

The dirty-during-push race (the subtle bug)

Picture this timeline:

t0  sync reads note (version 5, isDirty=true)
t1  sync sends version 5 to server...
t2  user edits the note → version 6, isDirty=true   ← happens DURING the push
t3  server acks version 5
t4  sync sets isDirty = false   ← BUG: version 6 was never pushed, now marked clean!

If we blindly clear isDirty after the ack, we lose the edit made at t2. The fix is compare-and-clear: only mark clean if the row hasn't changed since we read it.

Future<void> _markCleanIfUnchanged(Note pushed, int serverVersion) async {
  await (db.update(db.notes)
        ..where((n) =>
            n.id.equals(pushed.id) &
            n.version.equals(pushed.version)))   // only if STILL at the pushed version
      .write(NotesCompanion(
        isDirty: const Value(false),
        serverVersion: Value(serverVersion),
      ));
  // If version changed (user edited mid-push), 0 rows update → stays dirty → re-pushed.
}

The principle: capture the version you pushed, and only clear the flag if it's still that version. If the user edited during the push, the version moved, the conditional update affects zero rows, the note stays dirty, and the next sync pushes version 6. No lost edits. (The operation-log design sidesteps this differently — each edit is its own queued op.)

Deleting for real: purging tombstones after ack

Recall deletes are tombstones (Part 2) — isDeleted = true, still dirty. The push sends the deletion; once the server confirms it, the row has done its job and can be physically removed:

if (note.isDeleted) {
  final result = await api.pushNote(note);   // tells server it's deleted
  if (!result.conflict) {
    await (db.delete(db.notes)..where((n) => n.id.equals(note.id))).go(); // purge
  }
}

The tombstone exists only long enough to propagate the deletion. After the server (and eventually other devices, Part 7) know, keeping the empty row wastes space — so we purge it post-ack. Before the ack, never purge: the deletion would be forgotten and the note would resurrect.

Retries and backoff

Transient failures are normal. Two things make retries sane:

1. Leave it dirty — that is the retry mechanism. Every sync re-attempts all dirty rows.
2. Back off so you don't hammer a struggling server. Track attempts per row (or globally) and delay exponentially:

Duration backoff(int attempt) =>
    Duration(seconds: (1 << attempt).clamp(1, 300)); // 1,2,4,8,...,max 5min

Don't retry in a tight loop. A failed sync should wait — guided by backoff and the next connectivity/timer trigger (Part 5, Part 10). And distinguish transient errors (timeout, 503 → retry) from permanent ones (400 bad data, 401 auth → don't blindly retry; surface to the user).

Design B (when you need it): the operation log

For completeness — when order/intent matters, store operations instead of flags:

class Outbox extends Table {
  IntColumn get seq => integer().autoIncrement()();      // strict ordering
  TextColumn get entityId => text()();                   // which note
  TextColumn get op => text()();                          // 'create'|'update'|'delete'
  TextColumn get payload => text()();                     // JSON snapshot
  IntColumn get attempts => integer().withDefault(const Constant(0))();
  DateTimeColumn get createdAt => dateTime()();
}

Push drains it in seq order, deletes each entry on ack, and uses entityId+op for idempotency. Use this when you must replay a precise edit history (collaborative editing, audit logs). For FieldNotes, dirty-flag is plenty.

Whichever design, the contract is identical: a durable record of pending work, drained when online, items removed only on confirmed success, idempotent so retries are safe. That's "an outbox."

Practice Challenges

Challenge 1 — When to mark clean. At what exact moment may a dirty row be marked clean, and why not sooner?

Challenge 2 — Double-submit. The server saves a note but the response times out; the client retries. Why doesn't this create a duplicate?

Challenge 3 — The race. Show why blindly clearing isDirty after an ack can lose an edit, and the fix.

Challenge 4 — Purge timing. When is it safe to physically delete a tombstoned row, and what happens if you purge too early?

Challenge 5 — Transient vs permanent. Why must the push distinguish a 503 from a 400, and how should each be handled?

Questions to test yourself

Q1 (basic). What is an outbox, conceptually?

Q2 (basic). What drives which rows get pushed in the dirty-flag design?

Q3 (intermediate). How do client UUIDs give you exactly-once semantics?

Q4 (intermediate). Describe the dirty-during-push race and the compare-and-clear fix.

Q5 (advanced). Why must a tombstone be purged only after the server acknowledges the delete?

Q6 (advanced). Contrast the dirty-flag and operation-log outbox designs and when to choose each.

Wrapping up

• The outbox is a durable record of pending changes, drained when online, with items cleared only on confirmed server ack.
• Two designs: dirty-flag (push current state of changed rows — our choice) and operation-log (ordered ops — for when sequence/intent matters). Both honor the same contract.
• Client UUIDs as idempotency keys + upsert endpoints turn safe at-least-once retries into effectively exactly-once delivery.
• The dirty-during-push race is solved by compare-and-clear: only mark clean if the row is still at the pushed version.
• Tombstones are purged only after the delete is acknowledged; retries use backoff and distinguish transient from permanent errors.

In Part 7 we build the other half: pulling and merging — fetching only what changed on the server (delta sync) and applying it into the local DB without clobbering un-synced local edits.