Persistence, queues, and events

Neuron code targets the Waher.Persistence.Database abstraction. The configured provider can be the files database, lightweight files database, an external provider, or a test provider; application code should not depend on one storage engine’s files.

Define a persisted model

using Waher.Persistence.Attributes;

[CollectionName("ExampleJobs")]
[Index("State", "Created")]
public sealed class ExampleJob
{
    [ObjectId]
    public string ObjectId { get; set; } = string.Empty;

    public string State { get; set; } = "Pending";
    public DateTime Created { get; set; } = DateTime.UtcNow;

    [DefaultValueNull]
    public string? LastError { get; set; }
}

Index fields in the same order used by high-volume filters and sorts. Changing attributes changes persistence semantics, so test old records, missing properties, and migrations—not only new inserts.

CRUD

await Database.Insert(job);

ExampleJob? pending = await Database.FindFirstIgnoreRest<ExampleJob>(
    new FilterFieldEqualTo(nameof(ExampleJob.State), "Pending"));

if (pending is not null)
{
    pending.State = "Running";
    await Database.Update(pending);
}

Common filter classes include equality, range comparisons, regular expressions, logical FilterAnd/FilterOr, and paging/sort arguments. Use bounded reads. FindFirstDeleteRest deliberately deletes duplicate matches and should only be used when uniqueness and cleanup are intended.

Consistency

The generic object API does not turn an arbitrary read-modify-write sequence into a cross-object transaction. Design with:

unique business identifiers and idempotent commands;
explicit state transitions;
optimistic conflict checks where competing writers are possible;
persisted intent/outbox records before invoking external services;
reconciliation jobs for interrupted work.

Use Waher.Runtime.Transactions only after understanding the participating resources and recovery behavior.

Persistent queues

DatabaseQueue/IPersistedQueue provide durable FIFO work, and the /Queues service exposes authorized HTTP access when installed. Use distinct queue names per contract and environment. A consumer should:

dequeue a bounded batch with a timeout;
validate and deserialize defensively;
apply an idempotency key before side effects;
record success or a retry/dead-letter decision;
expose depth, age, throughput, and failures as counters/events.

Do not use an in-memory Task.Run loop for work that must survive restart.

Structured event log

Use Waher.Events.Log for operational events. Levels range from debug and informational through notice, warning, error, critical, alert, and emergency. Include stable object/actor/facility/event identifiers and structured tags so operators and the Event Log MCP server can search them.

Log.Informational(
    "Example job completed.",
    job.ObjectId,
    "ExampleWorker",
    "Example.Job.Completed");

Never log passwords, bearer tokens, private keys, verification codes, raw identity documents, or unredacted contract attachments. Event sinks can forward logs to files, XMPP, MQTT, queues, statistics, or custom destinations; registering a sink creates a data-egress path that needs its own security review.

Platform overview

Platform concepts

Neuro-Ledger

Extend the Neuron

Neuron Script

Protocols

IoT

Operate the Neuron

Define a persisted model

CRUD

Consistency

Persistent queues

Structured event log

​Define a persisted model

​CRUD

​Consistency

​Persistent queues

​Structured event log

Define a persisted model

CRUD

Consistency

Persistent queues

Structured event log