Aquila.AqlStore

About AqlStore

Aquila.AqlStore is a lightweight, persistent key-value store built for .NET applications that need fast reads, efficient updates, and reliable data persistence across restarts. It uses a snapshot + append-only log model, making it ideal for configuration, metadata, feature flags, user preferences, small caches, and runtime state.

Key advantages: O(1) lookups, append-only writes (no full rewrites), auto-compaction, prefix queries, thread-safety, and robust recovery.

How to Use AqlStore – Step by Step

1. Create / Open the Store

1. Choose a persistent directory (e.g. C:\MyApp\Data or Documents folder).
2. Choose a unique store name (e.g. "settings", "cache", "users").
3. Optionally set auto-compact threshold (default: 2000 writes).

using Aquila;
var store = new AqlStore("C:/MyApp/Data", "settings", 2000);

2. Store or Update Data

1. Use Set(key, value) — creates or overwrites the value.
2. Keys should be unique strings (e.g. "user.name", "app.version").
3. Values are strings — serialize complex objects if needed.

store.Set("user.name", "Alice");
store.Set("app.version", "2.3.1");

3. Retrieve Data

1. Use Get(key) — returns value or null if not found.
2. Use GetKeysByPrefix(prefix) to list related keys.

string? name = store.Get("user.name"); // "Alice"
var userKeys = store.GetKeysByPrefix("user.");

4. Update Data

1. Just call Set() again — it's an upsert operation.

store.Set("user.age", "30"); // creates or updates

5. Delete Data

1. Use Delete(key) — removes the key permanently.
2. Deletion is persisted after compaction or restart.

store.Delete("user.age");

6. Force Compaction & Cleanup

1. Call Compact() to write snapshot and clear log.
2. Use using block for auto-cleanup on dispose.

store.Compact();
using var store = new AqlStore("C:/MyApp/Data", "settings");
// ... work ...
// auto-compact on dispose

7. Check Count & Keys

1. Count property — current number of keys.
2. GetKeysByPrefix("") — lists all keys.

int total = store.Count; // e.g. 42
var allKeys = store.GetKeysByPrefix("");

All Public Methods & Properties

Examples – From Simple to Advanced

store.Set("user.name", "Bob");
store.Set("user.email", "bob@example.com");
string? email = store.Get("user.email"); // "bob@example.com"

store.Set("app.apiKey", "xyz789");
store.Set("app.debug", "false");
bool debug = store.Get("app.debug") == "true"; // false

// Save session
store.Set("session.userId", "12345");
store.Set("session.lastLogin", DateTime.Now.ToString("o"));
// Check session
string? userId = store.Get("session.userId");
// Clean old session
store.Delete("session.lastLogin");

// Settings with prefix
store.Set("theme.primary", "#6366f1");
store.Set("theme.secondary", "#10b981");
// List theme keys
var themeKeys = store.GetKeysByPrefix("theme.");
foreach (var key in themeKeys)
    Console.WriteLine($"{key} → {store.Get(key)}");

// Import 10,000 items (auto-compact triggers during loop)
for (int i = 1; i <= 10000; i++)
{
    store.Set($"item.{i}.title", "Item " + i);
    store.Set($"item.{i}.value", (i * 10).ToString());
}
// Fast prefix scan after bulk
var items = store.GetKeysByPrefix("item.");
Console.WriteLine($"Imported {items.Count()} items");

// Background sync task
async Task SyncDataAsync()
{
    // Fetch updates...
    store.Set("sync.lastTime", DateTime.UtcNow.ToString("o"));
    // Clean old entries
    var oldKeys = store.GetKeysByPrefix("temp.");
    foreach (var key in oldKeys)
        store.Delete(key);
}
// On app resume – fast re-read
string? lastSync = store.Get("sync.lastTime");

AQL vs JSON: Structure, Usage, and Performance

Now we’re getting to the core comparison — AQL vs JSON. Let’s break it down carefully in terms of structure, usage, and performance.

1️⃣ JSON (JavaScript Object Notation)

Structure

• Hierarchical, tree-based format
• Supports:
  • Objects: { "name": "Arun", "age": 25 }
  • Arrays: [1, 2, 3]
  • Primitives: string, number, boolean, null

Example

[
  { "id": 1, "name": "Arun", "age": 25, "city": "Coimbatore" },
  { "id": 2, "name": "Aruna", "age": 22, "city": "Neyveli" }
]

Characteristics

• Standardized: widely used across platforms
• Readable: humans can read/write
• Serialization: objects can be serialized/deserialized directly
• Random access: easy if loaded into memory
• No built-in persistence: JSON itself is just text; saving/loading is handled by your app
• Best for structured hierarchical data (nested objects/arrays)

2️⃣ AQL (as implemented in AqlStore)

Structure

• Key-value store, flat structure
• Keys = string, Values = string
• Supports prefix-based queries but not hierarchical objects inherently
• Can simulate objects by encoding values (e.g., JSON string inside AQL)

Example (your store)

@aql 1.0
username = "Arun"
age = "25"
city = "Coimbatore"
username2 = "Aruna"
age2 = "22"
city2 = "Neyveli"

Characteristics

• Persistent by default: snapshot + append-only log
• Thread-safe: concurrent access in same process
• Auto-compaction: removes deleted keys and old logs
• Flat storage: all keys at top level
• Single-process: not suitable for multi-process or distributed systems
• Flexible values: can store any string (including serialized JSON if needed)

3️⃣ Key Differences

4️⃣ Analogy

Think of it like this:

• JSON → Excel spreadsheet: you save the whole sheet, read and write big chunks, nested cells possible
• AQL → Key-value index card cabinet: each card = one key-value pair, you can quickly change a card without touching the others, automatically keeps track of changes

var json = JsonSerializer.Serialize(new { id = 1, name = "Arun", age = 25 });
aqlStore.Set("person_1", json);

AQL vs JSON for 50,000 Records

Perfect — let’s do a careful comparison of JSON vs AQL (AqlStore) for 50,000 records. I’ll break it down step by step, including storage behavior, memory, and update performance.

1️⃣ Scenario

We have 50,000 “person” records, each like:

• id: 1
• name: "Arun"
• age: 25
• city: "Coimbatore"

Option A — JSON Storage

Structure

[
  { "id": 1, "name": "Arun", "age": 25, "city": "Coimbatore" },
  { "id": 2, "name": "Aruna", "age": 22, "city": "Neyveli" },
  ...
  { "id": 50000, "name": "Name50000", "age": 30, "city": "City50000" }
]

Entire dataset is one JSON array
Saved in a single file: people.json

Pros

• Human-readable, standard format
• Works well if you read/write the entire dataset at once

Cons

• Updates are expensive: changing one record requires:
  1. Loading entire 50,000 records into memory
  2. Updating record
  3. Writing entire JSON file back (~MBs of data)
• Memory usage: must load all objects at once (~50,000 × size of object)
• Multi-threaded writes are tricky
• Append-only updates not efficient

Option B — AQL (AqlStore)

Structure

person_1 = "{"id":1,"name":"Arun","age":25,"city":"Coimbatore"}"
person_2 = "{"id":2,"name":"Aruna","age":22,"city":"Neyveli"}"
...
person_50000 = "{"id":50000,"name":"Name50000","age":30,"city":"City50000"}"

Stored in mydata.aql + mydata.aql.log
Flat key-value storage

Pros

• Update one record instantly:

  aqlStore.Set("person_1234", jsonString);

  → Only updates that key, no rewriting the full dataset
• Append-only log + snapshot ensures durability
• Memory efficient: you can load all keys, or just fetch one by key
• Thread-safe for same-process writes
• Auto-compaction cleans old/deleted keys

Cons

• Flat structure: querying multiple fields requires code
• Not human-readable like JSON unless you inspect the snapshot manually
• Multi-process writes not supported

2️⃣ Performance & Storage Comparison (Estimated)

Observation: ✅
For 50,000+ records, AQL is faster for per-record operations (update/add/delete)
JSON is better if you always load the entire dataset at once and rarely update individual records

3️⃣ When to use each

4️⃣ Practical Example for 50k records in AQL

var store = new AqlStore("C:\AqlData", "people");
for (int i = 1; i <= 50000; i++)
{
    var person = $"{{"id":{i},"name":"Name{i}","age":{20+i%30},"city":"City{i}"}}";
    store.Set($"person_{i}", person);
}
// Update a single record quickly
store.Set("person_12345", "{"id":12345,"name":"Updated","age":30,"city":"NewCity"}");
// Delete a record
store.Delete("person_54321");
// Retrieve a record
var json = store.Get("person_100");

✅ Efficient even for 50k+ entries
✅ Only updates the relevant key
✅ Snapshot + log ensures persistence

Real Processing Speed Comparison (Honest)

These are typical real-world observations (50k rows, ~30 MB file), not fake benchmarks.