Orbit Commerce Split-Tender Cashiering & Offline Resilience Incident Guide

1. Emergency Operational Posture & Incident Classification Matrix

Point-of-sale cash registers operate under intense operational pressure during peak retail trading hours. Network partitions, internet service provider outages, and cloud gateway latency can paralyze checkout queues, resulting in lost sales, abandoned shopping baskets, and customer dissatisfaction.

1.1 Incident Severity Classification Matrix

2. Multi-Rail Split-Tender Cashiering Architecture

The cashiering subsystem allows customers to settle sales checks across diverse payment rails, decomposing a single monetary liability into discrete payment legs:

• Cash: Physical currency notes and coins. Includes change calculation and cash drawer kick actuation.
• Credit / Debit Card: Integrated or standalone bank card terminal.
• Mobile Digital Wallets: QR and push rails (JazzCash, EasyPaisa, Online Transfers).
• Customer Store Credit (Khata): Informal house account credit governed by customer khata debt ledgers and fifo aging.
• Store Vouchers & Gift Cards: Electronic gift certificates and prepaid store credit cards.
• Customer Loyalty Points: Accumulated rewards converted into currency discounts.

2.1 Inviolable Split-Tender Business Rules

The modular backend application server enforces strict validation rules upon checkout submission:

1. Exact Mathematical Balance: $$\sum_^N \text_i \equiv \text \pm 0.01$$ Any discrepancy exceeding 0.01 currency units throws a 400 Bad Request exception, halting transaction commitment.
2. Single Cash Leg Invariant: Split checkouts allow at most one Cash payment leg. Permitting multiple cash legs introduces cash drawer accounting ambiguities and is rejected at the domain validation boundary.
3. Single Khata Leg Invariant: Split sales permit at most one Customer Khata leg. Charging multiple customer accounts on a single sales ticket is prohibited.
4. Prohibited Payment Rails: Vertical-specific healthcare rails (such as Sehat cards or panel health claims) are restricted to diagnostic billing and are strictly blocked from settling retail commerce sales.

3. Air-Gapped Local Operation & Offline Queue Durability Protocol

When wide-area network connectivity fails, point-of-sale registers must not freeze or display blocking error modals. Instead, the dynamic client view engine transitions into autonomous air-gapped execution.

3.1 Local Offline Receipt Generation

When operating offline, the terminal generates an offline sales document containing complete line item snapshots, tax computations, and split payment legs:

• Offline Receipt Number (offlineReceiptNo): Synthesized using the format OFF-YYYYMMDD-XXXX, where XXXX represents a high-resolution monotonic sequence counter.
• Offline Creation Timestamp (offlineCreatedAt): Captures the exact physical timestamp of the sale in ISO-8601 UTC.
• Offline Electronic Reference Enforcement: For card or mobile wallet legs executed offline, the cashier must enter the physical terminal approval code into the reference input field. The system blocks offline completion of electronic rails if the reference code is blank.

3.2 Local Encrypted Outbox Persistence

The offline sales document is serialized and written to an append-only local transactional store in client storage before the transaction is acknowledged to the cashier. This write-ahead persistence ensures that power interruptions, terminal battery depletion, or accidental browser reloads cannot destroy un-synchronized sales records.

3.3 Raw ESC/POS Receipt Printing During Partitions

Customer receipts and kitchen chits print directly over local communication channels (such as USB human interface device ports or local raw TCP sockets on port 9100). The print generation engine builds thermal raster buffers locally, embedding the offline receipt number and generating local fiscal QR matrices as detailed in digital receipt and fiscal qr.

3.4 Encrypted Monotonic Clock Watermarking & Rollback Detection

When operating offline without access to centralized Network Time Protocol (NTP) daemons, rogue operators might attempt to manipulate operating system system clocks to backdate sales or tamper with shift closure records:

1. Monotonic Watermark Persistence: The native systems crate continuously samples CPU monotonic hardware counters alongside the real-time clock. Every second, an encrypted watermark timestamp is recorded to local non-volatile storage.
2. Clock Rollback Intercept: If the terminal initializes or records a transaction with a system timestamp earlier than the persistent watermark (with a 300-second grace tolerance for legitimate NTP adjustments), the security core triggers a tamper lockout.
3. Emergency Override Protocol: A clock tamper lockout freezes cashier checkout until a Tier 3 Store Manager inputs an authorized administrative recovery PIN, logging a high-priority security audit event.

4. Reconnection Synchronization & Idempotent Backlog Replay Protocol

When wide-area network connectivity is restored, the terminal's background synchronization daemon initiates a controlled, idempotent backlog replay sequence.

4.1 Half-Open Circuit Breaker Probing

To prevent "thundering herd" gateway overloads upon network restoration:

1. The network monitor probes the health endpoint with 3 consecutive ping requests spaced 2,000 milliseconds apart.
2. Only after all three probes return HTTP 200 does the circuit breaker transition from OPEN to HALF_OPEN.
3. The sync worker dispatches transactions in throttled batches of 5 documents, verifying latency before increasing throughput.

4.2 Idempotency Deduplication Logic

Network glitches during replay can cause a client to re-send an offline ticket that was already committed by the server but unconfirmed due to a dropped socket.

The central ingestion kernel inspects the incoming offlineReceiptNo:

SELECT id, "invoiceNumber", status FROM invoices
WHERE "tenantId" = :tenantId
  AND "offlineReceiptNo" = :offlineReceiptNo;

If an invoice record is matched, the server returns the existing invoice payload with HTTP 200 OK without re-deducting inventory or duplicating general ledger entries. The client safely purges the record from its local outbox.

5. 409 Conflict Handling, Ghost Item Deletions & Quarantine Isolation

During an extended network partition, back-office staff on the central system might delete a catalog item, reassign a SKU, or mark inventory out-of-stock while cashiers on the offline register continue scanning and selling the item.

5.1 Non-Blocking Quarantine Isolation

In traditional point-of-sale systems, a single synchronization error causes the entire sync queue to halt (head-of-line blocking), freezing subsequent valid sales.

Orbit Commerce prevents head-of-line blocking through quarantine isolation:

1. If the central ingestion engine throws an unrecoverable validation error (such as an unknown item ID), the worker catches the rejection.
2. The complete payload, error trace, and cashier metadata are written to the quarantined_sales table.
3. The sync worker purges the item from the local device outbox and advances immediately to the next queued transaction.
4. The management console alerts store supervisors to review quarantined tickets for manual inventory reconciliation.

5.2 Automated Shift Replay Reconciliation & Manager Sign-Off

When an entire cashier shift is executed during a network partition, closing the register shift requires an automated reconciliation sequence upon network restoration:

1. Outbox Drain Verification: The shift closure gate blocks final Z-report generation until the local outbox queue depth reaches zero, ensuring all pending tickets are either committed or quarantined.
2. Replayed Tender Audit Matrix: The system contrasts offline tenders with committed bank slips: total cash counted during blind drop must balance against replayed cash legs, and physical credit card settlement rolls must match replayed card reference totals.
3. Supervisor Shift Certification: The store manager enters their administrative PIN to certify the shift report. Any quarantined sales are flagged on the cashier's shift summary report, preventing till count discrepancies from penalizing frontline operators.

6. Failure Modes, Root Cause Analysis & Operational Recovery Matrix

The following operational matrix guides systems administrators and lead cashiers through resolving cashiering exceptions and outbox sync failures:

7. Cross-Functional Platform Architecture Dependencies

Split-tender cashiering and offline synchronization operate in concert with platform modules across the store:

• Customer Credit Accounts: Split legs tendered to store credit update ledgers detailed in customer khata debt ledgers and fifo aging.
• Physical Barcode Scanners: Item entry during rapid offline checkout relies on scanner hardware rules in ean13 scale barcodes and scanner wedge processing.
• Physical Inventory Auditing: Reconciling variances resulting from quarantined sales is governed by physical stock take broadsheet and fefo reconciliation.
• Fiscal Compliance & Tax: Replayed offline invoices generate compliant tax records in accordance with zatca phase 2 e-invoicing and cryptographic clearance.
• Hospitality Offline Routing: Food service offline failover procedures align with offline mode and local queue.

8. Critical Outage Triage FAQs & Rapid Incident Resolution

What happens if an edge terminal battery completely drains with 30 un-synced offline orders in local queue?

Because orders are written to an encrypted append-only write-ahead log in persistent client storage before receipts are printed, data remains completely safe. Connecting the terminal to AC power and booting the application immediately resumes the synchronization worker without data loss.

Can an operator settle a bill using both Cash and Customer Khata?

Yes. The cashier allocates a portion of the bill to Cash (e.g. SAR 1,500.00) and the remaining portion to Khata (e.g. SAR 850.00). The system validates that the customer has sufficient credit limit and generates balanced ledger vouchers.

How does the system prevent cashiers from manipulating sales timestamps while offline?

The client application uses an internal monotonic clock watermark that tracks elapsed time since terminal boot. Any attempt to alter the operating system system clock backwards triggers an anti-tamper security fault, freezing offline operations until supervisor authentication is provided.

Are physical cash drops permitted while the store is operating in offline mode?

Yes. Cashiers can perform blind drops and mid-shift counts. The terminal logs the drop to the local outbox and prints an 80mm audit chit. The drop vouchers commit to the general ledger upon network reconnection.