RFC: Riding the JetStream

cbrake · January 26, 2024, 7:24pm

The initial experiments with NATS JetStream have been successful, so working on an architecture document for using it as the SIOT store:

github.com

simpleiot/simpleiot/blob/feature-adr-7/docs/adr/7-jetstream-store.md

# JetStream SIOT Store

- Author: Cliff Brake, last updated: 2024-01-24
- PR/Discussion:
- Status: discussion

## Problem

SQLite has worked well as a SIOT store. There are a few things we would like to
improve:

- synchronization of history
  - currently, if a device or server is offline, only the latest state is transferred when
    connected. We would like all history that has accumulated when offline to be transferred once reconnected.
- we want history at the edge as well as cloud
  - this allows us to use history at the edge to run more advanced algorithms like AI
- we currently have to re-compute hashes all the way to the root node anytime something
  changes
  - this may not scale to larger systems
  - is difficult to get right if things are changing while we re-compute hashes

This file has been truncated. show original

Comments welcome!

cbrake · January 26, 2024, 7:30pm

I’ve been using Excalidraw more. Overall, it does not seem quite as capable as draw.io, but is much faster, and I like the infinite canvas. So I’m always trying to keep a drawing open when I’m working so I always have a canvas ready for ideas. Some recent drawings for this document. Maybe it is my imagination, but it feels like nice drawings help me think better.