Simple IoT project update 2020-10-29

Lots happening lately:

  • switched data structure name from device → node – see
    this issue for dicussion
  • add page to message (currently SMS only) all users
  • UI simplification and cleanup
  • sort users on users page
  • changing backing store from bolthold to genji (this gives us the flexibility
    to use memory, bbolt, or badger backing stores as well as robust indexing)
  • switch to badger as default store as it does not have locking issues (locking issues with bbolt are currently being fixed)

Also exploring a simplified graph data store:

All these changes are taking time, but are laying the foundation for turning this project in a flexible IoT platform.

This week, we took at look at how we can resolve loops in the SIOT data. We store data as nodes and edges, which form a DAG.

DAG

However, there can be cases where users on two different instances make changes to the tree structure which cause a loop to be formed. A loop cannot be allowed, as it will form an infinite loop when rendering the tree, or walking back up the tree when processing new points. When a loop is detected, it must be broken and all instances need to converge to the same solution. To develop an algorithm, I started sketching out various scenarios and a solution emerged which seems fairly simple and robust. This is a complex problem syncing a DAG between distributed systems with very constrained network bandwidth (cellular CAT-M), but with some effort, complex problems can have simple solutions. This type of analysis is great fun – amazing what can happen when you get out some paper and a pencil and start drawing.

This completes our analysis of CRDT properties of the SIOT data structures. It seems we have a fairly solid base for moving forward.