Selecting a database

There are so many database options today, selecting one can seem overwhelming. This discussion highlights the tradeoffs between relational and graph databases:

https://news.ycombinator.com/item?id=28736405

One idea that seems to come up quite a bit is that graph databases are easier to optimize because the data is forced into a simpler model.

Another article advocating GraphQL and graph databases:

Similar to how big tech moved with the NoSQL movement, well-known tech innovators are at the fore of the GraphQL wave. Following Facebook’s initiation of the GraphQL movement, Netflix, Twitter, Airbnb, PayPal, Shopify, and many more tech companies have jumped on board. The wave hasn’t stopped with the tech industry - companies like Starbucks, the New York Times, NBC, Fairfax, KLM airlines, Intuit, and Yelp are converting their tech stacks to GraphQL in response to business needs. Even companies that service the tech industry like GitHub and Atlassian have moved to GraphQL. The GraphQL Foundation lists many GraphQL adopters, and the list is growing daily.

It’s easy to see why GraphQL awareness and adoption are growing when companies like Airbnb claim a 10x development improvement. Mark Stuart who leads Web Platform at PayPal calls it a game-changer. He remarked, “At PayPal, GraphQL has been a complete game-changer to the way we think about data, fetch data and build applications”.

This looks like an interesting project to create a GraphQL API from a Postgres database:

https://www.graphile.org/

Project to measure database performance for IoT use cases:

This came out of the industry 4.0 Discord forum.

Again, it looks like PostgreSQL is a solid performer.

Interesting database:

Dolt supports most git like operations, where you can clone remote databases, branch data, commit, push, etc. All operations are also versioned so you can see history, roll back, etc. Dolt also has a MySQL compatible interface. Written in #go. Likely not the fastest database out there, but would be really interesting for workflows.

Interesting article and discussion:

https://blog.sesse.net/blog/tech/2021-12-05-16-41_leaving_mysql.html

https://news.ycombinator.com/item?id=29455852

As with most things, the answer is – it depends – there are always tradeoffs to consider.

Interesting presentation by Ben Johnson on using SQLite in Go:

This is very interesting as Ben wrote the popular BoltDB embedded Go database, but since then it appears he favors SQLite.

In the presentation, Ben discusses durability:

Ben also wrote a tool that replicates a SQLite database to S3 in real-time.

This is all very interesting as I’d like a better database solution in Simple IoT. Genji has done pretty well, but the Db file format is still changing every release, so that makes SIOT upgrades more painful. I’m also realizing that we’ll need to cache quite a bit in SIOT to get the performance we need, so the database is less important. Anyway, it seems SQLite could be a good solution that will scale from embedded edge systems to server/cloud systems with real-time backups.

Interesting article:

https://news.ycombinator.com/item?id=30013919

It appears that the issue was caused by an edge case in boltdb, and fixed in the newer fork bbolt.

The boltdb author responds here:

https://news.ycombinator.com/item?id=30015913

As Ben appears to have moved on to working with SQLite, probably something we can learn here – databases are not trivial …

A detailed article on SQLite:

https://blog.wesleyac.com/posts/consider-sqlite

A good discussion about selecting a database and various tradeoffs. It appears the weakness of sqlite is high concurrent write load, but otherwise has been successful in many production apps.