Doing some load testing in SIOT by pumping 15KHz data through it – we have some work to do as we have too many NATS clients subscribing to all the data flowing through the system and we start getting NATS slow consumer messages.
It is interesting to view how much CPU is being used by a heavily loaded Go app. Below is a top snapshot. The cumulative processor usage is around 220%. Appears we have lots of threads using parts of a CPU. May not be the most efficient, but at least it’s highly concurrent at this point.
Decided that we need to bypass most of the normal stuff in SIOT for high-rate data and create some new NATs subjects specially for high rate data. Once I did that, 15KHz data started flowing through just fine. In viewing the data, learned that Grafana and the Influx UI’s only support ms time resolution:
There are historical reasons for keeping JS Dates at ms. precision, including Spectre-like timing attacks. Modern browsers block use of high-precision timers unless cross-origin security policies are more strict.
All of that said, I’m sure it’s possible to hack around this issue somehow.
