There are large Data Drop-Outs in a production Timeseries I work with, sometimes for more than a month, and then it comes back online. I am concerned that we are overwhelming the system at some point and creating a bottleneck.
I have several deployed edge devices running predix machine and pushing data to the same Time series instance. Each Predix Machine is connected to several sensors that are producing data at a rate in the millisecond range. Today, we send all raw sensor data to Timeseries from 15 instances of Predix Machine.
The data drop-outs tend to reflect one or more entire edge device at a time.
In total, Timeseries receives roughly 130 Mb/sec (conservative estimate) from all the PM instances.
Is it possible that there is a bottleneck at Timeseries or Predix Machine?
The best way to know is to check the Predix Machine log while it's happening, it shows that it's having trouble connecting and is trying to (re)send the data.
I suppose if the network is unavailable for long enough you could exceed a buffer. Also, I believe the persistence layer in Predix Machine would need to be on to ensure no data loss.
Once the data is received in the cloud, it should be guaranteed delivery. So, it's probably on the machine side due to network.
Thanks Tom, this helps narrow our troubleshooting to the edge and network!
