I would like to know what standard(s) may exist for formatting log data from sensors that record information such as:
- acceleration vector
- magnetic field vector
- location vector (ex: GPS)
I have been an end-user of SCADA software products which record various data fields for each stream of log data it receives into a MySQL database. However, I have never looked into determining how exactly that data was stored. I would like to know if there exists an industry consensus standard for storing sensor data that facilitates information interchange. For example, the Internet Engineering Task Force (IETF) publishes documents called "RFCs" for the purpose of standardizing implementations of commonly used protocols like IMAP (email), TLS (secure web browsing).
For example, such a standard might indicate to me the best practices for:
- formatting units of measurement (ex:
- formatting of timestamps of each measurement (ex: ISO-8601)
- file format type (CSV files vs. SQL database)
- delimiters for measurement data (ex:
- indicating origin and license of data (ex: end device owner/operator, equipment tag number, CC RDFa tags)
- encrypting measurements (esp. in a way compatible with sudden power-loss, ex: DTLS)
While researching this question I noticed that the IETF says the Internet of Things (IoT) is a topic of study for several of its working groups. Its definition of IoT is
The Internet of Things is the network of physical objects or "things" embedded with electronics, software, sensors, actuators, and connectivity to enable objects to exchange data with the manufacturer, operator and/or other connected devices.
However, I did not find any IETF standards document describing standard methods for storing log data. Is there an ISA standard covering methods for logging data? I really like their ISA 5.1 standard for P&ID symbols, "Instrumentation Symbols and Identification".
Thank you for any direction you may provide.
After several months of off-and-on research, the closest standard that defines what I was looking for is the Semantic Sensor Network (SSN) Ontology published by the W3C. It is an expanded version of the Sensor, Observation, Sample, and Actuator (SOSA) Ontology. It is capable of associating metadata about an observation (i.e. units, time, location, site-specific names, relationships to other entities in a data collection hierarchy) in an RDF graph data structure. RDF graphs can be "serialized" into JSON (specifically, JSON-LD). The JSON graph data can be formatted using the JSON lines specification so each observation can be appended to a newline-delimited log file in a streaming fashion. Because the logs are machine-readable, an appropriate software package could later import the readings into a database or whatever a user requires.
Search results aren't clear whether SSN will become an industry-standard for general encoding of sensor information but it seems to me the best method for storing sensor data in a self-documenting way. For now, I'm happy to have discovered the method of recording measurements + metadata as an RDF graph using the SSN ontology, serializing the graph as as JSON-LD, and formatting the JSON-LD as JSON Lines that are streamed and appended to a continuously growing log file.