Parallelspace Corporation Technical Note TN0101
IoT JSON Message Formats: JSON Blow Darts, JSON Arrows, JSON Spears, and JSON Freight Trains.
Parallelspace Confidential Information
Document Version 1.1
April 26, 2016
Michael Herman
Principal Architect
Parallelspace Corporation
mwherman@parallelspace.net
Abstract
This document describes motivation, problem statement, solution options, and recommendations for a layered or tiered approach for the Parallelspace TEAM IoT JSON message format based on a set of documented requirements. The final solution is based on the definition of four (4) message formats: JSON Blow Darts, JSON Arrows, JSON Spears, and JSON Freight Trains.
Introduction |
This document describes motivation, problem statement, solution options, and recommendations for a layered or tiered approach for the Parallelspace TEAM IoT JSON message format based on a set of documented requirements. The final solution is based on the definition of four (4) message formats: JSON Blow Darts, JSON Arrows, JSON Spears, and JSON Freight Trains.
CONTEXT |
The Context section provides background information for the topics covered in the Discussion section that follows immediately after this section.
Definitions
The following list of terms and their definitions are used in the Discussion section.
| Thing | A Thing is a physical or virtual object or entity. Physical Things are usually connected directly or indirectly to the Internet. |
| Event | A Change of State of a Thing |
| Change of State | Change in the number of properties, the name of a property, the attributes of a property, or the value of a property associated with a Thing. |
| Message | Serialization of an Event (i.e. the change in the state of a physical or virtual object or entity). In addition, a Message can be used to capture and communicate the initial and/current state of Things at rest (i.e. Things are haven’t undergone a recent change in state). |
| Event Hub | Local service that Things send Event information to. |
| Event Hub Gateway | Centralized, possibly distributed, usually cloud-based, services that aggregates Messages (Event information) from a Local Event Hub or possibly other Event Hub Gateways or Event Hub Pipelines. |
| Event Hub Pipelines | A pipeline of activities that post-process Messages received by an Event Hub Gateway. |
Additional Resources
The following list of additional resources will be helpful in further understanding the context of the Discussion section.
JSON-Schema standard on GitHub. https://github.com/json-schema/json-schema
DISCUSSION |
Motivation
Many IoT example applications use very simple message formats consisting of a device ID and small number of parameters. This approach is suitable for simple devices sending Change of State information to an Event Hub so that Messages can in turn be forwarded to a central, usually cloud-based Event Hub Gateway.
However, in production solution scenarios, there are additional requirements for an IoT messaging architecture to support additional metadata properties, message filtering, message routing, batching of multiple events into single messages, batching of multiple events into multiple messages, large properties (potentially forcing the creation of messages that are larger than the capabilities of the IoT technology stack, etc.)
Problem Statement
Define a series of IoT JSON message formats, from simple to more complex, that support the diverse range of production requirements.
Solution Options
The available solutions options can be organized around 2 primary axes:
Messaging requirements axis
Single, heterogeneous message format or a multi-level architecture that defines multiple message formats that build upon one and other.
Recommended Solution
The recommended solution incorporates and builds upon the 2 message format axes describe above. The messaging requirements include:
- Serialization of Change of Simple State
- Single Event per Message payload support
- Serialization of Change of Complex State
- Support for External Message Tags (most often defined by the IoT stack)
- Internal Message Properties (separate from the Event state information)
- Multiple Event per Message payload support
- Self-documenting Message formats based on JSON-Schema
- Base64 Encoding/serialization of large Internal Properties
- Multipart Message support for batching multiple Events into multiple Messages
Four levels or tiers of IoT JSON message formats are supported:
- JSON Blow Darts (lightweight)
- JSON Arrows (medium weight)
- JSON Spears (medium-heavy weight)
- JSON Freight Trains (heaviest weight message format the supports all of the listed requirements and multipart messages of unlimited length)
The relationship between the IoT JSON message formats and the capabilities they support are documented in Table 1. IoT JSON Message Format Capabilities.
Table 1. IoT JSON Message Format Capabilities
JSON Blow Darts
JSON Blow Darts are the simplest form of JSON messages consisting of a single, simple, serialized, JSON object with a minimal number of properties and values. For example:
{
“latitude”: 43.0525803,
“longitude”: -76.1546939,
“temperature”: 220,
“units”: “F”
}[1]
JSON Arrows
JSON Arrows include all of the capabilities of a Blow Dart plus optionally adds the following capabilities to a IoT JSON Message:
- Serialization of Change of Complex State
- Support for External Message Tags
The Message may include nested child objects (nested arbitrarily deep) as well as additional message tags or properties attached to the JSON Message as external properties. The latter external tags are often defined by the IoT Event Hub platform (e.g. Microsoft Azure Service Bus Event Hubs). The following is a simple extension of the previous Blow Dart example:
{
“latitude”: 43.0525803,
“longitude”: -76.1546939,
“temperature”: 220,
“units”: “F”,
“location”: {
“name”: “Dinosaur Bar-B-Que”,
“address”: “246 W Willow St, Syracuse, NY 13202, USA”,
“contacts”: {
“telephone”: “+1 315-476-4937”,
“website”: “http://www.dinosaurbarbque.com”
}
}
}
The following is an example of the external tags that may be attached to an Arrow. JSON notation is being used as a convenience. Often the IoT Event Hub platform will use a different mechanism, on-the-wire format and/or transport.
{
“signature”: “http://tempuri.org/Parallelspace/TEAM/”,
“sent”: “2016-04-24T08:48:45.001”
}
JSON Spears
JSON Spears are like Arrows but also provide the additional capabilities:
- Internal Message Properties (separate from the Event state information)
- Multiple Event per Message payload support
Below is an example of some additional properties that might be included as part of the body of an IoT JSON Message.
{
“eventsSource”: {
“timestamp”: “2016-04-24T08:48:45.001”,
“ipv4Address”: “192.168.150.202”,
“ipv6Address”: “fe80::e9d6:3cb2:9fa3:8c2e%28”,
“hostName”: “PSN-WIN8-057”
},
“processingHistory”: [
{
“timestamp”: “2016-04-24T08:48:45.001”,
“hostName”: “PSN-WIN81-001”,
“applicationComponent”: “C:/Windows/System32/WindowsPowerShell/v1.0/powershell.exe”,
“applicationArguments”: [“arg1”, “arg2”]
}
]
}
The next example illustrates internal properties are combined with the capability to support multiple events in a single IoT JSON Message:
{
“payload”: {
“eventsSource”: {
“timestamp”: “2016-04-24T08:48:45.001”,
“ipv4Address”: “192.168.150.202”,
“ipv6Address”: “fe80::e9d6:3cb2:9fa3:8c2e%28”,
“hostName”: “PSN-WIN81-001”
},
“processingHistory”: [
{
“timestamp”: “2016-04-24T08:48:45.001”,
“hostName”: “PSN-WIN81-001”,
“applicationComponent”: “C:/Windows/System32/WindowsPowerShell/v1.0/powershell.exe”,
“applicationArguments”: [“arg1”, “arg2”]
}
],
“events”: [
{
“latitude”: 43.0525803,
“longitude”: -76.1546939,
“temperature”: 220,
“units”: “F”,
“timestamp”: “2016-04-24T08:46:45.014”
},
{
“latitude”: 43.0525803,
“longitude”: -76.1546939,
“temperature”: 219,
“units”: “F”,
“timestamp”: “2016-04-24T08:48:45.015”
}
]
}
}
JSON Freight Trains
You can think of JSON Freight Trains as completely self-documenting and able to represent arrays of events of arbitrary large size. The head of the train includes a reference to the JSON-Schema used to define the overall Freight Train IoT JSON Message format. The box cars and tanker cars that follow the engine are the arrays of events that we’ve seen in JSON Spear messages. When the number of events in the array is too large (e.g. exceeds the capacity of the underlying IoT Event Hub platform), the array can be subdivided and sent as multiple, multipart JSON Freight Train messages. The multipart information is represented and transported as external message properties. Here is an example:
{
“$schema”: “http://tempuri.org/Parallelspace/TEAM/1.0/BBQ.Device/1.0/AcmeBBQ/1.0/Data/JSON-Schema.json”,
“payload”: {
“eventsSource”: {
“timestamp”: “2016-04-24T08:48:45.001”,
“ipv4Address”: “192.168.150.202”,
“ipv6Address”: “fe80::e9d6:3cb2:9fa3:8c2e%28”,
“hostName”: “PSN-WIN81-001”
},
“processingHistory”: [
{
“timestamp”: “2016-04-24T08:48:45.001”,
“hostName”: “PSN-WIN81-001”,
“applicationComponent”: “C:/Windows/System32/WindowsPowerShell/v1.0/powershell.exe”,
“applicationArguments”: [“arg1”, “arg2”]
}
],
“events”: [
{
“latitude”: 43.0525803,
“longitude”: -76.1546939,
“temperature”: 220,
“units”: “F”,
“timestamp”: “2016-04-24T08:46:45.014”
},
{
“latitude”: 43.0525803,
“longitude”: -76.1546939,
“temperature”: 219,
“units”: “F”,
“timestamp”: “2016-04-24T08:48:45.015”
}
]
}
}
The external message properties for a Freight Train IoT JSON message might look like the following.
{
“signature”: “http://tempuri.org/Parallelspace/TEAM/”,
“sent”: “2016-04-24T08:48:45.001”,
“psn.multipart”: {
“enabled”: “true”,
“numberofparts”: 6,
“partnumber”: 1,
“size”: 3425
},
“psn.masterMultipartID”: “ED3B42A2-3196-4937-91EE-0076747F79EB”,
“psn.multipartID”: “28A0C24B-CB84-4A16-8679-01035B844888”
}
Conclusions |
This document described motivation, problem statement, solution options, and recommendations for a layered or tiered approach for the Parallelspace TEAM IoT JSON message format based on a set of documented requirements. The final solution is based on the definition of four (4) message formats: JSON Blow Darts, JSON Arrows, JSON Spears, and JSON Freight Trains.
This document described each of these 4 IoT JSON message formats and provided an example of each format.
NOTICES |
The information contained in this document represents the current view of Parallelspace Corporation on the issues discussed as of the date of publication. Because Parallelspace Corporation must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Parallelspace Corporation, and Parallelspace Corporation cannot guarantee the accuracy of any information presented after the date of publication.
This document is for informational purposes only. Parallelspace Corporation MAKES NO WARRANTIES, EXPRESS, IMPLIED, OR STATUTORY, AS TO THE INFORMATION IN THIS DOCUMENT.
Complying with all applicable copyright laws is the responsibility of the user. Parallelspace Corporation grants you the right to reproduce this document, in whole or in part, specifically and solely for the purpose of personal education.
Parallelspace Corporation may have patents, patent applications, trademarks, copyrights, or other intellectual property rights covering subject matter in this document. Except as expressly provided in any written license agreement from Parallelspace Corporation, the furnishing of this document does not give you any license to these patents, trademarks, copyrights, or other intellectual property.
Unless otherwise noted, the example companies, organizations, products, domain names, e-mail addresses, logos, people, places and events depicted herein are fictitious, and no association with any real company, organization, product, domain name, email address, logo, person, place or event is intended or should be inferred.
© 2001-2016 Parallelspace Corporation. All rights reserved.
Parallelspace Corporation, Truly Collaborative Business Solutions, Parallelspace Total Enterprise Architecture Management and Parallelspace TEAM are either registered trademarks or trademarks of Parallelspace Corporation in the United States, Canada and/or other countries. The names of actual companies and products mentioned herein may be the trademarks of their respective owners
[1] Trivia: Fictitious temperature reading of one of the BBQ smokers at the Dinosaur Bar-B-Que in Syracuse, NY.
Hyperonomy Digital Identity Lab™ - hyperonomy.com 
To elaborate on the JSON Freight Train message metaphor, the Engine at the front of the train is JSON-Schema object. It describes the object type schema for the Cars that follow/are being pulled along by the Engine (actual JSON data objects that conform to the schema). If, with the dynamic environment of IoT, some Cars mid-way along the Train have a few properties that need to be added to or deleted from the schema, you can add another Engine (aka another JSON-Schema message) in front of the first Car that needs/uses the new schema).
MIchael Herman (Toronto)
LikeLike
Michael, this is such a great post. I am tasked with getting pricing estimates from two document databases to store data that we haven’t designed yet. This gives me a set of IoT-focused messages that I can leverage for that work. Bravo!
LikeLike