Monthly Archives: April 2016
Microsoft Azure Stack POC Architecture Reference Model (ARM): ArchiMate Model – version 1-0-7 – April 30, 2016
[Updated March 3, 2017]
PLEASE POST A COMMENT ABOUT WHY THIS PAGE IS IMPORTANT TO YOU.
This particular page is 1 of my top 5 most viewed pages (ever) and I’d like to understand why. Thank you!
Figure 1. Parallelspace Logical/Physical Architecture View: Microsoft Azure Stack POC (April 2016)
[Click here for a larger version of the ArchiMate model]
Notes
- The actual drive letters will vary from system to system. Don’t fret these details.
- I’ll keep adding more detail to the model as I work through the full deployment of the Microsoft Azure Stack POC.
The above ArchiMate enterprise architecture model was created with Archi 3.2 – The Free ArchiMate Modeling Tool. Download the latest version of Archi from here.
Here’s what the original Microsoft drawing (a Visio sketch – not a model) looks like in April 2016 (from https://azure.microsoft.com/en-us/documentation/articles/azure-stack-architecture/):
Figure 2. Microsoft Azure Conceptual Architecture View: Microsoft Azure Stack POC (April 2016)
[Click here for a larger version of the Microsoft drawing.] It’s mostly useless but typical of what you’d expect in a Microsoft marketecture diagram.
Microsoft has subsequently updated their conceptual architecture diagram (March 1, 2017). It now looks like this (at the same URL noted above). The new diagram is an improvement and I can’t help but imagine it was influenced by my ArchiMate model.
Figure 3. Microsoft Azure Architecture View: Microsoft Azure Stack POC (March 2017)
For a topic that in theory has a relatively narrow audience, this article has had an extraordinary number of views over the past year.
Best regards,
Michael Herman (Toronto)
mwherman@parallelspace.net
p.s. I can only assume it is Microsofties trying to learn a little bit more about enterprise architecture. You can see the (good) results in Figure 3 (above).
IoT JSON Message Formats: JSON Blow Darts, JSON Arrows, JSON Spears, and JSON Freight Trains
Parallelspace Corporation Technical Note TN0101
IoT JSON Message Formats: JSON Blow Darts, JSON Arrows, JSON Spears, and JSON Freight Trains.
Parallelspace Confidential Information
Document Version 1.1
April 26, 2016
Michael Herman
Principal Architect
Parallelspace Corporation
mwherman@parallelspace.net
Abstract
This document describes motivation, problem statement, solution options, and recommendations for a layered or tiered approach for the Parallelspace TEAM IoT JSON message format based on a set of documented requirements. The final solution is based on the definition of four (4) message formats: JSON Blow Darts, JSON Arrows, JSON Spears, and JSON Freight Trains.
Introduction |
This document describes motivation, problem statement, solution options, and recommendations for a layered or tiered approach for the Parallelspace TEAM IoT JSON message format based on a set of documented requirements. The final solution is based on the definition of four (4) message formats: JSON Blow Darts, JSON Arrows, JSON Spears, and JSON Freight Trains.
CONTEXT |
The Context section provides background information for the topics covered in the Discussion section that follows immediately after this section.
Definitions
The following list of terms and their definitions are used in the Discussion section.
| Thing | A Thing is a physical or virtual object or entity. Physical Things are usually connected directly or indirectly to the Internet. |
| Event | A Change of State of a Thing |
| Change of State | Change in the number of properties, the name of a property, the attributes of a property, or the value of a property associated with a Thing. |
| Message | Serialization of an Event (i.e. the change in the state of a physical or virtual object or entity). In addition, a Message can be used to capture and communicate the initial and/current state of Things at rest (i.e. Things are haven’t undergone a recent change in state). |
| Event Hub | Local service that Things send Event information to. |
| Event Hub Gateway | Centralized, possibly distributed, usually cloud-based, services that aggregates Messages (Event information) from a Local Event Hub or possibly other Event Hub Gateways or Event Hub Pipelines. |
| Event Hub Pipelines | A pipeline of activities that post-process Messages received by an Event Hub Gateway. |
Additional Resources
The following list of additional resources will be helpful in further understanding the context of the Discussion section.
JSON-Schema standard on GitHub. https://github.com/json-schema/json-schema
DISCUSSION |
Motivation
Many IoT example applications use very simple message formats consisting of a device ID and small number of parameters. This approach is suitable for simple devices sending Change of State information to an Event Hub so that Messages can in turn be forwarded to a central, usually cloud-based Event Hub Gateway.
However, in production solution scenarios, there are additional requirements for an IoT messaging architecture to support additional metadata properties, message filtering, message routing, batching of multiple events into single messages, batching of multiple events into multiple messages, large properties (potentially forcing the creation of messages that are larger than the capabilities of the IoT technology stack, etc.)
Problem Statement
Define a series of IoT JSON message formats, from simple to more complex, that support the diverse range of production requirements.
Solution Options
The available solutions options can be organized around 2 primary axes:
Messaging requirements axis
Single, heterogeneous message format or a multi-level architecture that defines multiple message formats that build upon one and other.
Recommended Solution
The recommended solution incorporates and builds upon the 2 message format axes describe above. The messaging requirements include:
- Serialization of Change of Simple State
- Single Event per Message payload support
- Serialization of Change of Complex State
- Support for External Message Tags (most often defined by the IoT stack)
- Internal Message Properties (separate from the Event state information)
- Multiple Event per Message payload support
- Self-documenting Message formats based on JSON-Schema
- Base64 Encoding/serialization of large Internal Properties
- Multipart Message support for batching multiple Events into multiple Messages
Four levels or tiers of IoT JSON message formats are supported:
- JSON Blow Darts (lightweight)
- JSON Arrows (medium weight)
- JSON Spears (medium-heavy weight)
- JSON Freight Trains (heaviest weight message format the supports all of the listed requirements and multipart messages of unlimited length)
The relationship between the IoT JSON message formats and the capabilities they support are documented in Table 1. IoT JSON Message Format Capabilities.
Table 1. IoT JSON Message Format Capabilities
JSON Blow Darts
JSON Blow Darts are the simplest form of JSON messages consisting of a single, simple, serialized, JSON object with a minimal number of properties and values. For example:
{
“latitude”: 43.0525803,
“longitude”: -76.1546939,
“temperature”: 220,
“units”: “F”
}[1]
JSON Arrows
JSON Arrows include all of the capabilities of a Blow Dart plus optionally adds the following capabilities to a IoT JSON Message:
- Serialization of Change of Complex State
- Support for External Message Tags
The Message may include nested child objects (nested arbitrarily deep) as well as additional message tags or properties attached to the JSON Message as external properties. The latter external tags are often defined by the IoT Event Hub platform (e.g. Microsoft Azure Service Bus Event Hubs). The following is a simple extension of the previous Blow Dart example:
{
“latitude”: 43.0525803,
“longitude”: -76.1546939,
“temperature”: 220,
“units”: “F”,
“location”: {
“name”: “Dinosaur Bar-B-Que”,
“address”: “246 W Willow St, Syracuse, NY 13202, USA”,
“contacts”: {
“telephone”: “+1 315-476-4937”,
“website”: “http://www.dinosaurbarbque.com”
}
}
}
The following is an example of the external tags that may be attached to an Arrow. JSON notation is being used as a convenience. Often the IoT Event Hub platform will use a different mechanism, on-the-wire format and/or transport.
{
“signature”: “http://tempuri.org/Parallelspace/TEAM/”,
“sent”: “2016-04-24T08:48:45.001”
}
JSON Spears
JSON Spears are like Arrows but also provide the additional capabilities:
- Internal Message Properties (separate from the Event state information)
- Multiple Event per Message payload support
Below is an example of some additional properties that might be included as part of the body of an IoT JSON Message.
{
“eventsSource”: {
“timestamp”: “2016-04-24T08:48:45.001”,
“ipv4Address”: “192.168.150.202”,
“ipv6Address”: “fe80::e9d6:3cb2:9fa3:8c2e%28”,
“hostName”: “PSN-WIN8-057”
},
“processingHistory”: [
{
“timestamp”: “2016-04-24T08:48:45.001”,
“hostName”: “PSN-WIN81-001”,
“applicationComponent”: “C:/Windows/System32/WindowsPowerShell/v1.0/powershell.exe”,
“applicationArguments”: [“arg1”, “arg2”]
}
]
}
The next example illustrates internal properties are combined with the capability to support multiple events in a single IoT JSON Message:
{
“payload”: {
“eventsSource”: {
“timestamp”: “2016-04-24T08:48:45.001”,
“ipv4Address”: “192.168.150.202”,
“ipv6Address”: “fe80::e9d6:3cb2:9fa3:8c2e%28”,
“hostName”: “PSN-WIN81-001”
},
“processingHistory”: [
{
“timestamp”: “2016-04-24T08:48:45.001”,
“hostName”: “PSN-WIN81-001”,
“applicationComponent”: “C:/Windows/System32/WindowsPowerShell/v1.0/powershell.exe”,
“applicationArguments”: [“arg1”, “arg2”]
}
],
“events”: [
{
“latitude”: 43.0525803,
“longitude”: -76.1546939,
“temperature”: 220,
“units”: “F”,
“timestamp”: “2016-04-24T08:46:45.014”
},
{
“latitude”: 43.0525803,
“longitude”: -76.1546939,
“temperature”: 219,
“units”: “F”,
“timestamp”: “2016-04-24T08:48:45.015”
}
]
}
}
JSON Freight Trains
You can think of JSON Freight Trains as completely self-documenting and able to represent arrays of events of arbitrary large size. The head of the train includes a reference to the JSON-Schema used to define the overall Freight Train IoT JSON Message format. The box cars and tanker cars that follow the engine are the arrays of events that we’ve seen in JSON Spear messages. When the number of events in the array is too large (e.g. exceeds the capacity of the underlying IoT Event Hub platform), the array can be subdivided and sent as multiple, multipart JSON Freight Train messages. The multipart information is represented and transported as external message properties. Here is an example:
{
“$schema”: “http://tempuri.org/Parallelspace/TEAM/1.0/BBQ.Device/1.0/AcmeBBQ/1.0/Data/JSON-Schema.json”,
“payload”: {
“eventsSource”: {
“timestamp”: “2016-04-24T08:48:45.001”,
“ipv4Address”: “192.168.150.202”,
“ipv6Address”: “fe80::e9d6:3cb2:9fa3:8c2e%28”,
“hostName”: “PSN-WIN81-001”
},
“processingHistory”: [
{
“timestamp”: “2016-04-24T08:48:45.001”,
“hostName”: “PSN-WIN81-001”,
“applicationComponent”: “C:/Windows/System32/WindowsPowerShell/v1.0/powershell.exe”,
“applicationArguments”: [“arg1”, “arg2”]
}
],
“events”: [
{
“latitude”: 43.0525803,
“longitude”: -76.1546939,
“temperature”: 220,
“units”: “F”,
“timestamp”: “2016-04-24T08:46:45.014”
},
{
“latitude”: 43.0525803,
“longitude”: -76.1546939,
“temperature”: 219,
“units”: “F”,
“timestamp”: “2016-04-24T08:48:45.015”
}
]
}
}
The external message properties for a Freight Train IoT JSON message might look like the following.
{
“signature”: “http://tempuri.org/Parallelspace/TEAM/”,
“sent”: “2016-04-24T08:48:45.001”,
“psn.multipart”: {
“enabled”: “true”,
“numberofparts”: 6,
“partnumber”: 1,
“size”: 3425
},
“psn.masterMultipartID”: “ED3B42A2-3196-4937-91EE-0076747F79EB”,
“psn.multipartID”: “28A0C24B-CB84-4A16-8679-01035B844888”
}
Conclusions |
This document described motivation, problem statement, solution options, and recommendations for a layered or tiered approach for the Parallelspace TEAM IoT JSON message format based on a set of documented requirements. The final solution is based on the definition of four (4) message formats: JSON Blow Darts, JSON Arrows, JSON Spears, and JSON Freight Trains.
This document described each of these 4 IoT JSON message formats and provided an example of each format.
NOTICES |
The information contained in this document represents the current view of Parallelspace Corporation on the issues discussed as of the date of publication. Because Parallelspace Corporation must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Parallelspace Corporation, and Parallelspace Corporation cannot guarantee the accuracy of any information presented after the date of publication.
This document is for informational purposes only. Parallelspace Corporation MAKES NO WARRANTIES, EXPRESS, IMPLIED, OR STATUTORY, AS TO THE INFORMATION IN THIS DOCUMENT.
Complying with all applicable copyright laws is the responsibility of the user. Parallelspace Corporation grants you the right to reproduce this document, in whole or in part, specifically and solely for the purpose of personal education.
Parallelspace Corporation may have patents, patent applications, trademarks, copyrights, or other intellectual property rights covering subject matter in this document. Except as expressly provided in any written license agreement from Parallelspace Corporation, the furnishing of this document does not give you any license to these patents, trademarks, copyrights, or other intellectual property.
Unless otherwise noted, the example companies, organizations, products, domain names, e-mail addresses, logos, people, places and events depicted herein are fictitious, and no association with any real company, organization, product, domain name, email address, logo, person, place or event is intended or should be inferred.
© 2001-2016 Parallelspace Corporation. All rights reserved.
Parallelspace Corporation, Truly Collaborative Business Solutions, Parallelspace Total Enterprise Architecture Management and Parallelspace TEAM are either registered trademarks or trademarks of Parallelspace Corporation in the United States, Canada and/or other countries. The names of actual companies and products mentioned herein may be the trademarks of their respective owners
[1] Trivia: Fictitious temperature reading of one of the BBQ smokers at the Dinosaur Bar-B-Que in Syracuse, NY.
Filed under IoT, JSON Messages
Enterprise IoT and Total Enterprise Architecture Awesome Scenario #2: Microsoft MAP Toolkit
[Caveat: The Microsoft Assessment and Planning (MAP) Toolkit strictly speaking does not use any of the Azure services that support the Microsoft Azure IoT Suite. That being said the MAP Toolkit is a great example of what the analysis experience might be like. Maybe some day there will be a direct pipe from Azure Event Hubs/Stream Analytics/Data Factory and the MAP Toolkit discovery, logging and tracking capabilities.]
Previously I highlighted that one of the Delve Analytics videos was a great example of what an Enterprise IoT experience might look like. In this post, I’m highlighting a second example: The Microsoft Assessment and Planning (MAP) Toolkit.
Here is some what dated overview video but check it out just the same. You’ll certainly get the idea.
Lastly, here’s a second screen shot from the MAP Toolkit sample database – the one the results from drilling down to learn more about your organization’s Windows Server 2008 R2 deployment/upgrade readiness.
Michael Herman (Toronto)
Filed under IoT, Parallelspace TDM
Enterprise IoT and Total Enterprise Architecture Awesome Scenario #1: Delve Analytics
[Caveat: Delve Analytics (and the Microsoft Office Graph) strictly speaking do not use any of the Azure services that support the Microsoft Azure IoT Suite. That being said Delve Analytics is a great example of what the analysis experience might be like. Maybe some day there will be a direct pipe from Azure Event Hubs/Stream Analytics/Data Factory into the Office Graph. Given this crosses product group boundaries, I wouldn’t expect this to come about any time soon.]
Today, I ran across this 9 minute video with Ryan Fuller about a new Office Graph capability called Delve Analytics.
This video is an absolutely great example of the Enterprise IoT landscape I talked about in my article from a couple days ago: External IoT vs. Internal IoT: Beware of the Hype Cycle. … an absolutely perfect example.
Here’s a link to the YouTube video Introducing Delve Analytics in Office 365 (April 6, 2016).
Watch it …it’s only 9 minutes.
Michael Herman (Toronto)
Filed under IoT, Parallelspace TDM
External IoT vs. Internal IoT: Beware of the Hype Cycle
Subtitle: Fusing Enterprise IoT and Traditional Enterprise Architecture
Context
External IoT – External world of devices, events, connections, storage, and analysis; the “traditional” Internet of Things; the world of devices outside the enterprise.
Internal IoT – Internal world within the enterprise consisting of business processes, business objects, actors and roles; application components, application services, application functionality, and data objects; and lastly, infrastructure consisting of servers, networks, data stores, foundation services, and foundational functionality. The “hum” within an Enterprise.
Enterprise IoT – The confluence or integration of External IoT and Internal IoT landscapes centered around a particular enterprise organization. Often represented and accessed as an enterprise graph.
Ecosystem IoT – The confluence or integration of 2 or more separate Enterprise IoT landscapes (complete or partial) centered around a specific ecosystem or community. Supporting Federated Enterprise Architecture.
Discussion
Do some of these latter terms sound familiar? If so, you likely have some exposure, background, and experience with the practice of Enterprise Architecture Management (EAM). Having lived on both sides of the IoT divide for more than a decade, it’s interesting to watch how the current rage around IoT is almost exclusively focused on External IoT and it’s coupling to Business Intelligence and Analysis.
What about what’s happening inside the business, information, application and infrastructure architecture of our own enterprises? …regardless of whether the “things” are internal to your organization, external to your organization, or, possibly, part of someone else’s organization (e.g. owned by a client, customer, partner, …), it’s all part of the Enterprise IoT landscape.
A good name for the combination of External IoT and Internal IoT is the “Enterprise of Things” …but another organization is already using this term.
Ultimately, this is all about the Internet of Things merged and being combined with Enterprise Architecture Management: Enterprise IoT.
“More news at 11…”
Michael Herman (Toronto)
Master Data Services in SQL Server 2016 RC2: Significant Improvement
The Master Data Services in SQL Server 2016 RC2 appears to be a significant improvement over what shipped with SQL Server 2014 GA. There’s obvious UI improvements. But more importantly with almost no effort I was able to get Verification to work properly using the local MDS web portal coupled with the MDS Excel integration. For the latter, I had Excel 2013 installed along with the MDS Excel add-in for SQL Server 2016 RC2 that I downloaded from the Microsoft Download Center.
Filed under Master Data Management
Glossary: Controlled Vocabulary, Dictionary, Glossary, Grammar, Taxonomy, Folksonomy, Ontology, Semantic Network, Knowledge Graph
Controlled Vocabulary (Vocabulary) – list of distinct, selected terms or keywords
Dictionary or Glossary – a vocabulary with definition(s)
Grammar – a set of structural rules governing the composition of clauses, phrases, and words in any given language
Taxonomy – terms from a vocabulary, dictionary or glossary that are structured or organized by a classification system that is most often hierarchical in nature (but this is not a requirement).
Folksonomy – terms from a vocabulary, dictionary or glossary are categorized by a set of users’ tags or keywords.
Ontology, Semantic Network, Knowledge Graphs – superset of capabilities relative to a taxonomy that includes properties (with or without data types) as well as potentially arbitrary interrelationships; a set of types, properties, and relationships.
Good (complete) references:
Filed under Definitions
Hyperonomy Digital Identity Lab™ - hyperonomy.com 