Subject: MS Azure Services: Is there an overarching architectural vision?

From: Michael Herman (Parallelspace)
Sent: Monday, May 16, 2016 7:58 AM
To: Mark Russinovich; Scott Guthrie
Subject: MS Azure Services: Is there an overarching architectural vision?

Hi Mark and Scott,

I’m pretty sure that you and I have never met – although I’ve been building apps on the Windows platform since 1986 (version 0.989 of the SDK) and I worked for MSFT from 1996-2001 – mostly on the SharePoint team.

My question is: Is there an overarching architectural vision for the (complete) MS Azure platform? There doesn’t appear to be and I’ll give you an example in a second.

It appears like we’re back in 1996 when Billg/Microsoft finally “got the Internet” and every product group began a completely different, disconnected, uncoordinated effort to add “Internet” to their own little piece of software (e.g. HTML publishing mostly; support for some Internet protocols; etc.) Every day something new was being “released to the web (RTW)” and the Microsoft platform was a mess.

What I saw with Microsoft and the Internet in 1996 is happening all over again with the Microsoft Azure strategy: a disconnected and uncoordinated and not well documented strategy and implementation. Azure [as a strategic platform] is a mess.

My example: MS Azure services, MS IoT Suite, MS Cortana Suite, Azure Logic Apps.

The whole MS approach to Azure Event Hubs is extraordinary (good): Lightweight, inexpensive, brain-dead easy to use, largely unrestricted in terms of how Event Hubs can be used and the use cases they can support.

If I have some event data in an Event Hub, the very next thing I want to do is perform some post-processing of the event data – ideally staying with a streaming, hyper-scalable architecture. I [also] want to create an easily composable pipeline of activities that each perform some sort of processing of each event as the events progress through the pipeline. I want the pipeline activities to be easy to design and implement (e.g. kind of like PowerShell cmdlets but it doesn’t have to that specific implementation pattern but with a similar simple pattern that is easy to build upon).

But which MS Azure services have native support for consuming event data stored in an Azure Event Hub? …almost none

…except for Azure Stream Analytics. But what about Azure Data Factory? What about Azure Logic Apps? Are there other Azure Services I should be considering? …without having to build and manage [several] custom Azure WebJobs and use all sorts of intermediate Azure storage just to connect the applicable Azure services together?

Of the 3 options I listed, you chose the one, Azure Stream Analytics, that has [the] most baroque programming model (some SQL derivation). None of the latter two have any native support for event data stored in an Event Hub and they are a lot easier to use than Stream Analytics SQL scripts.

Where is the Azure documentation that is targeted at Architects?

The MS IoT Suite team has started a good MS IoT Reference Architecture document: http://download.microsoft.com/download/A/4/D/A4DAD253-BC21-41D3-B9D9-87D2AE6F0719/Microsoft_Azure_IoT_Reference_Architecture.pdf

Where is the consistent set of similar documents for the other collections of MS Azure services? Where is the overarching MS Azure architecture document?

Given a starting point and a goal (and some intermediate subgoals), where is the “Google Map” of all of the possible connections and routings across all of the MS Azure services?

Can you help?

Best regards,
Michael Herman (Toronto)