Successful digital transformation requires eliminating unnecessary barriers. To that end, we announced Azure Synapse Analytics in November 2019, where we removed the barriers between data warehousing and big data analytics. And in 2020, we took this a step further and announced Azure Synapse Link for Azure Cosmos DB to break down the barriers that had long existed between operational data and analytical systems.
Over the past two years, customer feedback has made it clear that when data barriers are dissolved the impact of analytics grows exponentially. Today, we’re announcing the next step in bringing data insights to all by eliminating the barrier between business applications and analytical systems with Azure Synapse Link for Microsoft Dataverse.
Introducing Azure Synapse Link for Dataverse
The barrier between business applications data and analytical systems is a critical factor that impedes accelerated time-to-insight. As developers use platforms such as Microsoft Power Apps, Microsoft Power Automate, and Microsoft Dynamics 365 to create and manage business applications, the data that comes from these applications is massive. Today, customers store and manage this data in Dataverse—a common store for all Microsoft business applications. However, when customers want to discover deep insights from the data within Dataverse it
Azure Stream Analytics is a fully managed Platform as a Service (PaaS) that supports thousands of mission-critical customer applications powered by real-time insights. Out-of-the-box integration with numerous other Azure services enables developers and data engineers to build high-performance, hot-path data pipelines within minutes. The key tenets of Stream Analytics include Ease of use, Developer productivity, and Enterprise readiness. Today, we’re announcing several new features that further enhance these key tenets. Let’s take a closer look at these features:
Rollout of these preview features begins November 4th, 2019. Worldwide availability to follow in the weeks after.
In the past, changing Streaming Units (SUs) allocated for a Stream Analytics job required users to stop and restart. This resulted in extra overhead and latency, even though it was done without any data loss.
With online scaling capability, users will no longer be required to stop their job if they need to change the SU allocation. Users can increase or decrease the SU capacity of a running job without having to stop it. This builds on the customer promise of long-running mission-critical pipelines that Stream Analytics offers today.
Change SUs on a Stream Analytics job while it is running.
MATCH_RECOGNIZE in Azure Stream Analytics significantly reduces the complexity and cost associated with building, modifying, and maintaining queries that match sequence of events for alerts or further data computation.
What is Azure Stream Analytics?
Azure Stream Analytics is a fully managed serverless PaaS offering on Azure that enables customers to analyze and process fast moving streams of data and deliver real-time insights for mission critical scenarios. Developers can use a simple SQL language, extensible to include custom code, in order to author and deploy powerful analytics processing logic that can scale-up and scale-out to deliver insights with milli-second latencies.
Traditional way to incorporate pattern matching in stream processing
Many customers use Azure Stream Analytics to continuously monitor massive amounts of data, detecting sequence of events and deriving alerts or aggregating data from those events. This in essence is pattern matching.
For pattern matching, customers traditionally relied on multiple joins, each one detecting a single event in particular. These joins are combined to find a sequence of events, compute results or create alerts. Developing queries for pattern matching is a complex process and very error prone, difficult to maintain and debug. Also, there are limitations when trying to express more complex
Azure Stream Analytics is a fully managed PaaS offering that enables real-time analytics and complex event processing on fast moving data streams. Thanks to zero-code integration with over 15 Azure services, developers and data engineers can easily build complex pipelines for hot-path analytics within a few minutes. Today, at Inspire, we are announcing various new innovations in Stream Analytics that help further reduce time to value for solutions that are powered by real-time insights. These are as follows:
Bringing the power of real-time insights to Azure Event Hubs customers
Today, we are announcing one-click integration with Event Hubs. Available as a public preview feature, this allows an Event Hubs customer to visualize incoming data and start to write a Stream Analytics query with one click from the Event Hub portal. Once the query is ready, they will be able to operationalize it in few clicks and start deriving real time insights. This will significantly reduce the time and cost to develop real-time analytics solutions.
One-click integration between Event Hubs and Azure Stream Analytics
Augmenting streaming data with SQL reference data support
Reference data is a static or slow changing dataset used to augment real-time data streams to deliver more