Category Archives : Data Warehouse

07

Nov

Azure SQL Data Warehouse introduces new productivity and security capabilities

SQL Data Warehouse continues to provide a best in class price to performance offering, leading others in TPC-H and TPC-DS benchmarks based on independent testing. As a result, we are seeing customers, including more than 50 percent of Fortune 1000 enterprise such as Anheuser Busch InBev, Thomson Reuters, and ThyssenKrupp build new analytics solutions on Azure. 

With the launch of SQL Data Warehouse Gen2 in April 2018, customers have benefited tremendously from query performance and concurrency enhancements. To support our customers’ exponentially growing data volume and resulting analytics workloads, today we are sharing new SQL Data Warehouse features. Enhanced workload management, row-level security, and improved operational experiences.

Enhanced workload management

SQL Data Warehouse will offer workload management capabilities that optimize query execution to ensure that high-value work gets priority access to system resources. With features such as workload importance, customers can use a single SQL Data Warehouse database to more efficiently run multiple workloads, taking away the complexity of separate data warehouses for each solution. With this new capability, SQL Data Warehouse enables better control, utilization, and optimization over deployed resources. Workload importance will be available for all SQL Data Warehouse customers later this year at no additional cost.

Share

07

Nov

Workload insights into SQL Data Warehouse delivered through Microsoft Azure Monitor diagnostic logs

SQL Data Warehouse (SQL DW) now enables enhanced insights into analytical workloads by integrating directly with Microsoft Azure Monitor diagnostic logs. This new capability enables developers to analyze workload behavior over an extended time period and make informed decisions on query optimization or capacity management. SQL DW is a flexible, secure, and fully managed analytics platform for the enterprise optimized for running complex queries quickly across petabytes of data.

Today, customers leverage Dynamic Management Views (DMVs) to get insights into their data warehouse workload. These DMVs have a limit of 10,000 rows that can easily be exceeded for intensive enterprise data warehouse workloads with heavy query activity. Relying solely on DMVs hinders or blocks many query troubleshooting scenarios for active workloads. To work around this DMV limitation, custom logging solutions were required which consumed internal system resources, increased the total cost of the data warehouse solution, and introduced additional development complexities and maintenance effort.

We have now introduced an external logging process through Azure Monitor diagnostic logs, which provides additional insights into your data warehouse workload. With a single click of a button, you are now able to configure diagnostic logs for historical query performance troubleshooting capabilities using Log Analytics.

Share

07

Nov

https://azure.microsoft.com/blog/automatically-discover-workload-insights-for-advanced-performance-tuning-directly-in-the-azure-portal/Advanced tuning for Azure SQL Data Warehouse (SQL DW) just got simpler with additional data warehouse recommendations and metrics. SQL DW is a flexible, secure, and fully managed analytics platform for enterprises optimized for running complex queries quickly across petabytes READ MORE

Share

07

Nov

Azure SQL Data Warehouse provides frictionless development using SQL Server Data Tools

The highly requested feature for SQL Data Warehouse (SQL DW) is now in preview with the support for SQL Server Data Tool (SSDT) in Visual Studio! Teams of developers can now collaborate over a single, version-controlled codebase and quickly deploy changes to any instance in the world. SQL DW is a flexible, secure, and fully managed analytics platform for the enterprise optimized for running complex queries quickly across petabytes of data.

Currently, change management and deployment for SQL DW is a non-trivial effort where customers must build SQL and PowerShell scripts. This becomes an unmanageable experience as modern data warehouse solutions can have over hundreds of data pipelines and thousands of database objects. This issue is exacerbated as data warehouse deployments typically have multiple environments for development, tests, and production. A stable continuous integration and deployment process becomes critical at this point.

With SSDT, database project support enables a first-class enterprise-grade development experience for your modern data warehouse. You can check data warehouse scripts into source control and leverage Microsoft Azure DevOps within Visual Studio. As your business requirements around data evolve, increase your development velocity with SQL DW by seamlessly applying and deploying changes using features such as schema

Share

11

Oct

Data models within Azure Analysis Services and Power BI
Data models within Azure Analysis Services and Power BI

In a world where self-service and speed of delivery are key priorities in any reporting BI solution, the importance of creating a comprehensive and performant data semantic model layer is sometimes overlooked.

I have seen quite a few occurrences where the relational data store such as Azure SQL Database and Azure SQL Data Warehouse are well structured, the reporting tier is well presented whether that is SQL Server reporting services or Power BI, but still, the performance is not as expected.

Before we drill down to the data semantic model, I always advise that understanding your data and how you want to present and report on it is key. By creating a report that takes the end consumer through a data journey is the difference between a good and a bad report. Report designing should take into account who is consuming and what they want to achieve out of the report. For example, if you have a small number of consumers who need to view a lower level of hierarchical data with additional measures or KPIs, then it may not be suitable to visualize this on the first page. As the majority of consumers may want to view a more

Share

08

Oct

Re-define analytics with Azure SQL Data Warehouse Gen2 tier now generally available

Microsoft Azure Government regions give unparalleled flexibility for US government agencies and their partners. Today, we announce the general availability of industry-leading performance Azure SQL Data Warehouse Compute Optimized Gen2 tier for government customers in the US Gov Arizona and US Gov Texas regions. Azure SQL Data Warehouse is a fast, flexible, and secure analytics platform offering you a SQL-based view across data. It is elastic, enabling you to provision a cloud data warehouse in minutes and scale to terabytes in minutes.

With Azure Government only US federal, state, local, and tribal governments and their partners have access to this dedicated instance that only screened US citizens operate. Customers can choose from six government-only datacenter regions, including two regions granted an Impact Level 5 Provisional Authorization. Moreover, Azure Government offers the most compliance certifications of any cloud provider.

These new locations bring the product worldwide availability count for Compute Optimized Gen2 tier to 28 regions.

More performance, concurrency, and power

Azure SQL Data Warehouse Gen2 brings the best of Microsoft software and hardware innovations to dramatically improve query performance and concurrency. Our customers now get up to five times better performance on average for query workloads, four times more

Share

02

Oct

Enabling real-time data warehousing with Azure SQL Data Warehouse

Gaining insights rapidly from data is critical to competitiveness in today’s business world. Azure SQL Data Warehouse (SQL DW), Microsoft’s fully managed analytics platform leverages Massively Parallel Processing (MPP) to run complex interactive SQL queries at every level of scale.

Users today expect data within minutes, a departure from traditional analytics systems which used to operate on data latency of a single day or more. With the requirement for faster data, users need ways of moving data from source systems into their analytical stores in a simple, quick, and transparent fashion. In order to deliver on modern analytics strategies, it is necessary that users are acting on current information. This means that users must enable the continuous movement from enterprise data, from on-premise to cloud and everything in-between.

SQL Data Warehouse is happy to announce that Striim now fully supports SQL Data Warehouse as a target for Striim for Azure. Striim enables continuous non-intrusive performant ingestion of all your enterprise data from a variety of sources in real time. This means that users can use intelligent pipelines for change data capture from sources such as Oracle Exadata straight into SQL Data Warehouse. Striim can also be used to move fast

Share

26

Sep

Eight new features in Azure Stream Analytics
Eight new features in Azure Stream Analytics

This week at Microsoft Ignite 2018, we are excited to announce eight new features in Azure Stream Analytics (ASA). These new features include

Support for query extensibility with C# custom code in ASA jobs running on Azure IoT Edge.Custom de-serializers in ASA jobs running on Azure IoT Edge.Live data Testing in Visual Studio.High throughput output to SQL.ML based Anomaly Detection on IoT Edge.Managed Identities for Azure Resources (formerly MSI) based authentication for egress to Azure Data Lake Storage Gen 1.Blob output partitioning by custom date/time formats.User defined custom re-partition count.

The features that are generally available and the ones in public preview will start rolling imminently. For early access to private preview features, please use our sign up form.

Also, if you are attending Microsoft Ignite conference this week, please attend our session BRK3199 to learn more about these features and see several of these in action.

General availability features Parallel write operations to Azure SQL

Azure Stream Analytics now supports high performance and efficient write operations to Azure SQL DB and Azure SQL Data Warehouse to help customers achieve four to five times higher throughput than what was previously possible. To achieve fully parallel topologies, ASA will transition SQL writes

Share

25

Sep

Azure SQL Data Warehouse now supports maintenance scheduling
Azure SQL Data Warehouse now supports maintenance scheduling

As customer build their mission-critical analytics solutions using Azure SQL Data Warehouse (SQL DW), Microsoft’s fast, flexible and secure analytics platform, we want to ensure that we give them greater control over the management of their workloads.

Today we are excited to announce a preview of scheduled maintenance for Azure SQL DW. This new feature seamlessly integrates the Service Health Planned Maintenance Notifications, Resource Health Check Monitor and the Azure SQL Data Warehouse maintenance scheduling service. Customers can now choose a preferred maintenance window based on their business requirements, giving them the flexibility to select times when system maintenance is to be performed.

With the power of these combined services and the new functionality developed by the Azure SQL DW, the team provides a predictable and stable service upgrade and maintenance experience. Azure SQL Warehouse now gives you the tools you need to plan around this necessary service maintenance and minimize the impact on your day to day operations. You can learn more about how to use this function.

The new functionality can be accessed via your data warehouse overview blade and a new the resource menu option.

Azure SQL Data Warehouse maintenance scheduling is now available in the

Share

25

Sep

Seamless access to management insights for SQL Data Warehouse with Data Studio

Azure SQL Data Warehouse (SQL DW) is a fast, flexible, and secure analytics platform that helps our customers transform data into actionable insights. Starting today, Azure SQL DW customers can seamlessly get rich insights and build customizable dashboard widgets to better manage and tune their workloads.

This experience is made real by the general availability of Azure Data Studio, a tool that provides a Transact-SQL (T-SQL) code editor with IntelliSense, an integrated terminal for common command line tools, and customizable database widgets encapsulated through extensions. The Azure SQL DW insights extension can be leveraged with Azure Data Studio to automatically render dashboard widgets to help surface tuning insights into your data warehouse. These dashboard views are automatically refreshed to help you seamlessly apply best practices and ensure that your workload is optimized for performance within a single tool.

Currently, the following dashboard widgets are immediately available when downloading the extension.

Data distribution across your data warehouse

You can visualize the data distribution across your data warehouse to immediately detect whether your data warehouse is suffering from physical data skew which can impact query performance.

Table health

These views help you maximize columnstore row group quality, address suboptimal statistics,

Share