Azure SQL Data Warehouse (SQL DW) is a fast, flexible and secure, cloud data warehouse tuned for running complex queries fast and across petabytes of data. Continuing to deliver on this promise, we have announced the general availability of the next generation of SQL DW which includes an average of five times the performance boost, five times the increase in compute scalability, and four times the increase in concurrency. The release of Azure SQL DW Compute Optimized Gen2 tier comes with an expansion of 14 additional regions bringing the global region footprint of SQL DW Gen2 to 20 surpassing all other major cloud providers. The following regions are available:
East US 2
North Central US
South Central US
West US 2
With more global regions than any other
Azure Cosmos DB is Microsoft’s globally distributed, multi-model database. Azure Cosmos DB enables you to elastically and independently scale throughput and storage across any number of Azure’s geographic regions with a single click. It offers throughput, latency, availability, and consistency guarantees with comprehensive service level agreements (SLAs), a feature that no other database service can offer.
A database that holds sensitive data across international borders must meet high standards for security, privacy, and compliance. Additionally, the cloud service provider must anticipate and be ready for new standards, such as the General Data Protection Regulation (GDPR), which will soon govern the collection and use of EU resident’s data. Microsoft has pledged that Azure services will be GDPR compliant by the May 25 implementation date.
Microsoft’s cloud privacy policies state that we will use your customer data only to provide the services we have agreed upon, and for purposes that are compatible with providing those services. We do not share your data with our advertiser-supported services, nor do we mine it for marketing or advertising.
Azure Cosmos DB also implements stringent security practices. All the documents, attachments and backups stored in Azure Cosmos DB are encrypted at rest and
Apache Spark is a unified analytics engine for large-scale data processing. Today, you can use the built-in JDBC connector to connect to Azure SQL Database or SQL Server to read or write data from Spark jobs.
The Spark connector for Azure SQL Database and SQL Server enables SQL databases, including Azure SQL Database and SQL Server, to act as input data source or output data sink for Spark jobs. It allows you to utilize real-time transactional data in big data analytics and persist results for adhoc queries or reporting.
Compared to the built-in Spark connector, this connector provides the ability to bulk insert data into SQL databases. It can outperform row-by-row insertion with 10x to 20x faster performance. The Spark connector for Azure SQL Databases and SQL Server also supports Azure Active Directory authentication. It allows you to securely connect to your Azure SQL database from Azure Databricks using your AAD account. The Spark connector also provides similar interfaces with the built-in JDBC connector and is easy to migrate your existing Spark jobs to use this new connector.
The Spark connector for Azure SQL Database and SQL Server utilizes the Microsoft JDBC Driver for SQL Server to move data between
Today, we are excited to announce the general availability of Transparent Data Encryption (TDE) with Bring Your Own Key (BYOK) support for Azure SQL Database and Azure SQL Data Warehouse. This is one of the most frequently requested features by enterprise customers looking to protect sensitive data and meet regulatory or compliance obligations that require implementation of specific key management controls. TDE with BYOK support is offered in addition to TDE with service managed keys, which is enabled by default on all new Azure SQL Databases.
TDE with BYOK support uses Azure Key Vault, which provides highly available and scalable secure storage for RSA cryptographic keys backed by FIPS 140-2 Level 2 validated Hardware Security Modules (HSMs). Key Vault streamlines the key management process and enables customers to maintain full control of encryption keys and allows them to manage and audit key access.
Customers can generate and import their RSA key to Azure Key Vault and use it with Azure SQL Database and Azure SQL Data Warehouse TDE with BYOK support. Azure SQL Database handles the encryption and decryption of data stored in databases, log files, and backups in a fully transparent fashion by using a symmetric Database Encryption Key
Azure SQL Data Warehouse (SQL DW) continues to introduce updates to the Azure portal to provide a seamless user experience when monitoring, managing, and integrating your data warehouse.
Support for Azure Monitor metrics
SQL DW now supports Azure Monitor which is a built-in monitoring service that consumes performance and health telemetry for your data warehouse. Azure monitor not only enables you to monitor your data warehouse within the Azure portal, but its tight integration between Azure services also enables you to monitor your entire data analytics solution within a single interface. For this release, data warehouse metrics have been enabled to enables you to identify performance bottlenecks and user activity:
Successful/Failed/Blocked by firewall connections CPU IO DWU Limit DWU Percentage DWU used
These metrics now have a one-minute frequency for near real-time visibility into resource bottlenecks of your data warehouse. There is a default retention period of 90 days for all data warehouse metrics with Azure Monitor.
Pin configured charts for your data warehouse through Azure dashboards:
Safely manage costs by pausing
The pause feature for SQL
Continuing in our series of tutorials showcasing features of Azure SQL database that enable SaaS app management, we are introducing two new tutorials that explore disaster recovery strategies for recovering an app and its resources in the event of an outage. Disaster Recovery (DR) is an important consideration for many applications, whether for compliance reasons or business continuity. Should there be a prolonged service outage, a well-prepared DR plan can minimize business disruption.
The tutorials target the database-per-tenant architecture model of the Wingtip Tickets sample and demonstrate recovery using the geo-restore capabilities of Azure SQL database, and recovery using the geo-replication capabilities of Azure SQL database.
Disaster recovery using geo-restore
In this tutorial, you will explore a full disaster recovery scenario using a geo-restore-based DR strategy. You use geo-restore to recover the catalog and tenant databases from automatically maintained geo-redundant backups into an alternate recovery region. After the outage is resolved, you use geo-replication to repatriate new and changed databases to their original production region.
Disaster recovery using geo-replication
Azure SQL Database and Data Warehouse offer encryption-at-rest by providing Transparent Data Encryption (TDE) for all data written to disk, including databases, log files and backups. This protects data in case of unauthorized access to hardware. TDE provides a TDE Protector that is used to encrypt the Database Encryption Key (DEK), which in turn is used to encrypt the data. With the TDE and Bring Your Own Key (BYOK) offering currently in preview, customers can take control of the TDE Protector in Azure Key Vault.
Taking advantage of TDE with BYOK for databases that are geo-replicated to maintain high availability requires to configure and test the scenario carefully. This post will go over the most common configuration options.
To avoid creating a single point of failure in active geo-replicated instances or SQL failover groups, it is required to configure redundant Azure Key Vaults. Each geo-replicated server requires a separate key vault, that must be co-located with the server in the same Azure region. Should a primary database become inaccessible due to an outage in one region and a failover is triggered, the secondary database is able to take over using the secondary key vault.
For Geo-Replicated Azure SQL databases, the
Azure SQL Data Warehouse recently announced the preview release of the Optimized for Compute performance tier providing customers with an enhanced offering of the service. With this major release, the service now has a 5X increase in compute scalability and unlimited storage for columnar data. Along with the increased capacity, customers are realizing an average increase of 5X in performance for query workloads. For existing Optimized for Elasticity customers wanting to capitalize on these benefits, there is now an option to seamlessly upgrade via the Azure Portal. The easy to use upgrade experience via the Azure Portal has no downtime associated with exporting and reimporting of the data.
Upgrade to optimize for performance
You can now upgrade to the latest performance tier within the Azure Portal. This will result in no change to your connection string details:
To learn more about the upgrade process, visit our upgrade documentation. If you need help for a POC, contact us directly. Stay up-to-date on the latest Azure SQL DW news and features by following us on Twitter @AzureSQLDW.
This blog post was co-authored by James Ashley, MR and AI Architect, Microsoft MVP.
Developers sometimes get anxious when it comes to hooking up a database for their apps. However, with Azure Database for MySQL and Azure Database for PostgreSQL, quickly propping up and accessing a relational database is a piece of cake. These lightweight, open source database services provide a great way to get small apps and prototypes started with very little effort. Without any extra work on your part, you can automatically take advantage of built-in security, fault tolerance, and data protection. You also can use point-in-time restore to recover a server to an earlier state—as far back as 35 days.
Azure Database for MySQL and Azure Database for PostgreSQL will work with whatever kind of project you are creating, whether it is a Linux app running in a Docker container orchestrated by Kubernetes, a computer vision service using Python, or a simple ASP.NET website to display travel photos. If your app needs a relational database, you can easily plug one in and start writing to it with guidance from these connect & query quickstarts:
Azure Database for MySQL
We have made some great improvements to the SQL elastic pool configuration experience in the Azure portal. These changes are released alongside the new vCore-based purchasing model for elastic pools and single databases. Our goal is to simplify your experience configuring elastic pools and ensure you are confident in your configuration choices.
Changing service tiers for existing pools
Existing elastic pools can now be scaled up and down between service tiers. You can easily move between service tiers and discover the one that best fits your business needs. You can also switch between the DTU-based and the new vCore-based service tiers. You can also scale down your pool outside of business hours to save cost.
Simplifying configuration of the pool and its databases
Elastic pools offer many settings for customers to customize. The new experience aims to separate and simplify each aspect of pool management, between the pool settings, database settings, and database management. This enables you to more easily reason over each of these aspects of the pool while being able to save all settings changes in one batch.
Understanding your bill with new cost summary