Category Archives : Database

21

Jun

Resumable Online Index Create is in public preview for Azure SQL DB

We are delighted to announce that Resumable Online Index Create (ROIC) is now available for public preview in Azure SQL DB. The feature allows the ability to pause an index create operation and resume it later from where the index create operation was paused or failed, rather than having to restart the operation from the beginning. Additionally, this feature creates indexes using only a small amount of log space. You can use the new feature in the following scenarios:

Resume an index create operation after an index create failure, such as after a database failover or after running out of disk space. There is no need to restart the operation from the beginning. This can save a significant amount of time when creating indexes for large tables. Pause an ongoing index create operation and resume it later. For example, you may need to temporarily free up system resources in order to execute a high priority task or you may have a single maintenance window that is too short to complete the operation for a large index. Instead of aborting the index create process, you can pause the index create operation and resume it later without losing prior progress. Create large

Share

21

Jun

Resumable Online Index Create is in public preview for Azure SQL DB

We are delighted to announce that Resumable Online Index Create (ROIC) is now available for public preview in Azure SQL DB. The feature allows the ability to pause an index create operation and resume it later from where the index create operation was paused or failed, rather than having to restart the operation from the beginning. Additionally, this feature creates indexes using only a small amount of log space. You can use the new feature in the following scenarios:

Resume an index create operation after an index create failure, such as after a database failover or after running out of disk space. There is no need to restart the operation from the beginning. This can save a significant amount of time when creating indexes for large tables. Pause an ongoing index create operation and resume it later. For example, you may need to temporarily free up system resources in order to execute a high priority task or you may have a single maintenance window that is too short to complete the operation for a large index. Instead of aborting the index create process, you can pause the index create operation and resume it later without losing prior progress. Create large

Share

18

Jun

Announcing the general availability of Azure SQL Data Sync

We are delighted to announce the general availability (GA) of Azure SQL Data Sync! Azure SQL Data Sync allows you to synchronize data between Azure SQL Database and any other SQL endpoints unidirectionally or bidirectionally. It enables hybrid SQL deployment and allows local data access from both Azure and on-premises application. It also allows you to deploy your data applications globally with a local copy of data in each region, and keep data synchronized across all the regions. It will significantly improve the application response time and reliability by eliminating the impact of network latency and connection failure rate.

What’s new in Azure SQL Data Sync

With the GA announcement, Azure SQL Data Sync supports some new capabilities:

Better configuration experience – More reliable configuration workflow and more intuitive user experience. More reliable and faster database schema refresh – Load database schema more efficiently using new SMO library. More secured data synchronization – We reviewed the end-to-end sync workflow and ensured user data are always encrypted at rest and in transit. Data Sync service now meets GDPR compliance requirement. Get started today – Try out Azure SQL Data Sync

If you are building a hybrid platform or global distributed

Share

10

Jun

Microsoft Azure Data welcomes attendees to ACM SIGMOD/PODS 2018

Hello SIGMOD attendees!

Welcome to Houston, and to what is shaping up to be a great conference.  I wanted to take this opportunity to share with you some of the exciting work in data that’s going on in the Azure Data team at Microsoft, and to invite you to take a closer look.

Microsoft has long been a leader in database management with SQL Server, recognized as the top DBMS by Gartner for the past three years in a row.  The emergence of the cloud and edge as the new frontiers for computing, and thus data management, is an exciting direction—data is now dispersed within and beyond the enterprise, on-prem, on-cloud, and on edge devices, and we must enable intelligent analysis, transactions, and responsible governance for all data everywhere, from the moment it is created to the moment it is deleted, through the entire life-cycle of ingestion, updates, exploration, data prep, analysis, serving, and archival. 

These trends require us to fundamentally re-think data management.  Transactional replication can span continents.  Data is not just relational.  Interactive, real-time, and streaming applications with enterprise level SLAs are becoming common.  Machine learning is a foundational analytic task and must be supported while ensuring that

Share

07

Jun

Azure Backup for SQL Server on Azure now in public preview

Earlier this week, Corey Sanders announced preview of a new Azure Backup capability to backup SQL workloads running in Azure Virtual Machines in his post about why you should bet on azure for your infrastructure needs today and in the future. In this blog, we will elaborate on how this enterprise backup provides a new breakthrough in backup that differentiates Azure from any other public cloud. This workload backup capability is built as an infrastructure-less, Pay aA You Go (PAYG) service that leverages native SQL backup and restore APIs to provide a comprehensive solution to backup SQL servers running in Azure IaaS VMs.

Key benefits Zero-infrastructure backup: Freedom from managing backup infrastructure (e.g. backup server, agents or backup storage) or writing complex backup scripts. Centrally manage and monitor all backups using Recovery Services Vault: Create policies to specify the backup schedule and retention for both short-term and long-term retention needs using Grandfather-father-son style retention schemes. Re-use these policies across multiple databases across servers. Configure email notification for any backup or restore failure. Monitor the backup jobs using Recovery Services Vault dashboard for all workloads including Azure IaaS VMs, Azure Files and SQL server databases. Restore to any time,

Share

06

Jun

Azure Search is now certified for several levels of compliance

Compliance is an important factor for customers when looking at software and services as they look to meet their own compliance obligations across regulated industries and markets worldwide. For example, ISO 27001 certification is a security standard that provides a baseline set of requirements for many other international standards and regulations and HIPAA (Health Insurance Portability and Accountability Act) is a US law that establishes requirements for the use, disclosure, and safeguarding of protected health information (PHI).

For that reason, we are excited to announce that Azure Search has been certified for several levels of compliance including:

ISO 27001:2013 SOC 2 Type 2 GxP (21 CFR Part 11) HIPAA and the HITECH Act HITRUST PCI DSS Level 1 Australia IRAP Unclassified

With these certifications and attestations, we hope to enable Azure Search as a viable option for customers looking to meet and attain key international and industry-specific compliance standards within their solutions.

Azure compliance offerings are grouped into four segments: globally applicable, US government, industry specific, and region/country specific. To view an overview of Azure Search as well as other Microsoft Azure compliance offerings, please visit the Microsoft Trust Center. In addition, you can directly download a document that provides

Share

05

Jun

Use Azure Data Lake Analytics to query AVRO data from IoT Hub

Recently a customer asked me how to read blob data produced from the routing capability of Azure IoT Hub. To provide this customer with a complete answer, I put together a step-by-step guide that I am happy to share with you in the video below.

One of the common patterns of Internet of Things applications is called “cold path” and consists of storing all data produced by IoT devices in the cloud for later processing. To make such an implementation trivial, Azure IoT Hub supports routing of messages coming from devices directly to cloud storage services. IoT Hub can also apply simple rules based on both properties, and the message body can route messages to various custom endpoints of your choice. IoT Hub will write blob content in AVRO format, which has both message body and message properties. Great for data/message preservation, AVRO can be challenging for querying and processing the data. Here is a suggested solution to process this data.

Many of the big data patterns can be used for processing non-relational data files in custom file formats. Focusing on cost and deployment simplicity, Azure Data Lake Analytics (ADLA) is one of the only “pay per query” big data

Share

17

May

Announcing SQL Advanced Threat Protection (ATP) and SQL Vulnerability Assessment general availability

We are delighted to announce the general availability of SQL Vulnerability Assessment for Azure SQL Database! SQL Vulnerability Assessment (VA) provides you a one-stop-shop to discover, track and remediate potential database vulnerabilities. It helps give you visibility into your security state, and includes actionable steps to investigate, manage and resolve security issues, and enhance your database fortifications. VA is available for Azure SQL Database customers as well as for on-premises SQL Server customers via SSMS.

If you have data privacy requirements or need to comply with data protection regulations like the European Union General Data Protection Regulation (EU GDPR), then VA is your built-in solution to simplify these processes and monitor your database protection status. For dynamic database environments where changes are frequent and hard to track, VA is invaluable in detecting the settings that can leave your database vulnerable to attack.

New SQL Advanced Threat Protection (ATP)

VA is being released to general availability (GA) as part of a new security package for your Azure SQL Database, called SQL Advanced Threat Protection (ATP). ATP provides a single go-to location for discovering, classifying and protecting sensitive data, managing your database vulnerabilities, and detecting anomalous activities that could indicate a

Share

10

May

Extract management insights from SQL Data Warehouse with SQL Operations Studio

SQL Operations Studio can be leveraged with Azure SQL Data Warehouse (SQL DW) to create rich customizable dashboard widgets surfacing insights to your data warehouse. This unlocks key scenarios around managing and tuning your data warehouse to ensure it is optimized for consistent performance. Previously, developers had to manually and continuously execute complex DMV queries to extract insights from their data warehouse. This leads to a repetitious process when following development and tuning best practices with SQL DW. Now with SQL Operations Studio, customized insight widgets can be embedded directly within the query tool enabling you to seamlessly monitor and troubleshoot issues with your data warehouse.

The following widgets can be generated by using the provided T-SQL monitoring scripts within SQL Operations Studio for common data warehouse insights.

Data Skew

Detect data skew across distributions to help identify and troubleshoot query performance issues:

Columnstore health and statistics

Leverage views to help maximize columnstore row group quality and ensure table statistics are up to date for optimal query performance:

User Activity

Identify and understand workload patterns through active sessions queries, queued queries, loads, and backups:

Resource Bottlenecks

Ensure adequate resources are allocated such as memory and TempDB:

Share

09

May

Announcing new Async Java SDK for Azure #CosmosDB
Announcing new Async Java SDK for Azure #CosmosDB

We’re excited to announce a new asynchronous Java SDK for Cosmos DB’s SQL API open sourced on GitHub. This SDK leverages the popular RxJava library to add a new async API surface area for composing event-based programs with observable sequences. It also features an improved user experience and is also lighter weight than our previous synchronous Java SDK (yielding a 2x performance improvement on the client-side)!

You can add the library from Maven using:

<dependency> <groupId>com.microsoft.azure</groupId> <artifactId>azure-cosmosdb</artifactId> <version>1.0.1</version> </dependency> Connect to Cosmos DB

The new SDK uses convenient builder pattern to specify connectivity options:

asyncClient = new AsyncDocumentClient.Builder() .withServiceEndpoint(HOST) .withMasterKey(MASTER_KEY) .withConnectionPolicy(ConnectionPolicy.GetDefault()) .withConsistencyLevel(ConsistencyLevel.Eventual) .build(); Insert an item

To execute and coordinate Cosmos DB data operations asynchronously, and get the results you use observables:

Document doc = new Document(String.format(“{ ‘id’: ‘doc%d’, ‘counter’: ‘%d’}”, 1, 1)); Observable<ResourceResponse<Document>> createDocumentObservable = asyncClient.createDocument(collectionLink, doc, null, false); createDocumentObservable .single() // we know there will be one response .subscribe(documentResourceResponse -> { System.out.println(documentResourceResponse.getRequestCharge()); });

Note that the createDocument request will be issued only once .subscribe is called on the corresponding observable result.

Query

In Cosmos DB queries can return multiple pages of data. To efficiently read all the pages, simply subscribe and

Share