26

Apr

HDInsight tools for VS Code now supports argparse and Spark 2.2

We are happy to announce that HDInsight Tools for VSCode now supports argparse and accepts parameter based Pyspark Job submission. We also enabled the tools to support Spark 2.2 for PySpark author and job submission.

The argparse feature grants you great flexibility for your PySpark code author, test and job submission for both batch and interactive query. You can fully enjoy the advantage of PySpark argparse, and simply keep your configuration and your job-related arguments in the Json based configuration file.

The Spark 2.2 update allows you to benefit the new functionalities and to consume the new libraries and APIs from Spark 2.2 in VSCode. You can create, author and submit a Spark 2.2 PySpark job to Spark 2.2 cluster. With the backward compatibility of Spark 2.2, you can also submit your existing Spark 2.0 and Spark 2.1 PySpark scripts to a Spark 2.2 cluster.

Summary of key new features Argparse support – set up your arguments in Json format. Set up configurations: Go to command palate, choose command HDInsight: Set Configuration.

2. Set up the parameters in the xxx_hdi_settings.json file, including script to cluster, Livy configuration, Spark configuration, etc.

Spark 2.2 Support – Submit PySpark batch

Share

26

Apr

Train an IoT-equipped drone and compete to win at Microsoft Build

Are you ready for Microsoft’s ultimate developer event? You probably already know Microsoft Build, happening May 7–9 in Seattle, Washington, is where you need to be to connect with the experts, discover new tools, and boost your skills around cloud technologies, AI, mixed reality, and more.

Sure, there will be great speakers and tech sessions galore—but did you know you could win a drone?

That’s right, we’re having a drone contest. Participants will compete against fellow conference go-ers on who’s drone can complete the outdoor search and rescue course designed specifically for Microsoft Build. You’ll get hands-on, end-to-end experience with Microsoft’s intelligent cloud platform, Azure IoT Edge, and be eligible to win a DJI Mavic Air drone.

How cool is that?

Here’s how it works

Contestants will create training images using AirSim, an open-source aerial informatics and robotics simulation platform. Then contestants will build and train a realistic AI drone model in the cloud using Custom Vision, and then create a container for AI deployment to Microsoft Azure IoT Edge. This intelligent cloud platform lets you run artificial intelligence at the edge of the cloud, perform analytics, deploy IoT solutions from cloud to edge-enabled devices, and manage them centrally

Share

26

Apr

Get the Azure Quick Start Guide for .NET Developers
Get the Azure Quick Start Guide for .NET Developers

This blog post was co-authored by Barry Luijbregts, Azure MVP.

If you’re a .NET developer, we’re excited to introduce a new resource that can help you learn about Azure: The Azure Quick Start Guide for .NET Developers!

Download this free e-book here.

This guide shows .NET developers how they can start with Azure and get the most out of it. The e-book is also great for .NET developers who already use Azure and want to learn more about which Azure services are available to them, and the tools they can use to develop applications for the platform.

Specifically, the Azure Quick Start Guide for .NET Developers covers:

What Azure can do for you as a .NET developer. The guide describes how Azure can take care of things like automatic scaling, continuous integration/continuous delivery (CI/CD), and much more so you can focus on creating the things that really matter and add value. A catalog that explains what Azure services to use when for: Running your .NET applications in Azure. There are many services that you can use to run your application in Azure. For instance, the guide discusses the differences between running your app in a service like Azure App

Share

25

Apr

The April release of SQL Operations Studio is now available
The April release of SQL Operations Studio is now available

This post is authored by Alan Yu, Program Manager, SQL Server.

We are excited to announce the April release of SQL Operations Studio is now available.

Download SQL Operations Studio and review the Release Notes to get started.

SQL Operations Studio is a data management tool that enables you to work with SQL Server, Azure SQL DB and SQL DW from Windows, macOS and Linux. To learn more, visit our GitHub.

SQL Operations Studio was announced for Public Preview on November 15th at Connect(), and this April release is the fifth major update since the announcement. If you missed the March release announcement can be viewed here.

The April Public Preview release is focused on improving our Extensibility experience with the release of new extensions as well as addressing top Github issues.

Highlights for this build include the following.

Public preview release of SQL Agent extension Added new extensions and improved existing extensions Improvements to Server Reports Extension Release of SSMS Keymap extension Release of AlwaysOn Insights extension Release of MSSQL Instance Insights Release of MSSQL Db Insights Added Visual Studio Code 1.21 platform source code refresh Improved large and protected file support for saving Admin protected and

Share

25

Apr

The April release of SQL Operations Studio is now available
The April release of SQL Operations Studio is now available

This post is authored by Alan Yu, Program Manager, SQL Server.

We are excited to announce the April release of SQL Operations Studio is now available.

Download SQL Operations Studio and review the Release Notes to get started.

SQL Operations Studio is a data management tool that enables you to work with SQL Server, Azure SQL DB and SQL DW from Windows, macOS and Linux. To learn more, visit our GitHub.

SQL Operations Studio was announced for Public Preview on November 15th at Connect(), and this April release is the fifth major update since the announcement. If you missed the March release announcement can be viewed here.

The April Public Preview release is focused on improving our Extensibility experience with the release of new extensions as well as addressing top Github issues.

Highlights for this build include the following.

Public preview release of SQL Agent extension Added new extensions and improved existing extensions Improvements to Server Reports Extension Release of SSMS Keymap extension Release of AlwaysOn Insights extension Release of MSSQL Instance Insights Release of MSSQL Db Insights Added Visual Studio Code 1.21 platform source code refresh Improved large and protected file support for

Share

25

Apr

Transfer Learning for Text using Deep Learning Virtual Machine (DLVM)

This post is by Anusua Trivedi, Data Scientist, and Wee Hyong Tok, Data Scientist Manager, at Microsoft.

Motivation

Modern machine learning models, especially deep neural networks, can often benefit quite significantly from transfer learning. In computer vision, deep convolutional neural networks trained on a large image classification datasets such as ImageNet have proved to be useful for initializing models on other vision tasks, such as object detection (Zeiler and Fergus, 2014).

But how can we leverage the transfer leaning technique for text? In this blog post, we attempt to capture a comprehensive study of existing text transfer learning literature in the research community. We explore eight popular machine reading comprehension (MRC) algorithms (Figure 1). We evaluate and compare six of these papers – BIDAF, DOCQA, ReasoNet, R-NET, SynNet and OpenNMT. We initialize our models, pre-trained on different source question answering (QA) datasets, and show how standard transfer learning can achieve results on a large target corpus. For creating a test corpus, we chose the book Future Computed by Harry Shum and Brad Smith.

We compared the performance of the transfer learning approach for creating a QA system for this book using these pretrained MRC models. For our evaluation scenario, the

Share

25

Apr

Modernize to cloud with Azure SQL Database
Modernize to cloud with Azure SQL Database

Companies are embracing digital transformation by migrating to the cloud. IT organizations are continuously looking for new ways of optimizing how they use the data that fuels their operations. Modernizing to the cloud offers the right mix of operational efficiencies and business enablement to drive continued growth. IT optimization now requires new perspectives to consider:

eliminate time spent managing “long tail” of applications – lift and shift to managed cloud, free up limited IT resources to drive transformation, and migrate business critical apps to the cloud – extend and innovate

One way IT organizations can confront challenges on their transformation journey is by adopting a PaaS (Platform as a Service) solution like Azure SQL Database. Azure SQL Database is a fully-managed relational database-as-a-service that provides the best and most economical cloud destination for your SQL data. You can build data-driven applications and websites in the programming language of your choice, without needing to manage infrastructure.

Within SQL Database, there is a new deployment option called Managed Instance that enables you to quickly and confidently move your SQL Server data (2005 and newer) at scale to the cloud. SQL Database shares a common code base with SQL Server and supports most

Share

25

Apr

Modernize to cloud with Azure SQL Database
Modernize to cloud with Azure SQL Database

Companies are embracing digital transformation by migrating to the cloud. IT organizations are continuously looking for new ways of optimizing how they use the data that fuels their operations. Modernizing to the cloud offers the right mix of operational efficiencies and business enablement to drive continued growth. IT optimization now requires new perspectives to consider:

eliminate time spent managing long tail of applications lift and shift to managed cloud, free up limited IT resources to drive transformation, and migrate business critical apps to the cloud extend and innovate

One way IT organizations can confront challenges on their transformation journey is by adopting a PaaS (Platform as a Service) solution like Azure SQL Database. Azure SQL Database is a fully-managed relational database-as-a-service that provides the best and most economical cloud destination for your SQL data. You can build data-driven applications and websites in the programming language of your choice, without needing to manage infrastructure.

Within SQL Database, there is a new deployment option called Managed Instance that enables you to quickly and confidently move your SQL Server data (2005 and newer) at scale to the cloud. SQL Database shares a common code base with SQL Server and supports

Share

25

Apr

Azure #CosmosDB: Secure, private, compliant
Azure #CosmosDB: Secure, private, compliant

Azure Cosmos DB is Microsoft’s globally distributed, multi-model database. Azure Cosmos DB enables you to elastically and independently scale throughput and storage across any number of Azure’s geographic regions with a single click. It offers throughput, latency, availability, and consistency guarantees with comprehensive service level agreements (SLAs), a feature that no other database service can offer.

A database that holds sensitive data across international borders must meet high standards for security, privacy, and compliance. Additionally, the cloud service provider must anticipate and be ready for new standards, such as the General Data Protection Regulation (GDPR), which will soon govern the collection and use of EU resident’s data. Microsoft has pledged that Azure services will be GDPR compliant by the May 25 implementation date.

Privacy

Microsoft’s cloud privacy policies state that we will use your customer data only to provide the services we have agreed upon, and for purposes that are compatible with providing those services. We do not share your data with our advertiser-supported services, nor do we mine it for marketing or advertising. 

Encryption

Azure Cosmos DB also implements stringent security practices. All the documents, attachments and backups stored in Azure Cosmos DB are encrypted at rest and

Share

25

Apr

Accelerate real-time big data analytics with Spark connector for Microsoft SQL Databases

Apache Spark is a unified analytics engine for large-scale data processing. Today, you can use the built-in JDBC connector to connect to Azure SQL Database or SQL Server to read or write data from Spark jobs.

The Spark connector for Azure SQL Database and SQL Server enables SQL databases, including Azure SQL Database and SQL Server, to act as input data source or output data sink for Spark jobs. It allows you to utilize real-time transactional data in big data analytics and persist results for adhoc queries or reporting.

Compared to the built-in Spark connector, this connector provides the ability to bulk insert data into SQL databases. It can outperform row-by-row insertion with 10x to 20x faster performance. The Spark connector for Azure SQL Databases and SQL Server also supports Azure Active Directory authentication. It allows you to securely connect to your Azure SQL database from Azure Databricks using your AAD account. The Spark connector also provides similar interfaces with the built-in JDBC connector and is easy to migrate your existing Spark jobs to use this new connector.

The Spark connector for Azure SQL Database and SQL Server utilizes the Microsoft JDBC Driver for SQL Server to move data between

Share