02

Apr

Ingest, prepare, and transform using Azure Databricks and Data Factory

Today’s business managers depend heavily on reliable data integration systems that run complex ETL/ELT workflows (extract, transform/load and load/transform data). These workflows allow businesses to ingest data in various forms and shapes from different on-prem/cloud data sources; transform/shape the data and gain actionable insights into data to make important business decisions.

With the general availability of Azure Databricks comes support for doing ETL/ELT with Azure Data Factory. This integration allows you to operationalize ETL/ELT workflows (including analytics workloads in Azure Databricks) using data factory pipelines that do the following:

Ingest data at scale using 70+ on-prem/cloud data sources Prepare and transform (clean, sort, merge, join, etc.) the ingested data in Azure Databricks as a Notebook activity step in data factory pipelines Monitor and manage your E2E workflow

Take a look at a sample data factory pipeline where we are ingesting data from Amazon S3 to Azure Blob, processing the ingested data using a Notebook running in Azure Databricks and moving the processed data in Azure SQL Datawarehouse.

You can parameterize the entire workflow (folder name, file name, etc.) using rich expression support and operationalize by defining a trigger in data factory.

Get started today!

We are excited for you

02

Apr

New Microsoft Azure regions available for Australia and New Zealand

Two new Microsoft Azure regions in Australia are available to customers, making Microsoft the only global provider to deliver cloud services specifically designed to address the requirements of the Australian and New Zealand governments and critical national infrastructure, including banks, utilities, transport and telecommunications.

We build our cloud infrastructure to serve the needs of our customers by delivering innovation globally and listening locally. To support the mission-critical work of crucial organizations in Australia and New Zealand, we’re delivering our global cloud platform through a unique partnership with Canberra Data Centres.

A key to our new Australia Central regions is the ability for customers to deploy their own applications and infrastructure within Canberra Data Centres directly connected via Azure ExpressRoute to Microsoft’s global network. This offers a great deal of flexibility and surety on network performance and security, making these regions ideally suited to the complex challenge of modernising mission-critical applications over time. Australian federal government customers can leverage their Intra Government Communications Network (ICON) for direct connectivity.

With heightened scrutiny of supply chain assurance in government and critical national infrastructure, we are proud to deliver these services in partnership with Australian-owned Canberra Data Centres. They are the premier data centre

30

Mar

Essential tools and services for building mobile apps
Essential tools and services for building mobile apps

This blog post was co-authored by Rajen Kishna, Senior Product Marketing Manager, Azure Marketing – App Development.

Azure, Visual Studio, Xamarin, and Visual Studio App Center give you the flexible, yet robust tools and services to build, test, deploy, and continuously improve Android and iOS apps that your users will love. Use your favorite language and tools, to tap into robust cloud services, and quickly scale to millions of users on demand.

Cloud services for mobile developers

Azure provides many services to help you build cloud-connected mobile apps, including Mobile Backend as a Service, Data, and Artificial Intelligence (outlined below), as well as services to support additional functionality, including Search, Identity, and Communication.

Mobile Backend as a Service

Azure Functions is a serverless backend for your mobile apps, where you just enter your code (whether C#, JavaScript, Python, or any other supported language) and it executes on demand, triggered by events you can define. It is a fast way to get backend code up and running.

If you need a more complete solution, the Mobile Apps feature of Azure App Service provides you with a backend that can be written in C# or Node.js, giving you features such as

30

Mar

BigDL Spark deep learning library VM now available on Microsoft Azure Marketplace

This blog was co-authored by Sergey Ermolin, Intel and Patrick Butler, Microsoft

BigDL deep learning library is a Spark-based framework for creating and deploying deep learning models at scale. While it has previously been deployed on Azure HDInsight and Data Science VM, it is now also available on Azure Marketplace as a fixed VM image, representing a further step by Intel to reduce deployment complexity for users.

Because BigDL is an integral part of Spark, a user does not need to explicitly manage distributed computations. A BigDL application provides high-level control “knobs”, such as the number of compute nodes, cores, and batch size, a BigDL application also leverages stable Spark infrastructure for node communications and resource management during its execution. BigDL applications can be written in either Python or Scala and achieve high performance through both algorithm optimization and taking advantage of close integration with Intel’s Math Kernel Library (MKL).

For more information about BigDL features and capabilities, refer to the GitHub BigDL overview and Intel BigDL framework.

What is the Microsoft Azure Marketplace? The Azure Marketplace is an online applications and services marketplace that enables start-ups, independent software vendors (ISVs), and MSP/SIs to offer their Azure-based solutions or services

30

Mar

Azure Availability Zones now available for the most comprehensive resiliency strategy

We introduced Azure Availability Zones during Microsoft Ignite as part of our continuing expansion of Azure’s support for the most demanding, mission-critical workloads. Today I’m excited to announce the general availability of Availability Zones beginning with select regions in the United States and Europe.

With Availability Zones, in addition to the broadest global coverage, Azure now offers the most comprehensive resiliency strategy in the industry from mitigating rack level failures with availability sets to protecting against large scale events with failover to separate regions. Within a region, Availability Zones increase fault tolerance with physically separated locations, each with independent power, network, and cooling.

For many companies, especially those in regulated industries, who are increasingly moving their mission-critical applications to the cloud, resiliency and business continuity have become a crucial focus. From online commerce systems in retail to customer-facing applications in financial services, the stakes are high for organizations and enterprises to deliver for their customers. Even a minor issue can have a major impact on a company’s brand reputation, customer satisfaction, and bottom line. In this environment, it’s imperative to develop applications with the highest operations standards anchored by a multi-layered resiliency approach.

“Availability Zones give us the combination of

29

Mar

Secure your backups, not just your data!

In today’s digital world where data is the new currency, protecting this data has become more important than ever before. In 2017, attackers had a huge impact on businesses as we saw a large outbreak of ransomware attacks like WannaCry, Petya and Locky. According to a report from MalwareBytes, ransomware detections were up 90 and 93 percent for businesses and consumers respectively in 2017. When a machine gets attacked by ransomware, backups are usually the last line of defense that customers resort to.

With increasing innovations in the ransomware space, attackers are no longer restricting themselves to only data corruption. Backups are becoming the next line of attack for ransomware tools and when you are in a situation with no data or its backups you end up being hostage to the ransomware attacker. This month saw the advent of a new ransomware, Zenis (found by the MalwareHunterTeam) which not only encrypts your files but also purposely deletes your backups.

Combating these attacks requires more than just taking the backup. Taking a backup is only the first step in your protection, it becomes important to safe guard those backups in this model.

Ask these questions to check how secure your

29

Mar

On-premises data gateway March update is now available
On-premises data gateway March update is now available

https://powerbi.microsoft.com/en-us/blog/on-premises-data-gateway-march-update-is-now-available/Source: https://powerbi.microsoft.com/en-us/blog/on-premises-data-gateway-march-update-is-now-available/           We are excited to announce that we have just released the March update for the On-premises data gateway. Here are some of the things that we would like to highlight with this month’s READ MORE

29

Mar

Create Service Fabric Clusters from Visual Studio now available

We heard feedback that the current processes for creating secure clusters were complex and time-consuming. Based on your input, we’ve added the ability to create secure clusters directly from Visual Studio. This update shipped with the latest release of the Service Fabric tooling for Visual Studio 2015 and with Visual Studio 2017 as part of the Azure workload.

One of the main features of the latest release is the experience in Visual Studio for creating a cluster from the “Publish” dialog. This feature will allow you to modify the settings for the cluster, create required supporting resources, and generate the security certificate needed to create a secure cluster all from within Visual Studio. It will also import the security certificate to your local development machine so you can publish and troubleshoot your application in the cluster. This allows you to go directly from developing your service fabric application to testing it in the cloud without having to switch context. When the cluster has been created, Visual Studio will indicate this in the Output window.

You can try out this feature yourself by following the steps in the tutorial “Deploy Your Application to a Cluster“.

Below are screenshots of the new

29

Mar

How to get support for Azure IoT SDK
How to get support for Azure IoT SDK

Azure IoT SDKs make it easy for developers to begin coding and deploy applications for Azure IoT Hub and Azure IoT Hub Device Provisioning Service. The SDKs are production quality open-sourced project with support from Microsoft. If you need support for any step of your development phase, have issues during evaluation or production deployments, we are here to help! There are multiple channels where issues and requests are surfaced. We recommend the following channels for faster turnaround:

Have a feature request for SDKs? We have a User Voice for feature requests. Create new requests, upvote existing ones and help us prioritize! Have a technical question? Ask on Stack Overflow with tag “azure-iot-hub”. We monitor Stack Overflow for questions. Stack Overflow’s high standard for answers will ensure you will get a complete, detailed answer to help others in the community. Need support? If you have an Azure subscription, you can file support requests and get assistance to help you debug. Support requests also make it possible for you to communicate confidential information that would be insecure in a public GitHub issue, such as your hub name or subscription ID. When you file a support request, please make sure you specify

28

Mar

Power BI Developer community March update
Power BI Developer community March update

https://powerbi.microsoft.com/en-us/blog/power-bi-developer-community-march-update/Source: https://powerbi.microsoft.com/en-us/blog/power-bi-developer-community-march-update/           This blog post covers the latest updates for Power BI Developers community- Power BI Embedded and Custom Visuals.