Category Archives : Developer

17

Sep

HDInsight tools for Visual Studio Code: simplifying cluster and Spark job configuration management

We are happy to announce that HDInsight Tools for Visual Studio Code (VS Code) now leverage VS Code built-in user settings and workspace settings to manage HDInsight clusters and Spark job submissions. With this feature, you can manage your linked clusters and set your preferred Azure environment with VS Code user settings. You can also set your default cluster and manage your job submission configurations via VS Code workspace settings.

HDInsight Tools for VS Code can access HDInsight clusters in Azure regions worldwide. To grant more flexible access to HDInsight clusters, you can  access the clusters through your Azure subscriptions, by linking to your HDInsight cluster using your Ambari username and password, or by connecting to an HDInsight Enterprise Security Package Cluster via the domain name and password. All Azure settings and linked HDInsight clusters are kept in VS Code user settings for your future use. The Spark job submission can support up to a hundred parameters to give you the flexibility to maximize cluster computing resources usage, and also allow you to specify the right parameters to optimize your Spark job performance. By leveraging the VS Code workspace setting, you have the flexibility to specify parameters in JSON format.

13

Sep

HDInsight Tools for VSCode: Integrations with Azure Account and HDInsight Explorer

Making it easy for developers to get started on coding has always been our top priority. We are happy to announce that HDInsight Tools for VS Code now integrates with VS Code Azure Account. This new feature makes your Azure HDInsight sign-in experience much easier. For first-time users, the tools put the required sign-in code into the copy buffer and automatically opens the Azure sign-in portal where the user can paste the code and complete the authentication process. For returning users, the tools sign you in automatically. You can quickly start authoring PySpark or Hive jobs, performing data queries, or navigating your Azure resources.

We are also excited to introduce a graphical tree view for the HDInsight Explorer within VS Code. With HDInsight Explorer, data scientists and data developers can navigate HDInsight Hive and Spark clusters across subscriptions and tenants, and browse Azure Data Lake Storage and Blob Storage connected to these HDInsight clusters. Moreover, you can inspect your Hive metadata database and table schema.

Key Customer Benefits Support Azure auto sign-in and improve sign-in experiences via integration with Azure Account extension. Enable multi-tenant support so you can manage your Azure subscription resources across tenants. Gain insights into available HDInsight Spark,

11

Sep

Application Insights improvements for Java and Node.js
Application Insights improvements for Java and Node.js

Did you know that Application Insights supports Java and Node.js? That’s because at Microsoft our mission is to empower every person and every organization on the planet to achieve more. For those of us on the Azure Application Insights team, every person means every developer, DevOps practitioner and site reliability engineer – regardless of the tech stack that they use.

That’s why we’ve been working for over a year now to enable Java and Node.js teams to have a first-class monitoring experience in both their Azure and on-premises environments. So today I’m proud to share with you some of what our team has already accomplished, and I’m excited about the features and improvements that we will be continuing to release over the next several months. But first, let’s talk about Java.

Application Insights for Java

The second version of our Application Insights for Java SDK was released to Maven/Gradle and GitHub earlier this year, and the team has continued to crank out improvements since then, most recently with version 2.1.2. In addition to a myriad of bug fixes, the team has also added support for fixed rate sampling, enhanced support for Log4J, and cross-component telemetry correlation. We also auto

10

Sep

Introducing Azure DevOps
Introducing Azure DevOps

Today we are announcing Azure DevOps. Working with our customers and developers around the world, it’s clear DevOps has become increasingly critical to a team’s success. Azure DevOps captures over 15 years of investment and learnings in providing tools to support software development teams. In the last month, over 80,000 internal Microsoft users and thousands of our customers, in teams both small and large, used these services to ship products to you.

The services we are announcing today span the breadth of the development lifecycle to help developers ship software faster and with higher quality. They represent the most complete offering in the public cloud. Azure DevOps includes:

Azure Pipelines

CI/CD that works with any language, platform, and cloud. Connect to GitHub or any Git repository and deploy continuously. Learn More >

Azure Boards

Powerful work tracking with Kanban boards, backlogs, team dashboards, and custom reporting. Learn more >

Azure Artifacts

Maven, npm, and NuGet package feeds from public and private sources. Learn more >

Azure Repos

Unlimited cloud-hosted private Git repos for your project. Collaborative pull requests, advanced file management, and more. Learn more >

Azure Test Plans

All in one planned and exploratory testing solution. Learn more >

10

Sep

Announcing Azure Pipelines with unlimited CI/CD minutes for open source

With the introduction of Azure DevOps today, we’re offering developers a new CI/CD service called Azure Pipelines that enables you to continuously build, test, and deploy to any platform or cloud. It has cloud-hosted agents for Linux, macOS, and Windows, powerful workflows with native container support, and flexible deployments to Kubernetes, VMs, and serverless environments.

Microsoft is committed to fueling open source software development. Our next step in this journey is to provide the best CI/CD experience for open source projects. Starting today, Azure Pipelines provides unlimited CI/CD minutes and 10 parallel jobs to every open source project for free. All open source projects run on the same infrastructure that our paying customers use. That means you’ll have the same fast performance and high quality of service. Many of the top open source projects are already using Azure Pipelines for CI/CD, such as Atom, Cpython, Pipenv, Tox, Visual Studio Code, and TypeScript – and the list is growing every day.

In the following, you can see Atom running parallel jobs on Linux, macOS, and Windows for its CI.

Azure Pipelines app on GitHub Marketplace

Azure Pipelines has an app in the GitHub Marketplace so it’s easy to get started.

06

Sep

Powerful Debugging Tools for Spark for Azure HDInsight
Powerful Debugging Tools for Spark for Azure HDInsight

Microsoft runs one of the largest big data cluster in the world – internally called “Cosmos”. This runs millions of jobs across hundreds of thousands of servers over multiple Exabytes of data. Being able to run and manage jobs of this scale by developers was a huge challenge. Jobs with hundreds of thousands of vertices are common and to even quickly figure out why a job runs slow or narrow down bottlenecks was a huge challenge. We built powerful tools that graphically show the entire job graph including the various vertex execution times, playback etc. which helped developers greatly. While this was built for our internal language in Cosmos (called Scope), we are working very hard to bring this power to all Spark developers.

Today, we are delighted to announce the Public Preview of the Apache Spark Debugging Toolset for HDInsight for Spark 2.3 cluster and forward.   The default Spark history server user experience is now enhanced in HDInsight with rich information on your spark jobs with powerful interactive visualization of Job Graphs & Data Flows. The new features greatly assist HDInsight Spark developers in job data management, data sampling, job monitoring and job diagnosis.  

Spark History Server Enhancements

28

Aug

Helping Go developers build better cloud apps faster

As GopherCon kicks off today with nearly 2,000 gophers gathering in Denver, we’re excited to share our latest offerings and announcements for Go developers. Since making the Azure SDK for Go generally available earlier this year, our teams have been hard at work building Go tools and services for Azure, Visual Studio Code, and Visual Studio Team Services. We continue to strive to help Go developers build better cloud apps, faster, with an expanding range of services covering the cloud-native workflow:

Write and test code with Visual Studio Code, the most popular editor amongst Go developers. Debug with visual breakpoints, apply Go formatting on save, and use code completions to increase developer speed and productivity. Host your private Git repositories, then integrate and release apps with Visual Studio Team Services and Azure DevOps Projects. Run code, containers, and apps on Azure Kubernetes Service, a fully-managed Kubernetes cluster, or with Azure App Service. Or, use your favorite Linux distribution on Azure Virtual Machines. Store structured data in managed PostgreSQL and MySQL databases, objects in Blob Storage, and cache items in Redis Cache. You can also use Cosmos DB, a globally-replicated, multi-model database that is compatible with MongoDB. Communicate between microservices with

28

Aug

Helping Go developers build better cloud apps faster

As GopherCon kicks off today with nearly 2,000 gophers gathering in Denver, we’re excited to share our latest offerings and announcements for Go developers. Since making the Azure SDK for Go generally available earlier this year, our teams have been hard at work building Go tools and services for Azure, Visual Studio Code, and Visual Studio Team Services. We continue to strive to help Go developers build better cloud apps, faster, with an expanding range of services covering the cloud-native workflow:

Write and test code with Visual Studio Code, the most popular editor amongst Go developers. Debug with visual breakpoints, apply Go formatting on save, and use code completions to increase developer speed and productivity. Host your private Git repositories, then integrate and release apps with Visual Studio Team Services and Azure DevOps Projects. Run code, containers, and apps on Azure Kubernetes Service, a fully-managed Kubernetes cluster, or with Azure App Service. Or, use your favorite Linux distribution on Azure Virtual Machines. Store structured data in managed PostgreSQL and MySQL databases, objects in Blob Storage, and cache items in Redis Cache. You can also use Cosmos DB, a globally-replicated, multi-model database that is compatible with MongoDB. Communicate between microservices with

27

Aug

Turn your whiteboard sketches to working code in seconds with Sketch2Code

User interface design process involves a lot a creativity that starts on a whiteboard where designers share ideas. Once a design is drawn, it is usually captured within a photograph and manually translated into some working HTML wireframe to play within a web browser. This takes efforts and delays the design process. What if a design is refactored on the whiteboard and the browser reflects changes instantly? In that sense, by the end of the session there is a resulting prototype validated between the designer, developer, and customer. Introducing Sketch2Code, a web based solution that uses AI to transform a handwritten user interface design from a picture to a valid HTML markup code.

Let’s understand the process of transforming handwritten image to HTML using Sketch2Code in more details.

First the user uploads an image through the website. A custom vision model predicts what HTML elements are present in the image and their location. A handwritten text recognition service reads the text inside the predicted elements. A layout algorithm uses the spatial information from all the bounding boxes of the predicted elements to generate a grid structure that accommodates all. An HTML generation engine uses all these pieces of information

27

Aug

Turn your whiteboard sketches to working code in seconds with Sketch2Code

User interface design process involves a lot a creativity that starts on a whiteboard where designers share ideas. Once a design is drawn, it is usually captured within a photograph and manually translated into some working HTML wireframe to play within a web browser. This takes efforts and delays the design process. What if a design is refactored on the whiteboard and the browser reflects changes instantly? In that sense, by the end of the session there is a resulting prototype validated between the designer, developer, and customer. Introducing Sketch2Code, a web based solution that uses AI to transform a handwritten user interface design from a picture to a valid HTML markup code.

Let’s understand the process of transforming handwritten image to HTML using Sketch2Code in more details.

First the user uploads an image through the website. A custom vision model predicts what HTML elements are present in the image and their location. A handwritten text recognition service reads the text inside the predicted elements. A layout algorithm uses the spatial information from all the bounding boxes of the predicted elements to generate a grid structure that accommodates all. An HTML generation engine uses all these pieces of information