Azure Databricks provides a fast, easy, and collaborative Apache® Spark™-based analytics platform to accelerate and simplify the process of building big data and AI solutions that drive the business forward, all backed by industry-leading SLAs.
With Azure Databricks, you can set up your Spark environment in minutes and autoscale quickly and easily. You can also apply your existing skills and collaborate on shared projects in an interactive workspace with support for Python, Scala, R, and SQL, as well as data science frameworks and libraries like TensorFlow and PyTorch.
We’re continuously listening to customers and answering questions as we evolve this service. This blog outlines important service announcements that we are proud to deliver for our customers.
Azure Databricks Delta available in Standard and Premium SKUs
Azure Databricks Delta brings new levels of reliability and performance for production workloads based on a number of improvements including transaction support, schema validation, indexing, and data versioning.
Since the preview of Delta was announced, we have received overwhelmingly positive feedback on how it has helped customers build complex pipelines for both batch and streaming data, and simplified ETL pipelines. We are excited to announce that Delta is now available in our Standard SKU offering
You’ve followed an excellent walkthrough and built a solid prototype web app. You run npm start locally and browse to http://localhost and all looks great. Now you’re ready to put your app in the cloud, utilize a managed database and managed authentication, and share a link with all your coworkers and friends. But wait a minute, it looks like you’ll first have to set up cloud pipelines and container images, then brush up on Bash or PowerShell and write a Dockerfile. Getting your app to the cloud is more work than you anticipated. Is there a faster way?
We’re happy to share that yes there is a faster way. When you need to focus on app code you can delegate build and deployment to Azure with App Service web apps. Push your git repo directly to Azure or point to one hosted in GitHub, Azure DevOps, or BitBucket and we’ll take care of building and running your code the way you expect. You may be using this already for your .NET apps; we now support Node.js and Python as well.
IT organizations are under more pressure than ever to do more with less, they are expected to drive competitive advantage and innovation with higher quality while managing smaller teams. This shift to the enterprise cost-to-value equation has created a transformative inflection point across every business domain, underpinned by new enabling technologies and development paradigms. Organizations must now adapt by adopting rapid and strategic transformation while simultaneously working diligently to keep the lights on, and all with the important goal of reducing costs. When done right, three clear benefits appear:
Going cloud native means more than simply offloading datacenter costs and complexity. It means software architecture that is loosely coupled can allow features and bug fixes to be shipped whenever and wherever they need to be by smaller development, QA, release, and production support teams. DevOps is more than a facelift on release management. New pipeline tools coupled with new design and development patterns spur a cultural shift occurring inside IT organizations. DevOps is a revolution in how software is created and supported. Modernizing existing legacy applications and infrastructure doesn’t require a massive, time consuming, and expensive re-write. Through the judicious application of microservices, new development, and delivery methodologies, the elephant
Following the launch of Azure DevOps in September, we’re pleased to announce the official release of Azure DevOps Server 2019! Previously known as Team Foundation Server (TFS), Azure DevOps Server 2019 brings the power of Azure DevOps into your dedicated environment. You can install Azure DevOps Server 2019 into any datacenter or sovereign, and determine when to apply updates.
About Azure DevOps Server
Azure DevOps includes developer collaboration tools which can be used together or independently, including Azure Boards (Work), Azure Repos (Code), Azure Pipelines (Build and Release), Azure Test Plans (Test), and Azure Artifacts (Packages). These tools support all popular programming languages, any platform (including macOS, Linux, and Windows) or cloud, as well as on-premises environments. Like with TFS, you control where you install Azure DevOps Server and when you apply updates. If you prefer to let us manage, use Azure DevOps Services which is available in more geographic regions than any other cloud hosted developer collaboration service.
Download Azure DevOps Server 2019 What’s new?
The release notes describe the major updates from TFS 2018 to Azure DevOps Server 2019, but my key highlights include:
The new navigation, which enables users to easily navigate between services, is more
Today we are excited to launch two new key capabilities to Azure Firewall.
Threat intelligence based filtering Service tags filtering
Azure Firewall is a cloud native firewall-as-a-service offering which enables customers to centrally govern all their traffic flows using a DevOps approach. The service supports both application (such as *.github.com), and network level filtering rules. It is highly available and auto scales as your traffic grows.
Threat intelligence based filtering (preview)
Microsoft has a rich signal of both internal threat intelligence data, as well as third party sourced data. Our vast team of data scientists and cybersecurity experts are constantly mining this data to create a high confidence list of known malicious IP addresses and domains. Azure firewall can now be configured to alert and deny traffic to and from known malicious IP addresses and domains in near real-time. The IP addresses and domains are sourced from the Microsoft Threat Intelligence feed. The Microsoft Intelligent Security Graph powers Microsoft Threat Intelligence and provides security in multiple Microsoft products and services, including Azure Security Center and Azure Sentinel.
Threat intelligence-based filtering is default-enabled in alert mode for all Azure Firewall deployments, providing logging of all matching indicators. Customers can adjust behavior
Azure Database for PostgreSQL provides a fully managed, enterprise-ready community PostgreSQL database as a service. The PostgreSQL community edition helps you easily migrate existing apps to the cloud or develop cloud-native applications, using languages and frameworks of your choice. The service offers industry leading innovations such as built-in high availability, backed with 99.99 percent SLA, without the need to set up replicas and enabling customers to save over two times the cost. The capability also allows customers to scale compute up or down in seconds, helping you easily adjust to changes in workload demands.
Additionally, built-in intelligent features such as Query Performance Insight and performance recommendations help customers further lower their total cost of ownership (TCO) by providing customized recommendations and insights to optimize the performance of their Postgres databases. These benefits coupled with unparalleled security and compliance, Microsoft Azure’s industry leading global reach, and Azure IP Advantage, empower customers to focus on their business and applications rather than the database.
As part of the broader Postgres community, our aim is to contribute to and partner with others in the community to bring new features to Azure Database for PostgreSQL users. You can now take advantage of the Hasura GraphQL
With Azure DevOps Projects we want to make it is easy for you to set up a fully functional DevOps pipeline tailored to the development language and application platform you want to leverage.
We have been making continuous enhancements to Azure DevOps Projects and in the latest deployment now available to all customers, we have added support for Azure Cosmos DB and Azure Functions as target destinations for your application. This builds on the existing Azure App Service, Azure SQL Database, and Azure Kubernetes Service (AKS) support.
The support of Azure Cosmos DB in Azure DevOps Projects means that you will now be able to create a skeleton two tier Node.js application backed by Azure Cosmos DB in just a few clicks. Azure DevOps Projects creates all the scaffolding for your pipeline to give you everything you need to develop, deploy, and monitor your application including:
A Git code repository hosted in Azure Repos with a skeleton Node.js application A CI/CD pipeline in Azure Pipelines for deploying the database tier to Azure Cosmos DB and the web-tier on Azure Web Apps for containers, Azure Kubernetes Service, or as a Windows Web App in Azure Provisioning all the Azure resources in
One of the benefits of moving to the cloud is that you, our customer, don’t need to deal with hardware maintenance and repairs; you can focus your time on your business applications. Azure continuously monitors for hardware that shows signs of degradation or potential failure. When these conditions are detected, Azure will attempt to live migrate your virtual machines (VMs). If live migration isn’t possible, Azure will automatically redeploy VMs to a healthy machine. If you have a disaster recovery setup, which is highly recommended, the impact of this redeployment will be minimal. However, a redeployment to a healthy machine may be problematic for some applications that can’t tolerate disruption. We’ve received feedback that in this situation, when possible, customers prefer to control the time the redeployment to a healthy machine will occur.
We introduced Scheduled Events in Azure as a programmatic way to notify your VMs and act on upcoming maintenance events such as a live migration, redeployment, reboot, etc. Upon receiving the scheduled event, customers can take actions such as failover, saving state, drain sessions in the VMs, schedule a time for manual maintenance, notify customers, etc. We’re excited to announce that Scheduled Events will now be triggered