Govern your data wherever it resides with Azure Purview

By 2025, the volume of data generated per year is expected to grow to 175 zettabytes (ZB). Yet, most organizations aren't prepared to properly govern all this data. As a data professional in charge of managing this influx of data, are you prepared to answer questions like where did my organization’s data come from? Is the access and usage of that data by different departments compliant? And what regulatory requirements govern that data?

Read More: MB-220: Microsoft Dynamics 365 Marketing

The number and types of data sources are also multiplying. Data may be stored on-premises, across different public clouds, and in a multitude of software-as-a-service (SaaS) applications. Tracking all this data manually in Excel or in another spreadsheet will become increasingly difficult and cumbersome. An organization that does not have an automated and holistic understanding of its entire data estate across these different sources will find it increasingly challenging to adapt quickly to changing market and regulatory conditions.

The need for a comprehensive data management and data governance service has never been stronger. That is why we built Azure Purview, a unified data governance solution that helps organizations achieve a complete understanding of their data regardless of if it’s housed on-premises in services like SQL Server and Oracle, in different clouds like Amazon Web Services (AWS) S3, or in SaaS applications like Salesforce. You can easily create a unified map of your data assets and their relationships with automated data discovery and sensitive data classification, get insight into the location and movement of sensitive data across your hybrid landscape, and empower data consumers to find valuable data through a data catalog.

Azure Purview further extends your existing Microsoft and Azure investments by allowing you to quickly and easily add data governance capabilities to your current workloads. For analytics, Azure Purview is deeply integrated with Azure Synapse Analytics so you can search and interact with Azure Purview assets from within the Azure Synapse Studio. Azure Purview captures the lineage relationships between data assets all the way from raw data to business insights. These relationships are captured automatically and kept up to date with turnkey integrations of Azure Purview with SQL Server, Microsoft Power BI, Azure SQL, and more. Additionally, Azure Purview’s automated data classification uses more than 200 prebuilt and custom classifiers to detect sensitive data types such as business terms, government IDs, names, location data, and more. Furthermore, Azure Purview’s integration with Microsoft Information Protection ensures that sensitivity labels you have already defined in the Microsoft 365 Compliance Center can be applied consistently.

It's exciting to see customers rapidly adopting Azure Purview for their data management needs. London Heathrow Airport, for example, has already achieved tremendous success with Azure Purview. To keep weather disruptions, aircraft delays, and global pandemics from disrupting operations and to identify growth opportunities, Heathrow needed an effectively managed data estate. Heathrow’s data scientists use Azure Purview to better understand data fields and give more people the opportunity to analyze data and extract insights. The airport’s data teams also use lineage data in Azure Purview to find correlations between datasets. This is critical to determining how each airport service affects broader operations.

“We used to have a lot of data about things we generally already understood but now that we’ve adopted Azure Purview, we have insight into the unknown, meaning we can aggregate multiple data sources in a more user-friendly way to discover where we can create efficiencies and make better predictions.”

—Dave Draffin, Azure Cloud and Data Architect, Heathrow Airport

Today, we are excited to announce that Azure Purview is generally available. Every organization can now build a unified data governance solution to maximize the value of their data in the cloud.

Source: microsoft.com

Continue reading

5 Tips to Prepare for Microsoft Azure Cloud Certification Exam

For the last few years, the cloud has become a crucial part of almost every IT organization due to various benefits that come along with it. There are various IT giants that offer cloud services such as Google, Amazon, Microsoft, and various others. However, like all other tech-related fields, Microsoft is leading the cloud domain also with its most preferable cloud platform – Azure. Meanwhile, Microsoft offers an Azure Certification that can help you to get recognized as a cloud computing professional and will further land you up a high-paying job as well. Statistically, Microsoft Azure Certified professionals earn 15-20% more in comparison to the uncertified ones.

Before moving further, let’s take a brief introduction to the Azure Cloud Platform. Microsoft Azure, released in 2010, is a cloud computing platform that interacts with various services such as networking, storage, Internet of Things, development, etc. Azure offers various features such as less operational cost, easier implementation, better security, and many more. Now, get back to the point that how can you start to prepare for the Azure Cloud Certification. Here, in this article, we will discuss a roadmap regarding the Azure Cloud Certification with appropriate strategies and approaches. 

1. Choose the Relevant Azure Certification

The first and foremost task you need to do is select the right certification for yourself as per your goals. Yes, Microsoft offers various Azure exams and certifications as per the industry’s requirements. The certifications are categorized based on various levels – Fundamental, Associate, and Expert. The major Microsoft Azure Certifications as per the work-domain are listed below: 

◉ Microsoft Certified Azure Fundamentals (AZ-900)

◉ Microsoft Certified Azure Administrator (AZ-103)

◉ Microsoft Certified Azure Developer (AZ-203)

◉ Microsoft Certified Azure Data Engineer Associate (DP-200, DP-201)

◉ Microsoft Certified Azure Security Engineer (AZ-500)

◉ Microsoft Certified Azure AI Engineer Associate (AI-100)

◉ Microsoft Certified Azure Data Scientist (DP-100)

◉ Microsoft Certified Azure Solutions Architect (AZ-300, AZ-301)

◉ Microsoft Certified Azure DevOps Engineer (AZ-400)

◉ Microsoft Certified Azure for SAP Workloads (AZ-120)

◉ Microsoft Certified Azure IoT Developer (AZ-220)

Hence, you need to get familiar or aware of all the certifications and then identify the particular one to meet your career goals. 

2. Get Familiar with Exam Format

Once you’ll be aware of all the Azure Certifications, then you are required to understand the exam format and procedure. It will help you to analyze what type of questions are asked in the exam and make it easier for you to pass with flying colors. You can go through various standard resources over the web to get the right content. Meanwhile, you are recommended to read the Microsoft Azure Official Documentation provided on the certification page. In this documentation, you’ll get all the information regarding every topic and other exam prerequisites and resources. It will surely help you to get the right learning path and guidance. Moreover, each certification exam has its own set of questions according to its domain such as Development, Data Science, etc., hence you need to prepare as per the particular certification. 

3. Start the Learning Process

After getting familiar with all the exam-related schemes, now it’s time to start the learning process. You can start to learn the basics from various resources (both paid and free) available over the web. Apart from the self-learning process, you can also join various training programs offered for specifically Microsoft Azure training. You can also opt for Microsoft Learn, a free learning platform by Microsoft, to get some quality resources and make your preparation better.

Meanwhile, this Microsoft Learning Platform can be proved as the one-stop solution for all the exam-related queries. Moreover, you can join Instructor-led Training to get more exposure and efficient learning. Also, you are recommended to go through some standard books that will help you to learn and prepare more for the certification exam.

4. Get Some Hands-on Experience

Needless to say, the best way to learn anything is to learn by doing! You’re required to get some hands-on experience with the technology to have practical knowledge about Microsoft Azure. Apart from reading books, attending lectures, watching tutorials, etc. you are recommended to try all your understood concepts out on a real system to make it more effective. Meanwhile, Microsoft also offers you to create Azure free account and various hands-on labs where you can get to practice with the latest cloud products and services in a live environment and can enhance your cloud skills at zero cost. It will make you more proficient with the technology as you can practice your skills multiple times with real-world scenarios. 

5. Must Solve Practice Tests

Regardless of the type of exam or certification, practice papers are always considered as one of the best ways to assess your knowledge and performance. You are recommended to solve official practice papers offered by Microsoft or can prepare from other reliable resources. It will surely help you to cover all the important domains for the perspective of the Microsoft Azure Certification exam. Also, it will help you to get familiar with the real exam environment such as time limit, type of questions, etc. 

So, these are the strategies that you need to follow to prepare for the Azure Cloud Certification. Indeed, Mircosoft Azure is the leading player in the cloud industry and as per its rapid growth, it is the best skill to learn to make a career in the tech industry.

Source: geeksforgeeks.org

Continue reading

GAIA-X gets new support with European Eclipse Data Connector

Data has an increasingly important role in strengthening business models and offering improved public services but much of its potential remains untapped. Data sharing is an essential element to the promise of unlocking new business opportunities and broader economic growth for all industries. Europe has been a leading voice for years on the need to expand industry participation in data sharing. Business-to-business (B2B) data sharing is also a foundational concept of GAIA-X, an initiative to create a federated data infrastructure. Last month a coalition of leading European organizations announced the Eclipse Dataspace Connector (EDC) which is a European open-source project that enables multicloud, policy-based B2B data sharing. The EDC was showcased at the GAIA-X Hackathon in Munich on August 30 – 31, 2021.

Multicloud data sharing across organizations

Transmitting data from one organization to another is a technical problem that has been solved in innumerable ways. What is different about this project? Think about a large manufacturing company with its design organization using engineering software running on AWS, its factory systems running on Azure, its supply chain management systems hosted by SAP, and its custom-built cloud services for sales and marketing hosted by Deutsche Telekom. On top of that, they want to exchange data in their supply chain with their partners who have equally complicated systems. Each company may want to share data, but how to do that across multiple clouds and on-premises systems while remaining respectful of data sharing policies as well as privacy and security laws? In addition, what identity management system can support data sovereignty and federation requirements?

The EDC is made up of open-source components that enable multicloud, policy-based, federated data sharing based on European data sovereignty principles. Every party associated with the sharing and consumption of data needs to have a valid digital identity that provides them the level of sovereignty they desire based on their organizational requirements. Each party also needs to be able to declare the policies under which they are willing to exchange data and be able to enforce them. And ultimately, the sharing of data needs to be secure and efficient.

Enabling data innovation for modern business

Banks develop and provide cloud financial services, media content providers have cloud streaming services, airlines offer cloud ticketing and flight services—most modern businesses are cloud providers to their customers. Thus, every business—every cloud provider—will need to enable trusted data sharing. There are other problems that will need to be solved such as data semantics and the internal governance practices associated with the business decisions for sharing or receiving data. But putting the first layers of this complex structure in place is a significant step forward for enabling data innovation-based growth.

The fact that it is a European solution, governed by a European open-source software foundation, and led by organizations committed to protecting European values is critical to its acceptance in the GAIA-X process. Microsoft is pleased to support this project along with Fraunhofer Institute for Software and Systems Engineering ISST, Daimler TSS, BMW Group, Deutsche Telekom, Amazon AWS, SAP, Bosch, HPE, ZF Friedrichshafen and GAIA-X AISBL as well as the International Data Spaces Association.

Source: microsoft.com

Continue reading

Enable industrial device connectivity with thousands of partner-provided Azure IoT Plug and Play device profiles

Simplifying industrial IoT device connectivity has been top of mind for many of us at Microsoft. Recently we dramatically expanded our ability to support faster operational technology (OT) to cloud connectivity for up to 80 percent of all industrial equipment as part of our collaboration with key players in our partner ecosystem. This is big news—and something that everyone working on or selling IoT solutions should know as many companies struggle with connectivity implementation. Simply said, connecting existing industrial assets to the cloud just got a lot easier and you’ll save thousands of dollars in labor costs while dramatically accelerating your IoT project.

As a result of our partner collaborations and the latest Azure IoT technologies, we are able to offer rapid connectivity to tens of thousands of popular industrial IoT device profiles for use in industrial spaces. Providing connectivity for all types of sensors, devices, and machines—and expanding the opportunity for IoT implementation for everyone—as industrial grade machinery is present across many sectors and industries in the modern world.

The benefit of connected assets

As anyone who has delved into IoT knows, connecting devices and sensors are just the start of delivering on the value of IoT. It’s a time-consuming and often complex first step, but one that carries an enormous payoff. While digital transformation benefits for connected assets can be found in all sectors, industrial manufacturing, in particular, leads the way. For these companies, IoT solutions can reduce downtimes by 20–25 percent due to predictive maintenance and location monitoring—saving millions each year. And that’s before the operational efficiencies, business insights, and possibly new revenue streams from IoT big data and AI are applied.

Connectivity made easy with IoT Plug and Play

With the introduction of IoT Plug and Play, we’ve made the job of connecting devices to Azure IoT a lot easier, as it simplifies and democratizes IoT for our customers, partners, and device builders. This technology helps IT practitioners integrate smart devices into their solutions with minimal configuration. At its core, IoT Plug and Play uses a device model to describe the device’s capabilities using the Digital Twin Definition Language (DTDL). Once turned on, IoT Plug and Play-enabled devices quickly establish themselves as part of an IoT application in the cloud. Customers and partners can further accelerate projects with customizable dashboards and application templates in IoT Central.

Thousands of devices are IoT Plug and Play-enabled

For new IoT solutions and applications, our Azure Device Catalog currently offers hundreds of IoT Plug and Play-enabled certified devices. These devices can be easily connected to Azure IoT services such as Azure IoT, Azure IoT Hub, and Azure IoT Central.

The vast majority of the world’s industrial IoT projects are targeted towards existing assets and environments with a myriad IoT-capable sensors and devices that are already in use. For this reason, we are further expanding the use of IoT Plug and Play technology to these scenarios. And we are happy to share more about those partners who are helping us bring IoT Plug and Play technology to tens of thousands more industrial devices.

The two key partners we would like to highlight are:

CloudRail and their fully managed solution acquire data from industrial environments, pre-process it locally, and sends it to Azure IoT. CloudRail uses industry standards like OPC-UA to connect modern devices, while old machines are retrofitted with secondary sensors. Their database of over 12,000 sensor Azure Plug and Play-enabled definitions in combination with automated data normalization and device provisioning reduces the setup time for connecting a machine to the cloud from weeks to just hours. Using “CloudRail DMC”, their cloud-based device management solution, customers can securely run and manage a single connected machine or thousands.

Omnio offers a solution called Omnio Edge, which provides software-configured connectors for industrial assets Omnio Edge follows the IoT Plug and Play patterns to enable more than 60,000 device configurations. Their solution delivers a seamless experience for industrial legacy assets to be connected to cloud-based applications providing semantically coherent data labels, same units, same scaling, and synchronized timestamps.

With the combined power of Azure IoT Plug and Play and our partners, Azure IoT offerings at the edge are now more comprehensive and enabling—providing the ability to connect almost any type of industrial equipment while leveraging the full capabilities of the device spectrum. With these collaborations, Microsoft is enabling our customers and partners to deploy intelligent cloud workloads at the edge in a way that optimally balances power and simplicity.

Source: microsoft.com

Continue reading

Azure DDoS Protection—2021 Q1 and Q2 DDoS attack trends

In our 2020 retrospective, we highlighted shifts in the active cyberthreat landscape. With the huge surge in internet activity, particularly with the onset of the COVID-19 pandemic, Distributed Denial-of-Service (DDoS) attacks have ramped up significantly in both volume and complexity.

We continue to see such trends in the first half of the calendar year 2021. With the increased usage and supply of IoT devices as well as cryptocurrency like Bitcoin (which is hard to trace), we see a rise in ransomware and ransom DDoS attacks1, whose victims included Mexico’s national lottery sites2 as well as Bitcoin.org3, among others. The online gaming vertical continues to be a very attractive target of DDoS attacks, as experienced by Respawn Entertainment throughout the past few months who suffered significant disruptions to Titanfall’s gameplay4. More industries are being targeted, particularly higher education5, healthcare6, telecoms7, and public sectors. In May, a DDoS attack on Belnet, the internet service provider (ISP) for Belgium’s public sector, took down the websites of more than 200 organizations8 that included the Belgian government, parliament, universities, and research institutes.

Read More: AZ-900: Microsoft Azure Fundamentals

At Microsoft, the Azure DDoS Protection team protects every property in Microsoft and the entire Azure infrastructure. In this review, we share trends and insights into DDoS attacks we observed and mitigated throughout the first half of 2021.

Number of attacks

During the first half of 2021, we witnessed a sharp increase in DDoS attacks per day. Compared to Q4 of 2020, the average daily number of attack mitigations in the first half of 2021 increased by 25 percent. We mitigated an average of 1,392 attacks per day, the maximum reaching 2,043 attacks on May 24, 2021. In total, we mitigated upwards of 251,944 unique attacks against our global infrastructure during the first half of 2021.

Attack bandwidth

In the first half of 2021, the largest attack bandwidth reported on Azure resources was 625 Gbps, down from 1 Tbps in Q3 of 2020. However, the average attack size increased by 30 percent, from 250 Gbps to 325 Gbps.

Attack duration

As with 2020, we continue to see that most attacks are short-lived, with 74 percent being 30 minutes or less and 87 percent being one hour or less.

The proportion of short-lived attacks remained largely consistent across the first half of 2021. Seventy-six percent of attacks in Q1 of 2021 were 30 minutes or less duration, compared to 73 percent of attacks in Q2.

This year, we see more advanced techniques being employed by attackers, such as recycling IPs to launch short-burst attacks.

Top attack vectors

Compared to 2020, we see a rise in volumetric transmission control protocol (TCP) flood attacks. The first half of 2021 was characterized by a shift towards attacks against web applications, whereby TCP attacks are at 54 percent of all attack vectors (mainly TCP, SYN, SYN-ACK, and ACK floods).

User datagram protocol (UDP) attacks were the top vector in 2020 comprising more than 65 percent of all attacks. In the first half of 2021, they decreased to 39 percent of overall attack vectors, with amplification attacks accounting for 11 percent of total attacks.

While UDP attacks comprised the majority of attack vectors in Q1 of 2021, TCP overtook UDP as the top vector in Q2. From Q1 to Q2, the proportion of UDP dropped from 44 percent to 33 percent, while the proportion of TCP increased from 48 percent to 60 percent.

Top attacked regions

Similar to 2020, the United States (59 percent), Europe (19 percent), and East Asia (6 percent) were the most attacked regions due to the concentration of financial services and gaming industries in these regions. As financial institutions tend to rely on TCP workloads, it makes sense that these regions have been harder hit in the first half of 2021, given the rise in TCP flood attacks.

The United Arab Emirates has been increasingly hit by DDoS attacks on government, private, oil and gas, telecommunications, and healthcare sectors. The region was particularly hit hard in January, with 70 percent of its total attacks concentrated in that month.

As with 2020, East Asia (Hong Kong) remains a popular target of DDoS attacks, with 41 percent of its total attacks occurring in May and June. In June, we saw a huge uptick in SYN, SYN-ACK, and ACK flood attacks in the region and we mitigated multiple VIPs totaling up to 225M PPS of traffic.

Top attack sources

The top source countries to generate DDoS attacks were the United States (29 percent), China (28 percent), Russia (3 percent), and followed by South Korea (3 percent). Unknown sources (7 percent) indicate that the autonomous system numbers (ASNs) were either garbage, spoofed, or private ASNs that we could not translate.

New attack types observed

New zero-day attack vectors that we observed and defended against:

Microsoft Windows RDP abuse on UDP 3389

In January, Microsoft Windows servers with Remote Desktop Protocol (RDP) enabled on UDP/3389 were being abused to launch UDP amplification attacks. These attacks had an amplification ratio of 85.9:1 and a peak at ~750 Gbps.

Reflection and amplification DDoS attack mitigation.

D/TLS exploit reflection attack

In February, we saw instances of the Datagram Transport Layer Security (D/TLS) attack vector. Video streaming and gaming customers were getting hit by D/TLS refection attacks which exploited UDP source port 443.

~4,300 publicly reachable servers are posing a new DDoS hazard to the Internet—Ars Technica.

Plex Media server abuse

In June, we saw an emerging reflection attack iteration for the Simple Service Delivery Protocol (SSDP). This protocol normally uses source port 1900, and the new mutation was either on source port 32414 or 32410, also known as Plex Media Simple Service Delivery Protocol (PMSSDP).

Plex Media servers are being abused for DDoS attacks—ZDNet.

Protect your workloads with Azure DDoS Protection Standard

The world continues to be heavily dependent on digital services. We see a growing reliance on cloud-computing services, across sectors from financial services to healthcare. Cyberthreats are pervasive and ever-evolving, and it is always crucial for businesses to develop a robust DDoS response strategy and be proactive in protecting their public workloads.

Azure DDoS Protection Standard provides enhanced DDoS mitigation features to defend against DDoS attacks. It is automatically tuned to protect all public IP addresses in virtual networks. Protection is simple to enable on any new or existing virtual network and does not require any application or resource changes. Our recently released Azure built-in policies allow for better management of network security compliance by providing great ease of onboarding across all your virtual network resources and configuration of logs.

With the recent rise of web application DDoS attacks, it is best to use DDoS Protection Standard alongside Application Gateway web application firewall (WAF), or a third-party web application firewall deployed in a virtual network with a public IP, for comprehensive protection. This also works if you are using Azure Front Door alongside Application Gateway, or if your backend resources are in your on-premises environment. Additionally, when Application Gateway with WAF is deployed in a DDoS protected virtual network, there are no additional charges for WAF—you pay for the Application Gateway at the lower non-WAF rate.

If you have a web application that receives traffic from the Internet and is deployed regionally, you can host your application behind Application Gateway, then protect it with a WAF against Layer 7 web attacks and enable DDoS Protection Standard on the virtual network which contains the Application Gateway and WAF. The backend origins of your application will be in your on-premises environment, which is connected over the virtual private network (VPN). DDoS Protection Standard will defend your application by mitigating bad traffic and routing the supposed clean traffic to your application.

Azure DDoS Protection Standard offers the following key benefits:

◉ Backed by the Microsoft global network: We bring massive DDoS mitigation capacity to every Azure region, scrubbing traffic at the Azure network edge before it can impact the availability of your services. If we identify that the attack volume is significant, we leverage the global scale of Azure to defend the attack from where it is originating.

◉ Cost protection: DDoS attacks often trigger the automatic scale-out of the service running in Azure. This could lead to a significant increase in network bandwidth, the scaling-up of the virtual machine count, or both. In the event of an attack, you can receive Azure credits for any scale-out of resources, so you do not have to worry about setting your application to auto-scale or paying the excess cost for egress data transfer.

◉ DDoS Rapid Response: During an active attack or after an attack, you can engage the DDoS Protection Rapid Response team for help with attack investigation and specialized support. The DDoS Protection Rapid Response team follows the Azure Rapid Response support model.

◉ Rich attack analytics: With DDoS attack analytics, you can view metrics, configure alerts, and get detailed mitigation reports and flow logs that give you detailed visibility into attack traffic and actions we are taking to mitigate a DDoS attack. You can also connect your logs to Azure Sentinel, and view and analyze your data in workbooks. With Azure Security Center, we offer alerts whenever your public IP is under a DDoS attack, or if the attack has been mitigated by us, and we also offer recommendations to enable DDoS Standard for your unprotected virtual networks.

Source: microsoft.com

Continue reading

Video analytics at the edge, an ideal technology for 5G cloud monetization

Creating a programmable software infrastructure for telecommunication operations promises to reduce both the capital expenditure (CAPEX) and the operational expenses (OPEX) of the 5G telecommunications operators. What is exciting to many of us who work in this space is that the convergence of telecommunications, the cloud, and edge infrastructures will open up opportunities for new innovations and revenue for both the telecommunications industry and the cloud ecosystem.

In this blog, we focus on video, the dominant traffic type on the internet since the introduction of 4G networks. With 5G, not only will the volume of video traffic increase, but there will also be many new solutions for industries, from retail to manufacturing to healthcare and forest monitoring, infusing deep learning and AI for video analytics scenarios. The symbiotic evolution of video analytics and edge computing provides opportunities for operators to offer new services which they can monetize with their customers.

Video analytics, edge computing, and 5G  

Our first public disclosure of real-time video analytics was when we characterized it as the killer app for edge computing. Collaborating with the City of Bellevue, Washington we pursued their Vision Zero initiative to conduct a pilot study for live traffic congestion and safety at the cities’ camera-equipped major traffic intersections. We hosted a traffic dashboard powered by our video analytics solution to detect automobiles, pedestrians, and bicyclists at Bellevue’s Traffic Management Center (July 2017 to November 2018). The dashboard also helped Bellevue transportation planners understand traffic patterns over extended periods of time and led to the creation and assessment of a bicycle lane on one of its main streets. The project was a big success and the city won multiple national awards for its vision and pilots on incorporating video analytics for traffic management.

In parallel, as 5G began emerging, telecom operators began making large investments in their network infrastructure, with the lion’s share of the network capacity planned for video traffic. Interestingly, the aspiration of operators to use their infrastructure for digitization of various industries is beautifully aligned with Microsoft’s own investments in edge computing and video analytics. Edge computing is the catalyst that is leading to the convergence of the two infrastructures, and video analytics is the perfect service 5G operators can host on their edge computing servers.

The figure illustrates that the investments of 5G operators and Microsoft are aligned.

A revenue opportunity for 5G operators and Microsoft partners with new-age applications

There are several good examples we can envision for 5G operators to monetize video analytics services. Consider traffic monitoring and accident-avoidance solutions in smart cities, similar to what we implemented in our Vision Zero work with the City of Bellevue. A related application is integrating with self-driving cars with real-time video analysis from their cameras. Furthermore, consider modern smart enterprises where end-to-end experiences with video analytics and mixed reality are a natural component of private 5G network solutions. Additional examples include managing machines and robots in connected factories, or customer demands and services in retail stores and restaurants, or pedestrian traffic in sports arenas. In all these cases, 5G operators, in partnership with System Integrators (SIs), can use Azure edge computing products and Azure Video Analyzer to provide innovative solutions. 

The figure illustrates the coming together of 5G Operators, Azure edge video services, and Systems Integrators (SIs) to offer future video analytics services to various industries.

Microsoft already has an ergonomic, untethered holographic device featuring enterprise-ready applications that increase user productivity across industries from manufacturing to education, the Microsoft HoloLens. Looking at the not too distant future, offloading video processing from HoloLens to a nearby Azure Edge over a low-latency, high-bandwidth 5G network represents yet another example of how operators can offer new products. Microsoft cloud gaming platform, xCloud, also comes to mind as it delivers next-generation global game streaming. Leveraging the power of low-latency, high bandwidth 5G networks, alongside live video analytics on edge devices, operators can support a significantly enhanced gaming experience.

How Microsoft’s advanced technology makes all this possible

Microsoft has invested many years into developing large-scale live video analytics systems. We have published research papers with substantial platform advances, have developed related products, and open-source technologies. For instance, Microsoft Rocket is an open-source platform, which enables the easy construction of video pipelines for efficient processing of video streams. Its cascaded video pipeline, when combined with Azure Video Analyzer, makes it easy and affordable for developers to build video analytics applications into IoT solutions. The combination of Azure Video Analyzer and Microsoft Rocket along with Azure Arc enables easy configuration of resource-accuracy tradeoffs and orchestration over a distributed edge-cloud hierarchy. Azure Video Analyzer and Microsoft Rocket achieve an order-of-magnitude improvement in throughput per edge core for video analytics without compromising accuracy, lowering the total cost of ownership (TCO) at the edge.

Privacy preservation has been a central pillar of Microsoft Rocket’s goal to democratize video analytics. We embraced edge computing as a natural ally to preserve privacy with techniques to transform video at the edge that prevented leakage of personal information in the analytics. We also rely on secure hardware to protect against snooping and provide confidentiality during the analytics.

Specific to 5G, we have also incorporated extensive network monitoring and adaptations for fault tolerance and load balancing in the video processing pipeline to handle dynamic network conditions that are inevitable in all wireless networks. Our system, which we refer to as edge video services (EVS), works well with heterogeneous edge hierarchies supporting diverse hardware. For this, we created new technology for computation partitioning and an inter-edge orchestrator. EVS partitions the computation to make the best use of the available hardware at the edge and cloud infrastructure, while also co-existing with other workloads on the edge, as captured in the figure below.

The figure illustrates how the edge video services (EVS) partitions the computation to make best use of the available hardware at the edge and cloud infrastructure, while also co-existing with other workloads on the edge.

Tailoring Azure Video Analyzer for real-world operation over 5G networks

We have been evolving our systems through pilots with operators and 5G network equipment vendors. Our engagement with Telstra, a prominent Australian telecom operator, is an example of an operator who wants to light up EVS. As part of Telstra’s mission to build a connected future for everyone, Telstra adopted Azure Video Analyzer and Microsoft Rocket along with Azure Stack Edge and Azure Percept preview. By intelligently distributing AI across edges, the amount of data processed was reduced by 50 times, thus leading to better utilization of Telstra’s 5G network. Telstra is developing scalable, cost-efficient solutions that help its customers optimize traffic flow and increase construction safety.

In our collaboration with Fujitsu, we trialed a private 5G solution for monitoring parking lots by analyzing video feeds from Fujitsu’s smart wireless cameras. In order to build autonomous networks with minimal complexity, Fujitsu adapted Microsoft Rocket into their 5G infrastructure where Microsoft Rocket and Fujitsu’s RAN containers execute alongside each other on an Azure Stack Edge. Microsoft Rocket substantially lowered the compute and network demands while providing low-latency and accurate visualization of the parking lot’s occupancy.

In another example, in collaboration with academic colleagues at Princeton University, Microsoft developed the world’s first 5G-based multi-hop camera network. This relay-based camera network uses edge servers and cameras fitted with WiGig radios to create a fully connected millimeter-wave (mmWave) network. This then allows for efficient streaming and analysis of live video in areas where direct line of sight to the base stations is often problematic, as shown in this demo video.

Looking to the future

In the years to come, people around the world will access and use 5G networks every day. 5G networks will continue to provide value across industries, providing high-capacity and low-latency connectivity to support an abundance of complex and useful applications. At Microsoft, we believe privacy-preserving live video analytics applications are an ideal fit for 5G networks. Our research and innovation, outlined in this post, continue to move us forward by clearing a path for inventing the next generation of live video analytics applications that will revolutionize our world—making it safer, more efficient, and more entertaining.

Source: microsoft.com

Continue reading

4 ways AI, computer vision, and related technologies expand IoT solutions

Inspecting five million vehicle welds every day requires the ability to check a weld’s quality every 17 milliseconds—an impossible challenge for a human. This type of quality control task is just one of many where the combined technologies of computer vision and AI excel.

Cameras, microphones, and a wide array of sophisticated sensors used in IoT solutions are increasingly tying together the physical and digital worlds. Using devices with the analytical capabilities of AI, solutions can quickly scan medical images for potentially concerning anomalies, listen to machinery noises for maintenance problems, or provide more thorough remote monitoring in a variety of environments.

Intel and Microsoft Azure are working together to help enterprises deploy intelligent IoT technologies and services, including AI’s deep learning abilities, computer vision, and audio or speech capabilities. Adding these capabilities enables solutions to solve more business challenges, uniting two or more—adding both computer vision and AI, for example greatly expands the potential uses for IoT solutions.

Four ways enterprises are benefitting from AI and computer vision

The strength of AI and machine learning algorithms is their ability to analyze vast amounts of data faster than humans and drive real-time decisions. Here are four ways that IoT solutions with Intel and Azure technologies are helping in a wide range of scenarios:

1. Improving common IoT-enabled applications

AI, computer vision, and audio or speech technology can enhance common IoT-enabled tasks, such as remote monitoring and predictive maintenance. Automated analysis of multiple video streams is able to detect movement or unusual actions and send instant notifications when they occur. Computer vision enables telemedicine technology to remotely monitor patients at home. Microphones can collect sound from industrial machinery and analyze it for deviations from normal operating sounds, which indicates if maintenance may be required soon.

2. Enhancing employee safety, patient care, and customer service

Pairing computer vision and AI can keep people safer. To assist medical technicians in scanning medical images, these technologies can automatically look for abnormalities and notify medical personnel which images require further examination. AI and machine learning algorithms can analyze video streams to monitor employee safety and alert them when there’s a potential danger. In retail stores, a similar system can limit customer numbers in a crowded space, as well as monitor inventory and notify employees when items need to be re-ordered.

3. Reducing complexity for developers and users

Even as IoT devices and services advance, technology providers are trying to simplify their deployment. Intel and Azure both encourage developers to design IoT plug-and-play compatible devices and offer toolkits, such as the Intel® Distribution of OpenVINO™ toolkit, that connect quickly and speed up deployment. This focus helps make devices and systems more understandable to those who use them every day.

4. Accelerating potential return on investment

Adding advanced technologies also has the potential to create faster returns on investment (ROI) in IoT solutions. Companies using AI in IoT solutions have fewer projects in the learning phase and more projects in the purchasing phase compared to surveyed companies who aren’t using AI in a solution, according to Microsoft IoT Signals Report surveys. Ninety-six percent of those companies integrating AI also indicate overall satisfaction with IoT technology, compared to 87 percent among IoT adopters. They’re also more likely to view IoT as critical to the success of their business, leading to more IoT investment and use. The technology itself has the potential to save enterprises significant costs and time. For example, DC Water streamlined sewer pipe inspections with an AI-powered video analysis solution that reduces pipe scanning costs by 75 percent and can create an ROI of 350 percent over three years.

Learn more about advanced technologies in IoT solutions

As IoT technology evolves, its ability to solve business challenges increases. Adding advanced capabilities, especially in tandem with each other, multiplies the beneficial effects that deploying an IoT solution can bring to your organization.

The partnership between Intel and Azure aims to make it easier for businesses of all sizes across multiple industries to learn about and choose the right IoT devices and services to help reach their goals. The two companies offer the hardware, software, edge and cloud services, and support needed to build an end-to-end solution. Additionally, a growing collection of partners are using our technology to build market-ready solutions that are targeted for use in manufacturing, retail, healthcare, transportation, smart spaces, and more.

To learn more about how AI, computer vision, and other advanced technologies can unlock new capabilities in IoT solutions, read our white paper, "4 ways AI, computer vision, and related technologies expand IoT solutions." The white paper highlights real-world solutions that successfully implement AI, computer vision, and related capabilities in industry-specific scenarios and discusses how different hardware, platforms, and tools can help maximize return on investment for various use cases.

Contest to harness edge solutions for sustainability

Intel also aims to harness AI, the intelligent cloud, and edge computing to create solutions and new use cases that focus on reducing environmental impacts. With support from Microsoft and others, the InnovateFPGA contest is open to ecologically-minded students, professionals, startups, and hobbyists who develop their projects using Cloud FPGA connectivity kits from Intel and Azure services. The contest, which offers cash prizes, takes place through this year.

Source: microsoft.com

Continue reading

Improve availability with zone-redundant storage for Azure Disk Storage

As organizations continue to accelerate their cloud adoption, the need for reliable infrastructure is critical to ensure business continuity and avoid costly disruptions. Azure Disk Storage provides maximum resiliency for all workloads with an industry-leading zero percent annual failure rate and single-instance service-level agreements (SLA) for all disk types, including a best-in-class single-instance SLA of 99.9 percent availability for Azure Virtual Machines using Azure Premium SSD or Azure Ultra Disk Storage.

Today, we continue our investments to further improve the reliability of our infrastructure with the general availability of zone-redundant storage (ZRS) for Azure Disk Storage. ZRS enables you to increase availability for your critical workloads by providing the industry’s only synchronous replication of block storage across three zones in a region, enabling your disks to tolerate zonal failures which may occur due to natural disasters or hardware issues. ZRS is currently supported for Azure Premium SSDs and Azure Standard SSDs.

We have seen strong interest and great feedback from many enterprise customers from various industries during our preview. These customers are planning to use ZRS for disks to provide higher availability for a wide range of scenarios such as clustering for SAP and SQL Server workloads, container applications, and legacy applications.

Increase availability for your clustered or distributed applications

Last year, we announced the general availability of shared disks for Azure Disk Storage, which is the only shared block storage in the cloud that supports both Windows and Linux-based clustered and distributed applications. This unique offering allows a single disk to be simultaneously attached and used from multiple virtual machines (VMs), enabling you to run your most demanding enterprise applications in the cloud, such as clustered databases, parallel file systems, persistent containers, and machine learning applications, without compromising on well-known deployment patterns for fast failover and high availability. Customers can now further improve the availability of their clustered applications, like SQL failover cluster instance (FCI) and SAP ASC/SCS leveraging Windows Server Failover Cluster (WSFC), with the combination of shared disks and ZRS.

Using Availability Zones for VMs, you can allocate primary and secondary VMs in different zones for higher availability and attach a shared ZRS disk to the VMs in different zones. If a primary VM goes down due to a zonal outage, WSFC will quickly failover to the secondary VM providing increased availability to your application. Customers can also use ZRS with shared disks for their Linux-based clustering applications that use IO fencing with SCSI persistent reservations. Shared disks are now available on all Premium SSD and Standard SSD sizes, enabling you to optimize for different price and performance options.

Figure 1: Multi-zone Windows Server Failover Cluster with shared ZRS disk

Take advantage of ZRS disks with multi-zone Azure Kubernetes Service clusters

Customers can take advantage of ZRS disks for their container applications hosted on multi-zone Azure Kubernetes Service (AKS) for higher reliability. If a zone goes down, AKS will automatically fail over the stateful pods to a healthy zone by detaching and attaching ZRS disks to nodes in the healthy zone. We recently released the ZRS disks support in AKS through the CSI driver.

Achieve higher availability for legacy applications

You can achieve high availability for your workloads using application-level replication across two zones (such as SQL Always On). However, if you are using industry-specific proprietary software or legacy applications like older versions of SQL Server, which don't support application-level synchronous replication, ZRS disks will provide improved availability through storage-level replication. For example, if a zone goes down due to natural disasters or hardware failures, the ZRS disk will continue to be operational. If your VM in the affected zone becomes unavailable, you could use a virtual machine in another zone and attach the same ZRS disk.

Build highly available, cost-effective solutions

To build highly available software-as-a-service (SaaS) solutions, independent software vendors (ISVs) can take advantage of ZRS disks to increase availability while also reducing costs. Previously, ISVs would need to host VMs in two zones and replicate data between the VMs. This resulted in extra costs as they had to deploy twice the amount of locally redundant storage (LRS) disks to maintain two copies of data in two zones and an additional central processing unit (CPU) for replicating the data to two zones. ISVs can now use shared ZRS disks to deliver a more cost-effective solution with 1.5 times lower costs on the disks and no additional replication costs. In addition, ZRS disks also offer lower write latency than VM to VM replication of the data as the replication is performed by the platform. NetApp describes the value that ZRS provides them and their customers:

“Many customers wish to have their data replicated cross-zone to improve business continuity against zonal failures and reduce downtime. ZRS for Azure Disk Storage combined with shared disks is truly a game-changer for us as it enables us to improve the availability of our solution, allows applications to achieve their full performance, and reduces replication costs by offloading the replication to the backend infrastructure. NetApp is excited to extend its CVO High Availability solution using ZRS disks as this helps us provide a comprehensive high availability solution at lower costs for our mutual customers.”—Rajesh Rajaraman, Senior Technical Director at NetApp

Performance for ZRS disks

The IOPS and bandwidth provided by ZRS disks are the same as the corresponding LRS disks. For example, a P30 (128 GiB) LRS Premium SSD disk provides 5,000 IOPS and 200 MB/second throughput, which is the same for a P30 ZRS Premium SSD disk. Disk latency for ZRS is higher than that of LRS due to the cross zonal copy of data.

Source: microsoft.com

Continue reading

Boost your network security with new updates to Azure Firewall

Today, we are announcing new Azure Firewall capabilities as well as updates for August 2021.

◉ Azure Firewall supports US West 3, Jio India West, and Brazil Southeast.

◉ Auto-generated self-signed certificates for Azure Firewall Premium SKU.

◉ Secure Hub now supports Availability Zones.

◉ Deploy Azure Firewall without public IP in Forced Tunnel mode.

◉ Configure pre-existing Azure Firewalls in Force Tunnel mode using stop or start commands.

Azure Firewall regional expansion

We expanded Azure Firewall service to three new public regions including US West 3, Jio India West, and Brazil Southeast. Both Standard and Premium Firewall SKUs are available in the new public regions. Firewall Policy support is planned to release shortly.

Azure Firewall Premium SKU support for self-signed certificates

For non-production deployments, you can use the Azure Firewall Premium certification auto-generation mechanism, which automatically creates for you the following three resources, ties them together, and sets up transport layer security (TLS) inspection with a single click of a button:

1. Managed Identity.

2. Key Vault.

3. Self-signed intermediate CA certificate.

Secure Hub now supports Availability Zones

With Availability Zones, your availability increases to 99.99 percent uptime. A secured virtual hub can now be configured during deployment to span multiple Availability Zones for increased availability.

Deploy Azure Firewall without public IP in Forced Tunnel mode

Azure Firewall service requires public IP for its operational purposes. While secure, some deployments don’t prefer exposing public IP directly to the internet. In such cases, customers can deploy Azure Firewall in Forced Tunnel mode. This configuration creates a management NIC which is used by Azure Firewall for its operations. The Tenant Datapath network can be configured without a public IP, and internet traffic can be forced tunneled to another Firewall or completely blocked.

Note that Forced Tunnel mode cannot be configured at Run Time. You can either redeploy the Firewall or use the stop and start facility to reconfigure existing Azure Firewalls in Forced Tunnel mode. Secure Hub Firewalls are always deployed in Forced Tunnel mode.

Source: microsoft.com

Continue reading

Guarantee capacity access with on-demand capacity reservations—now in preview

Microsoft customers rely on Azure to host mission-critical workloads and services. A substantial percentage use the traditional infrastructure-as-a-service (IaaS) approach to deploying virtual machines, often orchestrated with Azure Virtual Machine Scale Sets or third-party tools.

Customers rely on Azure to provide compute capacity on demand. However, in some situations, customers need more specific assurances:

◉ Business-critical applications—certain business processes can be disrupted even by a temporary interruption.

◉ Disaster recovery (DR)—in the event of a natural disaster, businesses require capacity guarantees to recover and resume operations in a different location.

◉ Special events—sometimes a substantial surge in capacity is expected, obtaining capacity assurance can ensure that business performance remains at the highest levels.

Until now, the only way to obtain a service level agreement (SLA) for Azure Virtual Machines capacity has been to deploy actual virtual machines (VM), whether running or stopped. This can lead to management overhead to keep all VMs up to date and impedes the use of compute capacity for other business purposes, for example when the VMs are not needed for a primary workload.

Introducing on-demand capacity reservations

Today we are announcing the preview of on-demand capacity reservations for Azure Virtual Machines. This new feature allows you to deploy and manage the compute capacity required to run Azure VMs separately from the VMs themselves. On-demand capacity reservations will come with a defined SLA (forthcoming at general availability).

This creates new options for you to better manage your capacity needs. More specifically, as outlined above:

◉ Business-critical applications—use on-demand capacity reservations to protect your capacity, for example when taking these VMs offline to perform updates.

◉ Disaster recovery—you now have the option to set aside compute capacity to ensure a seamless recovery in the event of a natural disaster. The compute capacity can be repurposed to run other workloads whenever DR is not in effect. The VM maintenance can be handled by keeping core images up to date without the need to deploy or maintain VMs outside of DR testing.

◉ Special events—claiming capacity ahead of time provides assurance that your business can handle the extra demand.

An on-demand capacity reservation will operate as a deployable object: specify a VM size, location, and quantity using the Compute Resource Provider. Upon successful completion of a reservation, customers can then assign VMs to the reserved capacity. The reservation is on-demand because there is no term commitment required. You can delete on-demand capacity reservations whenever they are no longer needed.

The lack of a term commitment means that reservations can be made using pay-as-you-go terms based on the underlying price of the VM size (without the cost of the operating system you may want to deploy later on), location, and quantity of the reservation. Within the Azure billing system, any available term discounts from purchased Reserved VM Instances (RIs) will be automatically applied. The rules for applying RIs to VM usage will be the same that we have in place today for reservations and virtual machines. 

Here is a comparison between on-demand capacity reservations and RIs:

*SLA guarantees will apply only after general availability

Once the feature becomes generally available, if you successfully create an on-demand capacity reservation, the SLA will supersede the capacity priority option on any applicable RIs. You can configure your RIs for size flexibility to maximize your discounted usage.

Existing and future capabilities

During the preview, register each subscription to access the API and Azure portal interfaces.

The preview supports general-purpose Azure VM families:

◉ Burstable (B)

◉ General-purpose (Av2, Dv2, and later generations including Dav4)

◉ General purpose, memory-optimized (Ev3 and later generations including Eav4)

◉ Compute-optimized (Fv1 and newer)

We are planning to support additional VM series. Expect additional announcements later this year. Region support is inclusive of Azure Public Cloud locations that are generally available. Microsoft plans to extend support to Azure for US Government and Azure China in the near future.

The ability to use reserved capacity will be extended to more scenarios in the coming months. For example, stay tuned for the direct integration with Azure Site Recovery (coming soon in preview).

Source: microsoft.com

Continue reading