Microsoft Connected Vehicle Platform: trends and investment areas

The past year has been eventful for a lot of reasons. At Microsoft, we’ve expanded our partnerships, including Volkswagen, LG Electronics, Faurecia, TomTom, and more, and taken the wraps off new thinking such as at CES, where we recently demonstrated our approach to in-vehicle compute and software architecture.

Looking ahead, areas that were once nominally related now come into sharper focus as the supporting technologies are deployed and the various industry verticals mature. The welcoming of a new year is a good time to pause and take in what is happening in our industry and in related ones with an aim to developing a view on where it’s all heading.

In this blog, we will talk about the trends that we see in connected vehicles and smart cities and describe how we see ourselves fitting in and contributing.

Trends

Mobility as a Service (Maas)

MaaS (sometimes referred to as Transportation as a Service, or TaaS) is about people getting to goods and services and getting those goods and services to people. Ride-hailing and ride-sharing come to mind, but so do many other forms of MaaS offerings such as air taxis, autonomous drone fleets, and last-mile delivery services. We inherently believe that completing a single trip—of a person or goods—will soon require a combination of passenger-owned vehicles, ride-sharing, ride-hailing, autonomous taxis, bicycle-and scooter-sharing services transporting people on land, sea, and in the air (what we refer to as “multi-modal routing”). Service offerings that link these different modes of transportation will be key to making this natural for users.

With Ford, we are exploring how quantum algorithms can help improve urban traffic congestion and develop a more balanced routing system. We’ve also built strong partnerships with TomTom for traffic-based routing as well as with AccuWeather for current and forecast weather reports to increase awareness of weather events that will occur along the route. In 2020, we will be integrating these routing methods together and making them available as part of the Azure Maps service and API. Because mobility constitutes experiences throughout the day across various modes of transportation, finding pickup locations, planning trips from home and work, and doing errands along the way, Azure Maps ties the mobility journey with cloud APIs and iOS and Android SDKs to deliver in-app mobility and mapping experiences. Coupled with the connected vehicle architecture of integration with federated user authentication, integration with the Microsoft Graph, and secure provisioning of vehicles, digital assistants can support mobility end-to-end. The same technologies can be used in moving goods and retail delivery systems.

The pressure to become profitable will force changes and consolidation among the MaaS providers and will keep their focus on approaches to reducing costs such as through autonomous driving. Incumbent original equipment manufacturers (OEMs) are expanding their businesses to include elements of car-sharing to continue evolving their businesses as private car ownership is likely to decline over time.

Connecting vehicles to the cloud

We refer holistically to these various signals that can inform vehicle routing (traffic, weather, available modalities, municipal infrastructure, and more) as “navigation intelligence.” Taking advantage of this navigation intelligence will require connected vehicles to become more sophisticated than just logging telematics to the cloud.

The reporting of basic telematics (car-to-cloud) is barely table-stakes; over-the-air updates (OTA, or cloud-to-car) will become key to delivering a market-competitive vehicle, as will command-and-control (more cloud-to-car, via phone apps). Forward-thinking car manufacturers deserve a lot of credit here for showing what’s possible and for creating in consumers the expectation that the appearance of new features in the car after it is purchased isn’t just cool, but normal.

Future steps include the integration of in-vehicle infotainment (IVI) with voice assistants that blend the in- and out-of-vehicle experiences, updating AI models for in-market vehicles for automated driving levels one through five, and of course pre-processing the telemetry at the edge in order to better enable reinforcement learning in the cloud as well as just generally improving services.

Delivering value from the cloud to vehicles and phones

As vehicles become more richly connected and deliver experiences that overlap with what we’ve come to expect from our phones, an emerging question is, what is the right way to make these work together? Projecting to the IVI system of the vehicle is one approach, but most agree that vehicles should have a great experience without a phone present.

Separately, phones are a great proxy for “a vehicle” in some contexts, such as bicycle sharing, providing speed, location, and various other probe data, as well as providing connectivity (as well as subsidizing the associated costs) for low-powered electronics on the vehicle.

This is probably a good time to mention 5G. The opportunity 5G brings will have a ripple effect across industries. It will be a critical foundation for the continued rise of smart devices, machines, and things. They can speak, listen, see, feel, and act using sensitive sensor technology as well as data analytics and machine learning algorithms without requiring “always on” connectivity. This is what we call the intelligent edge. Our strategy is to enable 5G at the edge through cloud partnerships, with a focus on security and developer experience.

Optimizations through a system-of-systems approach

Connecting things to the cloud, getting data into the cloud, and then bringing the insights gained through cloud-enabled analytics back to the things is how optimizations in one area can be brought to bear in another area. This is the essence of digital transformation. Vehicles gathering high-resolution imagery for improving HD maps can also inform municipalities about maintenance issues. Accident information coupled with vehicle telemetry data can inform better PHYD (pay how you drive) insurance plans as well as the deployment of first responder infrastructure to reduce incident response time.

As the vehicle fleet electrifies, the demand for charging stations will grow. The way in-car routing works for an electric car is based only on knowledge of existing charging stations along the route—regardless of the current or predicted wait-times at those stations. But what if that route could also be informed by historical use patterns and live use data of individual charging stations in order to avoid arriving and having three cars ahead of you? Suddenly, your 20-minute charge time is actually a 60-minute stop, and an alternate route would have made more sense, even if, on paper, it’s more miles driven.

Realizing these kinds of scenarios means tying together knowledge about the electrical grid, traffic patterns, vehicle types, and incident data. The opportunities here for brokering the relationships among these systems are immense, as are the challenges to do so in a way that encourages the interconnection and sharing while maintaining privacy, compliance, and security.

Laws, policies, and ethics

The past several years of data breaches and elections are evidence of a continuously evolving nature of the security threats that we face. That kind of environment requires platforms that continuously invest in security as a fundamental cost of doing business.

Laws, regulatory compliance, and ethics must figure into the design and implementation of our technologies to as great a degree as goals like performance and scalability do. Smart city initiatives, where having visibility into the movement of people, goods, and vehicles is key to doing the kinds of optimizations that increase the quality of life in these cities, will confront these issues head-on.

Routing today is informed by traffic conditions but is still fairly “selfish:” routing for “me” rather than for “we.” Cities would like a hand in shaping traffic, especially if they can factor in deeper insights such as the types of vehicles on the road (sending freight one way versus passenger traffic another way), whether or not there is an upcoming sporting event or road closure, weather, and so on.

Doing this in a way that is cognizant of local infrastructure and the environment is what smart cities initiatives are all about.

With the Microsoft Connected Vehicle Platform (MCVP) and an ecosystem of partners across the industry, Microsoft offers a consistent horizontal platform on top of which customer-facing solutions can be built. MCVP helps mobility companies accelerate the delivery of digital services across vehicle provisioning, two-way network connectivity, and continuous over-the-air updates of containerized functionality. MCVP provides support for command-and-control, hot/warm/cold path for telematics, and extension hooks for customer/third-party differentiation. Being built on Azure, MCVP then includes the hyperscale, global availability, and regulatory compliance that comes as part of Azure. OEMs and fleet operators leverage MCVP as a way to “move up the stack” and focus on their customers rather than spend resources on non-differentiating infrastructure.

Innovation in the automotive industry

At Microsoft, and within the Azure IoT organization specifically, we have a front-row seat on the transformative work that is being done in many different industries, using sensors to gather data and develop insights that inform better decision-making. We are excited to see these industries on paths that are trending to converging, mutually beneficial paths. Our colleague Sanjay Ravi shares his thoughts from an automotive industry perspective in this great article.

Turning our attention to our customer and partner ecosystem, the traction we’ve gotten across the industry has been overwhelming:

The Volkswagen Automotive Cloud will be one of the largest dedicated clouds of its kind in the automotive industry and will provide all future digital services and mobility offerings across its entire fleet. More than 5 million new Volkswagen-specific brand vehicles are to be fully connected on Microsoft’s Azure cloud and edge platform each year. The Automotive Cloud subsequently will be rolled out on all Group brands and models.

Cerence is working with us to integrate Cerence Drive products with MCVP. This new integration is part of Cerence’s ongoing commitment to delivering a superior user experience in the car through interoperability across voice-powered platforms and operating systems. Automakers developing their connected vehicle solutions on MCVP can now benefit from Cerence’s industry-leading conversational AI, in turn delivering a seamless, connected, voice-powered experience to their drivers.

Ericsson, whose Connected Vehicle Cloud connects more than 4 million vehicles across 180 countries, is integrating their Connected Vehicle Cloud with Microsoft’s Connected Vehicle Platform to accelerate the delivery of safe, comfortable, and personalized connected driving experiences with our cloud, AI, and IoT technologies.

LG Electronics is working with Microsoft to build its automotive infotainment systems, building management systems and other business-to-business collaborations. LG will leverage Microsoft Azure cloud and AI services to accelerate the digital transformation of LG’s B2B business growth engines, as well as Automotive Intelligent Edge, the in-vehicle runtime environment provided as part of MCVP.

Global technology company ZF Friedrichshafen is transforming into a provider of software-driven mobility solutions, leveraging Azure cloud services and developer tools to promote faster development and validation of connected vehicle functions on a global scale.

Faurecia is collaborating with Microsoft to develop services that improve comfort, wellness, and infotainment as well as bring digital continuity from home or the office to the car. At CES, Faurecia demonstrated how its cockpit integration will enable Microsoft Teams video conferencing. Using Microsoft Connected Vehicle Platform, Faurecia also showcased its vision of playing games on the go, using Microsoft’s new Project xCloud streaming game preview.

Bell has revealed AerOS, a digital mobility platform that will give operators a 360° view into their aircraft fleet. By leveraging technologies like artificial intelligence and IoT, AerOS provides powerful capabilities like fleet master scheduling and real-time aircraft monitoring, enhancing Bell’s Mobility-as-a-Service (MaaS) experience. Bell chose Microsoft Azure as the technology platform to manage fleet information, observe aircraft health, and manage the throughput of goods, products, predictive data, and maintenance.

Luxoft is expanding its collaboration with Microsoft to accelerate the delivery of connected vehicle solutions and mobility experiences. By leveraging MCVP, Luxoft will enable and accelerate the delivery of vehicle-centric solutions and services that will allow automakers to deliver unique features such as advanced vehicle diagnostics, remote access and repair, and preventive maintenance. Collecting real usage data will also support vehicle engineering to improve manufacturing quality.

We are incredibly excited to be a part of the connected vehicle space. With MCVP, our ecosystem partners and our partnerships with leading automotive players, both vehicle OEMs and automotive technology suppliers, we believe we have a uniquely capable offering enabling at global scale the next wave of innovation in the automotive industry as well as related verticals such as smart cities, smart infrastructure, insurance, transportation, and beyond.

Continue reading

Azure IoT improves pharmaceutical sample management and medication adherence

For the recent IoT Signals report, commissioned by our Azure IoT team and conducted by Hypothesis Group, more than 3,000 decision makers at enterprise companies across the US, UK, Germany, France, China, and Japan who were currently involved in IoT, participated in a 20-minute online survey. Healthcare was one of the industries included in the research. Of the healthcare executives surveyed, 82 percent said they have at least one IoT project in either the learning, proof of concept, purchase, or use phase, with many reporting they have one or more projects currently in ‘use.’ The top use cases cited by the healthcare executives included:

◉ Tracking patient staff and inventory.

◉ Remote device monitoring and service.

◉ Remote health monitoring and assistance.

◉ Safety, security, and compliance.

◉ Facilities management.

Today we want to shed light on how two innovative companies are building upon this momentum and their own research to build IoT-enabled solutions with Azure IoT technologies that support medication management and adherence. These solutions address the safety, security, compliance, and inventory use cases highlighted in the report.

The Cost of Pharmaceutical Samples

According to a January 2019 article published by JAMA, Medical Marketing in the United States, 1997-2016, “Marketing to health care professionals by pharmaceutical companies accounted for [the] most promotional spending and increased from $15.6 billion to $20.3 billion, including $5.6 billion for prescriber detailing, $13.5 billion for free samples.”

Improving sample management

With billions of dollars on the line, one of our partners has developed an innovative way to ensure that pharmaceutical companies manage their samples in a cost-effective way. Using their own knowledge of the pharmaceutical industry and in-depth research, P360 (formerly Prescriber360), developed Swittons to Example of a branded virtual rep devicebridge the gap between pharmaceutical companies and physicians. Designed as a “virtual pharmaceutical representative,” this IoT-enabled device offers real-time, secure communications between the physician and the pharmaceutical company. With this single device, physicians can order a sample, request a visit from a medical science liaison (MSL) or sales rep, or connect with the pharmaceutical company’s inside sales rep (as shown in the graphic below).

Designed to be branded with each pharmaceutical company’s product, the device is a physician engagement tool that enables pharmaceutical companies to customize and manage a sales channel that remains fully authentic to their brand experience. Furthermore, it provides an audit trail to manage samples more economically, enabling pharmaceutical companies to penetrate market whitespace and extend efficient sampling in areas that were previously unreachable.

Built on our Azure IoT platform, Swittons takes advantage of the latest in cloud, security, telecommunications, and analytics technology. “We strategically selected Azure IoT as the foundation for our Swittons ‘Virtual Rep.’ Microsoft’s vision, investments and the breadth of Azure cloud were the key criteria for selection. Having a reliable IoT platform along with world-class data and security infrastructure in Azure made the choice very easy,” commented Anupam Nandwana, CEO, P360, parent company of Swittons.

On the other end of the pharmaceutical supply chain is another scenario that dramatically affects the efficacy of pharmaceutical products—medication adherence.

Ensuring medication adherence

In the US today, 25 to 50 percent of all adults fail to take their prescribed medication on time, contributing to poor health outcomes, over-utilization of healthcare services and significant cost increases.

The causes of low levels of medication adherence are multi-faceted and include factors like carelessness, fear, supply, cost, and lack of understanding or information, with forgetfulness as the primary cause.

Furthermore, as cited in an editorial from BMJ Quality and Safety, “medication adherence thus constitutes one of the ‘big hairy problems’ or ‘big hairy audacious goals’ of healthcare. As well as affecting patients’ long-term outcomes, non-adherence can increase healthcare costs through consumption of medicines below the threshold of adherence required for clinical benefit, as well as contributing to healthcare resource use such as hospital admissions.

In response to this, the global market for medication adherence (hardware-based automation and adherence systems and software-based applications) was worth nearly $1.7 billion in 2016. The market is expected to reach more than $3.9 billion by 2021, increasing at a CAGR of 18.0 percent from 2016 through 2021. This steep increase is fueled by burgeoning demand for advanced medication adherence systems and a growing number of people worldwide with chronic diseases.

Personal experience leads to action

Emanuele Musini knows all too well the implications of not taking medications properly. In fact, it was the pain of losing his father in 2005 from a chronic condition and a lack of adhering to the prescribed medication regimen that became the catalyst for Emanuele to start studying the issue in-depth, searching for a solution. In 2015, Emanuele, along with his multidisciplinary team of doctors, entrepreneurs, engineers, and user-experience professionals, created Pillo Health, a health platform centered around a robot and digital assistant designed to prevent other family members from enduring what Emanuele and his family experienced. Since their founding, they’ve partnered with leading manufacturers, such as Stanley Black & Decker, to bring in-home medication management solutions to market with solutions like Pria, a winner of the 2019 CES Innovation Awards.”

The Pillo Health team built their medication adherence solution on Microsoft Azure Cloud Services using Azure Cognitive Services for voice technology and facial recognition, and services from the Azure IoT platform, including IoT Hub. The result is a voice-first, personalized, cloud-enabled, medication assistant that can help people maintain their medication regimen through social connectivity and delivery of important medical information at home. In a 4-week study conducted with AARP in 2018 for diabetic patients who were prescribed Metformin, Pillo delivered an average medication adherence rate of more than 87 percent—a meaningful 20 to 30 percent improvement from conventional reported standards.

Antonello Scalmato, Director of Cloud Services at Pillo Health noted, “We selected Microsoft Azure because it provided the best infrastructure for PaaS applications, allowed us to speed up the development of our complex product and avoided the overhead of machine and security management for traditional web API infrastructure. Moreover, IoT Hub provides a channel for secure communications and notifications to our users, and also enables simple device management that protects our product, from the factory into the users' homes.”

Continue reading

Exactly Those Tips That Will Help You Prepare for Microsoft 70-467 Certification Exam

About Microsoft 70-467 Certification

In this 70-467 exam, you will learn how to design, implement, and maintain a Business Intelligence (BI) platform with Microsoft SQL Server 2014. You will learn to create a data warehouse and Business Intelligence solutions, including ETL, BI Delivery, Reporting Services, Excel Reporting, SharePoint Server BI, with how to observe and optimize a BI solution.

You will learn big data ideas and analysis using Microsoft Azure HDInsight plus how to design, execute, and process big data solutions.

Microsoft 70-467 Exam Details

• Name: Designing Business Intelligence Solutions with Microsoft SQL Server

• Code: 70-467

• Duration: 150 minutes

• Questions: 40-60

• Languages: English, Chinese (Simplified), French, German, Japanese, Portuguese (Brazil)

• Passing Score: 700/1000

• Exam Price: 165 USD

• Technology: Microsoft SQL Server

• Credit Toward Certification: MCSE

Microsoft 70-467 Exam Syllabus Topics

• Plan business intelligence (BI) infrastructure (15-20%)

• Design BI infrastructure (15-20%)

• Design a reporting solution (20-25%)

• Design BI data models (30-35%)

• Design an ETL solution (10-15%)

Who Should Take the 70-467 Exam?

The 70-467 exam is intended for business intelligence (BI) architects, who are bound for the overall design of a BI infrastructure and how it compares to other data systems in use.

Who Can Go for Microsoft SQL Server Certifications?

Anyone who has a fundamental knowledge of SQL and database can go for these certifications.

Mainly:

• Business Intelligence Professionals

• Software Developers and IT Professionals

• SQL and Database Administrators

• Business Analysts and Managers

• Project Managers

• Big Data and Hadoop Professionals

• Those aiming for a career in SQL development

This 70-467 uses Microsoft SQL Server 2014.

This 70-467 exam includes material from the Official Microsoft Learning Product 20467: Designing Business Intelligence Solutionswith Microsoft SQL Server 2014. This program plans to skills and knowledge measured by Microsoft exam 70-467, and in combination with on-the-job experience, can ease prepare you for the exam.

Prerequisites for Microsoft 70-467 Certification

• Basic knowledge of dimensional modeling (star schema) for data warehouses.

• The ability to perform Integration Services packages that include control flows and data flows.

• The talent to create a primary multidimensional solid with Analysis Services.

• The ability to generate a primary tabular model with PowerPivot and Analysis Services.

• The capacity to form Reporting Services reports with Report Designer.

• The strength to make authentication and support in the SQL Server database engine, Analysis Services, and Reporting Services.

• Awareness with SharePoint Server and Microsoft Office applications, especially Excel.

Where to Go from Here?

After you pass the Designing Business Intelligence Solutionswith Microsoft SQL Server 2012 (70-467) exam, you may want to take the following exams:

• Querying Microsoft SQL Server 2012/2014 (70-461)

• Administering Microsoft SQL Server 2012/2014 Databases (70-462)

• Implementing a Data Warehouse with Microsoft SQL Server 2012/2014 (70-463)

• Developing Microsoft SQL Server Databases (70-464)

• Designing Database Solutions for Microsoft SQL Server (70-465)

• Designing Business Intelligence Solutions with Microsoft SQL Server (70-466)

Tips Should You Follow to Preparation for Microsoft 70-467 Certification Exam

There are so many sites and so many hints out there on how to pass the Microsoft 70-467 exam. However, you must assess the reliability of the information and its source. One reliable website you can trust is AnalyticsExam.

Here are some helpful tips for passing the 70-467 certification test:

• First of all, do not forget to register for the trial.

• Make sure that you clearly understand the objectives and expectations of Microsoft 70-467.

• Devise your study plan and approach you are going to take to prepare for the exam.

• Ensure that you create a practical plan and stick to it during the preparation period to increase your opportunities of acing Microsoft70-467 at the first attempt.

• Use the practice tests from AnalyticsExam as your primary way of studying.

• Join active study groups and online forums. This will help you ensure that you are investing yourself with people who are also preparing for the same test as you.

• Ensure that you are completing your preparation a few days before the real exam so that you can have ample time to change what you have studied and to rest before taking the certification test.

• Make sure that you are getting plenty of rest the night before the exam day. Go to bed early and sleep for at least 8 hours to ensure that you are waking up fresh and energetic. You will get it more convenient to recall many things when you are new.

Summary

After all this hard work, you will be getting practice exam scores of 80% and above. At that point, you are ready to book in the exam! You can take a 70-467 exam at a test center, but Microsoft has lately set up a system where you can take the exam at home!

It is a bit of a bother setting everything up and meeting all of the requirements, but it can save you some time and money not having to travel.

Read: Revealed: The Best Tips for Passing Microsoft Certification Exams

Good luck with your 70-467 exam, and remember success is earned, not given away. Put in the effort, and you will pass.

Continue reading

Assess your servers with a CSV import into Azure Migrate

At Microsoft Ignite, we announced new Azure Migrate assessment capabilities that further simplify migration planning. In this post, we will demonstrate how to import servers into Azure Migrate Server Assessment through a CSV upload. Virtual servers of any hypervisor or cloud as well as physical servers can be assessed. You can get started with the CSV import feature by creating an Azure Migrate project or using your existing project.

Previously, Server Assessment required setting up an appliance in customer premises to perform discovery of VMware, Hyper-V virtual machines (VMs), and physical servers. We now also support importing and assessing servers without deploying an appliance. Import-based assessments provide support for Server Assessment features like Azure suitability analysis, migration cost planning, and performance-based rightsizing. The import-based assessment is helpful in the initial stages of migration planning, when you may not be able to deploy the appliance due to pending organizational or security constraints that prevent you from sending data to Azure.

Importing your servers is easy. Simply upload the server inventory in a CSV file as per the template provided by Azure Migrate. Only four data points are mandatory — server name, number of cores, size of memory, and operating system name. While you can run the assessment with this minimal information, we recommend you provide disk data as well to avail disk sizing in assessments.

Azure suitability analysis

The assessment determines whether a given server can be migrated as-is to Azure. Azure support is checked for each server discovered; if it is found that a server is not ready to be migrated, remediation guidance is automatically provided. You can customize your assessment by changing its properties, and regenerate the assessment reports. You can also generate an assessment report by choosing a VM series of your choice and specify the uptime of the workloads you will run in Azure.

Cost estimation and sizing

Assessment reports provide detailed cost estimates. You can optimize on cost using performance-based rightsizing assessments; the performance utilization value you specify of your on-premises server is taken into consideration to recommend an appropriate Azure Virtual Machine and disk SKU. This helps to optimize and right-size on cost as you migrate servers that might be over-provisioned in your on-premises data center. You can apply subscription offers and Reserved Instance pricing on the cost estimates

Assess your imported servers in four simple steps

1. Create an Azure Migrate project and add the Server Assessment solution to the project. If you already have a project, you do not need to create a new one. Download the CSV template for importing servers.

2. Gather the inventory data from a configuration management database (CMDB), or from your vCenter server, or Hyper-V environments. Convert the data into the format of the Azure Migrate CSV template.

3. Import the servers into Azure Migrate by uploading the server inventory in a CSV file as per the template.

5. Once you have successfully imported the servers, create assessments and review the assessment reports.

When you are ready to deploy an appliance, you can leverage the performance history gathered by the appliance for more accurate sizing, as well as plan migration phases using dependency analysis.

Get started right away by creating an Azure Migrate project. Note that the inventory metadata uploaded is persisted in the geography you select while creating the project. You can select a geography of your choice. Server Assessment is available today in Asia Pacific, Australia, Brazil, Canada, Europe, France, India, Japan, Korea, United Kingdom, and United States geographies.

Continue reading

Advancing safe deployment practices

When running IT systems on-premises, you might try to ensure perfect availability by having gold-plated hardware, locking up the server room and throwing away the key. Software wise, IT would traditionally prevent as much change as possible — avoiding applying updates to the operating system or applications because they’re too critical, and pushing back on change requests from users. With everyone treading carefully around the system, this ‘nobody breathe!’ approach stifles continued system improvement, and sometimes even compromises security for systems that are deemed too crucial to patch regularly. As Mark mentioned above, this approach doesn't work for change and release management in a hyperscale public cloud like Azure. Change is both inevitable and beneficial, given the need to deploy service updates and improvements, and given our commitment to you to act quickly in the face of security vulnerabilities. As we can’t simply avoid change, Microsoft, our customers, and our partners need to acknowledge that change is expected, and we plan for it. Microsoft continues to work on making updates as transparent as possible and will deploy the changes safely as described below. Having said that, our customers and partners should also design for high availability, consume maintenance events sent by the platform to adapt as needed. Finally, in some cases, customers can take control of initiating the platform updates at a suitable time for their organization.

Changing safely

When considering how to deploy releases throughout our Azure datacenters, one of the key premises that shapes our processes is to assume that there could be an unknown problem introduced by the change being deployed, plan in a way that enables the discovery of said problem with minimal impact, and automate mitigation actions for when the problem surfaces. While a developer might judge it as completely innocuous and guarantee that it won't affect the service, even the smallest change to a system poses a risk to the stability of the system, so ‘changes’ here refers to all kinds of new releases and covers both code changes and configuration changes. In most cases a configuration change has a less dramatic impact on the behavior of a system but, just as for a code change, no configuration change is free of risk for activating a latent code defect or a new code path.

Teams across Azure follow similar processes to prevent or at least minimize impact related to changes. Firstly, by ensuring that changes meet the quality bar before the deployment starts, through test and integration validations. Then after sign off, we roll out the change in a gradual manner and measure health signals continuously, so that we can detect in relative isolation if there is any unexpected impact associated with the change that did not surface during testing. We do not want a change causing problems to ever make it to broad production, so steps are taken to ensure we can avoid that whenever possible. The gradual deployment gives us a good opportunity to detect issues at a smaller scale (or a smaller ‘blast radius’) before it causes widespread impact.

Azure approaches change automation, aligned with the high level process above, through a safe deployment practice (SDP) framework, which aims to ensure that all code and configuration changes go through a lifecycle of specific stages, where health metrics are monitored along the way to trigger automatic actions and alerts in case of any degradation detected. These stages (shown in the diagram that follows) reduce the risk that software changes will negatively affect your existing Azure workloads.

This shows a simplification of our deployment pipeline, starting on the left with developers modifying their code, testing it on their own systems, and pushing it to staging environments. Generally, this integration environment is dedicated to teams for a subset of Azure services that need to test the interactions of their particular components together. For example, core infrastructure teams such as compute, networking, and storage share an integration environment. Each team runs synthetic tests and stress tests on the software in that environment, iterate until stable, and then once the quality results indicate that a given release, feature, or change is ready for production they deploy the changes into the canary regions.

Canary regions

Publicly we refer to canary regions as “Early Updates Access Program” regions, and they’re effectively full-blown Azure regions with the vast majority of Azure services. One of the canary regions is built with Availability Zones and the other without it, and both regions form a region pair so that we can validate data geo-replication capabilities. These canary regions are used for full, production level, end to end validations and scenario coverage at scale. They host some first party services (for internal customers), several third party services, and a small set of external customers that we invite into the program to help increase the richness and complexity of scenarios covered, all to ensure that canary regions have patterns of usage representative of our public Azure regions. Azure teams also run stress and synthetic tests in these environments, and periodically we execute fault injections or disaster recovery drills at the region or Availability Zone level, to practice the detection and recovery workflows that would be run if this occurred in real life. Separately and together, these exercises attempt to ensure that software is of the highest quality before the changes touch broad customer workloads in Azure.

Pilot phase

Once the results from canary indicate that there are no known issues detected, the progressive deployment to production can get started, beginning with what we call our pilot phase. This phase enables us to try the changes, still at a relatively small scale, but with more diversity of hardware and configurations. This phase is especially important for software like core storage services and core compute infrastructure services, that have hardware dependencies. For example, Azure offers servers with GPU's, large memory servers, commodity servers, multiple generations and types of processors, Infiniband, and more, so this enables flighting the changes and may enable detection of issues that would not surface during the smaller scale testing. In each step along the way, thorough health monitoring and extended 'bake times' enable potential failure patterns to surface, and increase our confidence in the changes while greatly reducing the overall risk to our customers.

Once we determine that the results from the pilot phase are good, the deployment systems proceed by allowing the change to progress to more and more regions incrementally. Throughout the deployment to the broader Azure regions, the deployment systems endeavor to respect Availability Zones (a change only goes to one Availability Zone within a region) and region pairing (every region is ‘paired up’ with a second region for georedundant storage) so a change deploys first to a region and then to its pair. In general, the changes deploy only as long as no negative signals surface.

Safe deployment practices in action

Given the scale of Azure globally, the entire rollout process is completely automated and driven by policy. These declarative policies and processes (not the developers) determine how quickly software can be rolled out. Policies are defined centrally and include mandatory health signals for monitoring the quality of software as well as mandatory ‘bake times’ between the different stages outlined above. The reason to have software sitting and baking for different periods of time across each phase is to make sure to expose the change to a full spectrum of load on that service. For example, diverse organizational users might be coming online in the morning, gaming customers might be coming online in the evening, and new virtual machines (VMs) or resource creations from customers may occur over an extended period of time.

Global services, which cannot take the approach of progressively deploying to different clusters, regions, or service rings, also practice a version of progressive rollouts in alignment with SDP. These services follow the model of updating their service instances in multiple phases, progressively deviating traffic to the updated instances through Azure Traffic Manager. If the signals are positive, more traffic gets deviated over time to updated instances, increasing confidence and unblocking the deployment from being applied to more service instances over time.

Of course, the Azure platform also has the ability to deploy a change simultaneously to all of Azure, in case this is necessary to mitigate an extremely critical vulnerability. Although our safe deployment policy is mandatory, we can choose to accelerate it when certain emergency conditions are met. For example, to release a security update that requires us to move much more quickly than we normally would, or for a fix where the risk of regression is overcome by the fix mitigating a problem that’s already very impactful to customers. These exceptions are very rare, in general our deployment tools and processes intentionally sacrifice velocity to maximize the chance for signals to build up and scenarios and workflows to be exercised at scale, thus creating the opportunity to discover issues at the smallest possible scale of impact.

Continuing improvements

Our safe deployment practices and deployment tooling continue to evolve with learnings from previous outages and maintenance events, and in line with our goal of detecting issues at a significantly smaller scale. For example, we have learned about the importance of continuing to enrich our health signals and about using machine learning to better correlate faults and detect anomalies. We also continue to improve the way in which we do pilots and flighting, so that we can cover more diversity of hardware with smaller risk. We continue to improve our ability to rollback changes automatically if they show potential signs of problems. We also continue to invest in platform features that reduce or eliminate the impact of changes generally.

With over a thousand new capabilities released in the last year, we know that the pace of change in Azure can feel overwhelming. As Mark mentioned, the agility and continual improvement of cloud services is one of the key value propositions of the cloud – change is a feature, not a bug. To learn about the latest releases, we encourage customers and partners to stay in the know at Azure.com/Updates. We endeavor to keep this as the single place to learn about recent and upcoming Azure product updates, including the roadmap of innovations we have in development. To understand the regions in which these different services are available, or when they will be available, you can also use our tool at Azure.com/ProductsbyRegion.

Source: azure.microsoft.com

Continue reading

Backup Explorer now available in preview

As organizations continue to expand their use of IT and the cloud, protecting critical enterprise data becomes extremely important. And if you are a backup admin on Microsoft Azure, being able to efficiently monitor backups on a daily basis is a key requirement to ensuring that your organization has no weaknesses in its last line of defense.

Up until now, you could use a Recovery Services vault to get a bird’s eye view of items being backed up under that vault, along with the associated jobs, policies, and alerts. But as your backup estate expands to span multiple vaults across subscriptions, regions, and tenants, monitoring this estate in real-time becomes a non-trivial task, requiring you to write your own customizations.

What if there was a simpler way to aggregate information across your entire backup estate into a single pane of glass, enabling you to quickly identify exactly where to focus your energy on?

Today, we are pleased to share the preview of Backup Explorer. Backup Explorer is a built-in Azure Monitor Workbook enabling you to have a single pane of glass for performing real-time monitoring across your entire backup estate on Azure. It comes completely out-of-the-box, with no additional costs, via native integration with Azure Resource Graph and Azure Workbooks.

Key Benefits

1) At-scale views – With Backup Explorer, monitoring is no longer limited to a Recovery Services vault. You can get an aggregated view of your entire estate from a backup perspective. This includes not only information on your backup items, but also resources that are not configured for backup, ensuring that you don’t ever miss protecting critical data in your growing estate. And if you are an Azure Lighthouse user, you can view all of this information even across multiple tenant, enabling truly boundary-less monitoring.

2) Deep drill-downs – You can quickly switch between aggregated views and highly granular data for any of your backup-related artifacts, be it backup items, jobs, alerts or policies.

3) Quick troubleshooting and actionability – The at-scale views and deep drill-downs are designed to aid you in getting to the root cause of a backup-related issue. Once you identify an issue, you can act on it by seamlessly navigating to the backup item or the Azure resource, right from Backup Explorer.

Backup Explorer is currently supported for Azure Virtual Machines. Support for other Azure workloads will be added soon.

At Azure Backup, Backup Explorer is just one part of our overall goal to enable a delightful, enterprise-ready management-at-scale experience for all our customers.

Getting Started

To get started with using Backup Explorer, you can simply navigate to any Recovery Services vault and click on Backup Explorer in the quick links section.

You will be redirected to Backup Explorer which gives a view across all the vaults, subscriptions, and tenants that you have access to.

Continue reading

10 recommendations for cloud privacy and security with Ponemon research

Today we’re pleased to publish Data Protection and Privacy Compliance in the Cloud: Privacy Concerns Are Not Slowing the Adoption of Cloud Services, but Challenges Remain, original research sponsored by Microsoft and independently conducted by the Ponemon Institute. The report concludes with a list of 10 recommended steps that organizations can take to address cloud privacy and security concerns, and in this blog, we have provided information about Azure services such as Azure Active Directory and Azure Key Vault that help address all 10 recommendations.

The research was undertaken to better understand how organizations undergo digital transformation while wrestling with the organizational impact of complying with such significant privacy regulations as the European Union’s General Data Protection Regulation (GDPR). The research explored the reasons organizations are migrating to the cloud, the security and privacy challenges they encounter in the cloud, and the steps they have taken to protect sensitive data and achieve compliance.

The survey of over 1,000 IT professionals in the US and EU found that privacy concerns are not slowing cloud adoption and that most privacy-related activities are easier in the cloud, while at the same time, most organizations don’t feel they have control and visibility they need to manage online privacy.  The report lists ten steps organizations can take to improve security and privacy.

Key takeaways from the research include:

◉ Privacy concerns are not slowing the adoption of cloud services, as only one-third of US respondents and 38 percent of EU respondents say privacy issues have stopped or slowed their adoption of cloud services. The importance of the cloud in reducing costs and speeding time to market seem to override privacy concerns.

◉ Most privacy-related activities are easier to deploy in the cloud. These include governance practices such as conducting privacy impact assessments, classifying or tagging personal data for sensitivity or confidentiality, and meeting legal obligations, such as those of the GDPR. However, other items such as managing incident response are considered easier to deploy on premises than in the cloud.

◉ 53 percent of US and 60 percent of EU respondents are not confident that their organization currently meets their privacy and data protection requirements. This lack of confidence may be because most organizations are not vetting cloud-based software for privacy and data security requirements prior to deployment.

◉ Organizations are reactive and not proactive in protecting sensitive data in the cloud. Specifically, just 44 percent of respondents are vetting cloud-based software or platforms for privacy and data security risks, and only 39 percent are identifying information that is too sensitive to be stored in the cloud.

◉ Just 29 percent of respondents say their organizations have the necessary 360-degree visibility into the sensitive or confidential data collected, processed, or stored in the cloud. Organizations also lack confidence that they know all the cloud applications and platforms that they have deployed.

The Ponemon report closes with a list of recommended steps that organizations can take to address cloud privacy and security concerns, annotated below with relevant Azure services that can help you implement each of the recommendations:

1. Improve visibility into the organization’s sensitive or confidential data collected, processed, or stored in the cloud environment.

Azure service: Azure Information Protection helps discover, classify, and control sensitive data.

2. Educate themselves about all the cloud applications and platforms already in use in the organization.

Azure service: Microsoft Cloud App Security helps discover and control the use of shadow IT by identifying cloud apps, infrastructure as a service (IaaS), and platform as a service (PaaS) services.

3. Simplify the authentication of users in both on-premises and cloud environments.

Azure service: Azure Active Directory provides tools to manage and deploy single sign-on authentication for both cloud and on-prem services.

4. Ensure the cloud provider offers event monitoring of suspicious and anomalous traffic in the cloud environment.

Azure service: Azure Monitor enables customers to collect, analyze, and act on telemetry data from both Azure and on-premises environments.

5. Implement the capability to encrypt sensitive and confidential data in motion and at rest.

Azure service: Azure offers a variety of options for encrypting both data at rest and in transit.

6. Make sure that the organization uses and manages its own encryption keys (BYOK).

Azure service: Azure Key Vault allow you to import or generate keys in hardware security modules (HSMs) that never leave the HSM boundary.

7. Implement multifactor authentication before allowing access to the organization’s data and applications in the cloud environment.

Azure service: Azure Active Directory offers multiple options for deploying multifactor authentication for both cloud and on-prem services.

8. Assign responsibility for ensuring compliance with privacy and data protection regulations and security safeguards in the cloud to those most knowledgeable: the compliance and IT security teams. Privacy and data protection teams should also be involved in evaluating any cloud applications or platforms under consideration.

Azure service: Role-based access control (RBAC) helps manage who has access to Azure resources, what they can do with those resources, and what areas they have access to.

9. Identify information that is too sensitive to be stored in the cloud and assess the impact that cloud services may have on the ability to protect and secure confidential or sensitive information.

Azure service: Azure Information Protection helps discover, classify, and control sensitive data.

10. Thoroughly evaluate cloud-based software and platforms for privacy and security risks.

Azure service: Microsoft Cloud App Security Assess the risk levels and business readiness of over 16,000 apps.

source: microsoft.com

Continue reading