Delivering consistency and transparency for cloud hardware security

When it comes to building the Microsoft Cloud, our work to standardize designs for systems, boards, racks, and other parts of our datacenter infrastructure is paramount to facilitating forward progress and innovation across the computing industry. Microsoft has made a number of contributions to and collaborated with various members of the Open Compute Project (OCP) community, the leading industry group dedicated to open source hardware innovation. This year, we are excited to showcase some of our newest projects at the OCP Global Summit and share our learnings on the path of building a more reliable, trusted, and sustainable cloud. One of the key areas where we’ve seen continued focus and opportunity is driving industrywide standards around platform security. To dive deeper into our contributions in this area, I’ve invited Mark Russinovich, CTO and Technical Fellow, Azure, and Bryan Kelly, Partner Architect, Azure Hardware Systems and Infrastructure, to share more about Microsoft’s newest security contributions to OCP that standardize the foundations of trust, integrity, and reliability in computing.

Securing customer workloads from the cloud to the edge

Microsoft Azure is a leader in cloud security and privacy offering a broad range of confidential computing services to help organizations run workloads that keep business and customer data private with advanced levels of security. As the demand for confidential computing grows from cloud to edge, so do the requirements for consistency and transparency of the security mechanisms that protect workloads. With the rise of edge computing, the resultant growth in the exposed attack surface also presents a need for stronger physical security solutions. In this context, there is an increased need for greater transparency in the infrastructure that underpins these technologies and upholds hardware security promises.

Caliptra: Integrating trust into every chip

At the Open Compute Project (OCP) Summit, we are jointly announcing Caliptra, an open source root of trust (RoT) that produces cryptographic proofs about the hardware protections in place for confidential workloads. Designed with security experts and industry leaders in confidential computing across AMD, Google, Microsoft, and NVIDIA, Caliptra is a forward-looking approach casting transparency into hardware security. As a reusable open source, silicon-level block for integration into systems on a chip (SoCs)—such as CPUs, GPUs, and accelerators—Caliptra provides trustworthy and easily verifiable attestation.

At its core, Caliptra provides foundational security properties that underpin the integrity of higher-level security protection for confidential workloads. The Caliptra RoT has the following essential security properties:

◉ Identity: A unique device manufacturer’s cryptographic identity for attestation endorsement. The identity is consistent with TCG DICE and includes intrinsic attestation of the Caliptra firmware.

◉ Compartmentalization: Hardware protection barriers that isolate Caliptra’s security assets.

◉ Measurement: Cryptographic digests that represent the SoC security configuration in a concise, cryptographically verifiable manner.

◉ Renewable security: The security state of the hardware can be brought forward to a secure state and the life cycle management extends through manufacturing, field operations, circular economy, and eventually end of life.

◉ Ownership: Provides verification of both authenticity and integrity of all mutable firmware loaded into an SoC. This property uses hardware-enforced digital signatures to ensure integrity policies are upheld.

◉ Attestation: The endorsement of measurement claims with Caliptra’s cryptographic identity. These measurements provide precise information about the security state of the trusted computing base.

The initial Caliptra 0.5 contribution release to OCP contains a series of specifications describing architecture, integration, and implementation. An open sourced register-transfer level (RTL) code implementation of Caliptra that can be synthesized into current SoC designs will be made available, along with the cloud-designed firmware written entirely in Rust. With this trusted foundation designed for confidential cloud devices, Caliptra supports the consistent scaling of confidential workloads across distributed systems.

With deep ecosystem collaboration at the heart of Microsoft’s open source philosophy, we look forward to continuing working closely with our partners and engaging the industry to advance Caliptra. Caliptra RTL and firmware project collaboration will be done under the auspices of the CHIPS Alliance.

Hydra: A new secure Baseboard Management Controller (BMC)

We are also introducing Hydra, a new secure BMC in partnership with Nuvoton. A BMC is typically designed into every server system and expansion chassis—for example, JBOD or GPU. As a diagnostic and recovery controller, the BMC has special privileged hardware interfaces for acquiring debug data and telemetry from CPUs. These interfaces present security concerns, as they are targets for attacks that bypass conventional security defenses.

Azure uses Cerberus, a contribution we made to OCP in 2017 for hardware security, to improve BMC security by enforcing firmware integrity and preventing the persistence of malware in the BMC. However, as threat models evolve to restrict admins with physical access to hardware, the BMC needs security properties to establish secure links to an external RoT.

Microsoft collaborated with Nuvoton to design a new security-focused BMC, with enhanced hardware security throughout the BMC SoC. The silicon-integrated root of trust supports TCG DICE identity flows with hardware engines for fast cryptographic operations and hardware-managed keys. The RoT has a one-way bridge for activity monitoring and controlling the BMC security configuration, including which internal security peripherals the BMC can assess. This unique feature allows fine-grained BMC interface authorization, enabling scenarios whereby temporary access to a debug interface can be granted to the BMC only after it attests its trustworthiness.

Kirkland: A secure Trusted Platform Module (TPM)

While Microsoft provides multilayered security across our datacenters, infrastructure, and operations, we believe in defense-in-depth and that all interconnects should be cryptographically secured from interposer-based attack vectors. In partnership with Google, Infineon, and Intel, we are announcing Project Kirkland at OCP. Project Kirkland demonstrates how, using firmware-only updates to the TPM stack and CPU RoT, the interconnect between the TPM and CPU can be secured in a way that prevents substitution attacks, interposing, and eavesdropping. We are open sourcing this methodology and plan to work with the Trusted Computing Group on standardizing this approach while working with other TPM manufacturers to adopt the same methodology, so these techniques become available to all.

A discrete TPM is a chip typically used to protect secrets for the software running on the CPU and conditionally released based on the CPU's boot measurements. Historically, the bus between the CPU and the TPM is susceptible to attack from physical adversaries wishing to falsify attested measurements or obtain TPM-bound secrets. The standards-based firmware techniques used in Project Kirkland defend against such attacks by using cryptography to authenticate the caller and protect the transmission of secrets over the bus.

 

Open hardware innovation at cloud scale

A community-driven approach to infrastructure innovation is vital—not just for continued advancements in trust, efficiency, and scalability, but in service of a larger vision of empowering the ecosystem towards building the for computing needs of tomorrow.

We are also contributing several new hardware designs such as a new modular chassis (Mt. Shasta), a converged architecture that brings form factor, power, and management interface into a modular design—optimized for advanced workloads like high-performance computing, artificial intelligence, and video codecs. In partnership with Quanta and Molex, Mt. Shasta is designed to be fully compatible with Open Rack V3, with flexibility in changing module-module connectivity. Earlier this year, we also collaborated with Intel and contributed the Scalable I/O Virtualization (SIOV) specification to OCP. SIOV enables device and platform manufacturers to an industry standard for hyperscale virtualization of PCI Express and Compute Express Link devices in cloud servers, enabling more scalable, efficient, and cost-effective hardware designs for datacenters.

As the demand for cloud-scale computing and digital services continues to grow, Microsoft is committing to deep ecosystem collaboration with OCP and industry partners to deliver the systems and infrastructure that maximize performance, trust, and resiliency for cloud customers.

Source: microsoft.com

Continue reading

Azure Network Security

In this blog, I will focus on security from a network perspective and describe how you can use Azure network capabilities to build highly secure cloud services. Four distinct areas highlight how we provide a secure network to customers:

◈ The foundation is Azure Virtual Network to provide a secure network fabric that provides an isolation boundary for customer networks.
◈ Virtual Network configuration and policies protect cloud applications.
◈ Active monitoring systems and tools provide security validation.
◈ An underlying physical network infrastructure with built-in advanced security hardening protects the entire global network.

Isolating customer networks in single shared physical network

To support the tremendous growth of our cloud services and maintain a great networking experience, Microsoft owns and operates one of the largest dark fiber backbones in the world—it connects our datacenters and customers. In Azure, we run logical overlay networks on top of the shared physical network to provide isolated private networks for customers.

Figure 2. Isolated customer virtual networks run on the same physical network

The overlay networks are implemented by Azure’s software defined networking (SDN) stack. Each overlay network is specifically created on demand for a customer via an API invocation. All configuration for building such networks is performed in software—this is why Azure can scale up to create thousands of overlay networks in seconds. Each overlay network is its own Layer 3 routing domain that comprises the customer’s Virtual Network (VNet).

Azure Virtual Network

Azure Virtual Network is a secure, logical network that provides network isolation and security controls that you treat like your on-premises network. Each customer creates their own structure by using: subnets—they use their own private IP address range, configure route tables, network security groups, access control lists (ACLs), gateways, and virtual appliances to run their workloads in the cloud. 

Figure 3 shows an example of two customer virtual networks. Customer 1’s VNet has connectivity to an on premises corporate network, while Customer 2’s VNet can be accessed only via Remote Desktop Protocol (RDP). Network traffic from the Internet to virtual machines (VMs) goes through the Azure load balancer and then to the Windows Server host that’s running the VM. Host and guest firewalls implement network port blocking and ACL rules.

Figure 3. Customer isolation provided by Azure Virtual Network

The VMs deployed into the VNet can communicate with one another using private IP addresses. You control the IP address blocks, DNS settings, security policies, and routing tables. Benefits include:

◈ Isolation: VNets can be isolated from one another, so you can create separate networks for development, testing, and production. You can also allow your VNets to communicate with each other.

◈ Security: By using network security groups, you can control the traffic entering and exiting the subnets and VMs.

◈ Connectivity: All resources within the VNet are connected. You can use VNet peering to connect with other Virtual Networks in the same region. You can use virtual private network (VPN) gateways to enable IPsec connectivity to VNets via the Internet from on-premises sites and to VNets in other regions. ExpressRoute provides private network connectivity to VNets that bypasses the Internet.

◈ High availability: Load balancing is a key part of delivering high availability and network performance to customer applications. All traffic to a VM goes through the Azure Load Balancer.

Securing your applications

A December 2016 survey of security professionals showed that their biggest year-over-year drop in confidence was in “the security of web applications, [which was] down 18 points from 80 percent to 62 percent.” Microsoft addresses potential vulnerabilities by building security into our applications and providing features and services to help customers enhance the security of their cloud-hosted applications from the development phase all the way to controlling access to the service.

Azure has a rich set of networking mechanisms that customers can use to secure their applications. Here are some examples.

Network ACLs can be configured to restrict access on public endpoint IP addresses. ACLs on the endpoint further restrict the traffic to only specific sources IP addresses.

Network Security Groups (NSGs) control network access to VMs in your VNet. This collection of network ACLs allows a full five-tuple (source IP address, source port, destination IP address, destination port, protocol) set of rules to be applied to all traffic that enters or exits a subnet or a VM’s network interface. The NSGs, associated to a subnet or VM, are enforced by the SDN stack.

Network virtual appliances (NVAs) bolster VNet security and network functions, and they’re available from numerous vendors via the Azure Marketplace. NVAs can be deployed for highly available firewalls, intrusion prevention, intrusion detection, web application firewalls (WAFs), WAN optimization, routing, load balancing, VPN, certificate management, Active Directory, and multifactor authentication.

Many enterprises have strict security and compliance requirements that require on-premises inspection of all network packets to enforce specific polices. Azure provides a mechanism called forced tunneling that routes traffic from the VMs to on premises by creating a custom route or by Border Gateway Protocol (BGP) advertisements through ExpressRoute or VPN.

Figure 4 shows an example of using NSG rules on segregated subnets and an NVA to protect the front end subnet. 

Figure 4. A perimeter network architecture built using Network Security Groups

Azure Application Gateway, our Layer 7 load balancer, also provides Web Application Firewall (WAF) functionality to protect against the most common web vulnerabilities.

Securely connecting from on-premises to Azure can be achieved via the Internet using IPsec to access our VPN Gateway service or with a private network connection using ExpressRoute. Figure 4 illustrates a perimeter network–style enhanced security design where Virtual Network access can be restricted using NSGs with different rules for the front end (Internet-facing) web server and the back-end application servers.

Figure 5. A secured VNet connected to an Internet front-end and back-end connected to on-premises

Security validation

Azure offers many tools to monitor, prevent, detect, and respond to security events. Customers have access to the Azure Security Center, which gives you visibility and control over the security of your Azure resources. It provides integrated security monitoring and policy management, helps detect threats, and works with a broad ecosystem of security solutions.

We also provide Network Watcher to monitor, diagnose, and gain insights into your Azure network. With diagnostic and visualization tools to monitor your network’s security and performance, you can identify and resolve network issues. For example, to view information about traffic coming into and going out of an NSG, Network Watcher provides NSG flow logs. You can verify that the NSGs are properly deployed, and see which unauthorized IPs are attempting to access your resources.

Figure 6. Capture NSG Flow Logs using Network Watcher

Network infrastructure security hardening

According to a 2015 Ponemon study, for businesses, the average cost per security breach is $15 million. To help protect your organization’s assets, Microsoft Cloud datacenters are protected by layers of defense-in-depth security, including perimeter fencing, video cameras, security personnel, secure entrances, real-time communications networks, and all physical servers are monitored. These regularly audited security measures help Azure achieve our strong portfolio of compliance certifications. 

For many years, we’ve used encryption in our products and services to protect our customers from online criminals and hackers. We don’t want to take any chances with customer data being breached and are addressing this issue head on. We have a comprehensive engineering effort to strengthen the encryption of customer data across our networks and services. This effort will provide protection across the full lifecycle of customer-created content.

Azure traffic between our datacenters stays on our global network and does not flow over the Internet. This includes all traffic between Microsoft Azure public cloud services anywhere in the world. For example, within Azure, traffic between VMs, storage, and SQL stays on the Microsoft network, regardless of the source and destination region. Intra-region VNet-to-VNet, as well as cross-region VNet-to-VNet traffic, stays on the Microsoft network.

Distributed denial of service (DDoS) attacks are a continually rising threat. Protecting against the growing scale and complexity of such attacks requires significant infrastructure deployed at global scale. Azure has a built-in DDoS protection system to shield all Microsoft cloud services. Therefore, all Azure public IPs fall under this protection deployed across all Azure datacenters. Our DDoS system uses dynamic threat detection algorithms to prevent common DDoS volumetric attacks (such as UDP floods, SYN-ACK attacks, or reflection attacks). We monitor hundreds of daily mitigated attack attempts and continually expand our protection.

Azure itself is also protected through active monitoring and intelligence gathering across the Internet. We continuously perform threat intelligence research into the dark web to identify and mitigate potential risks and attacks. This knowledge is applied to our protection techniques and mitigations. The Microsoft Cyber Defense Operations Center, highlighting our commitment, responds to security incidents.

Putting these investments together, we provide a layered security model, as shown in Figure 8 to protect your services running in Azure.

Figure 8. A layered approach to securing Azure

Secure Azure Networking

Azure has made significant investments in security. Customers can use Virtual Networks and our other security features and services to design, configure, and monitor their cloud applications. We aggressively monitor and continually harden our global infrastructure to address the ever-changing landscape of new cyber threats.

Microsoft continues to be a leader in the prevention of network security attacks. With our global footprint and experience running the most popular cloud services, we have both scale and a breadth of inputs to secure our network and help you secure your services. We will continue to invest in network security technologies so that you can safely—and in a compliant manner—build, deploy, monitor, and run your services in Azure.

Continue reading