Choosing between Azure VNet Peering and VNet Gateways

As customers adopt Azure and the cloud, they need fast, private, and secure connectivity across regions and Azure Virtual Networks (VNets). Based on the type of workload, customer needs vary. For example, if you want to ensure data replication across geographies you need a high bandwidth, low latency connection. Azure offers connectivity options for VNet that cater to varying customer needs, and you can connect VNets via VNet peering or VPN gateways.

It is not surprising that VNet is the fundamental building block for any customer network. VNet lets you create your own private space in Azure, or as I call it your own network bubble. VNets are crucial to your cloud network as they offer isolation, segmentation, and other key benefits.

VNet peering

VNet peering enables you to seamlessly connect Azure virtual networks. Once peered, the VNets appear as one, for connectivity purposes. The traffic between virtual machines in the peered virtual networks is routed through the Microsoft backbone infrastructure, much like traffic is routed between virtual machines in the same VNet, through private IP addresses only. No public internet is involved. You can peer VNets across Azure regions, too – all with a single click in the Azure Portal.

◈ VNet peering - connecting VNets within the same Azure region

◈ Global VNet peering - connecting VNets across Azure regions

VPN gateways

A VPN gateway is a specific type of VNet gateway that is used to send traffic between an Azure virtual network and an on-premises location over the public internet. You can also use a VPN gateway to send traffic between VNets. Each VNet can have only one VPN gateway.

Which is best for you?

While we offer two ways to connect VNets, based on your specific scenario and needs, you might want to pick one over the other.

VNet Peering provides a low latency, high bandwidth connection useful in scenarios such as cross-region data replication and database failover scenarios. Since traffic is completely private and remains on the Microsoft backbone, customers with strict data policies prefer to use VNet Peering as public internet is not involved. Since there is no gateway in the path, there are no extra hops, ensuring low latency connections.

VPN Gateways provide a limited bandwidth connection and is useful in scenarios where encryption is needed, but bandwidth restrictions are tolerable. In these scenarios, customers are also not as latency-sensitive.

VNet Peering and VPN Gateways can also co-exist via gateway transit

Gateway transit enables you to use a peered VNet’s gateway for connecting to on-premises instead of creating a new gateway for connectivity. As you increase your workloads in Azure, you need to scale your networks across regions and VNets to keep up with the growth. Gateway transit allows you to share an ExpressRoute or VPN gateway with all peered VNets and lets you manage the connectivity in one place. Sharing enables cost-savings and reduction in management overhead.

With gateway transit enabled on VNet peering, you can create a transit VNet that contains your VPN gateway, Network Virtual Appliance, and other shared services. As your organization grows with new applications or business units and as you spin up new VNets, you can connect to your transit VNet with VNet peering. This prevents adding complexity to your network and reduces management overhead of managing multiple gateways and other appliances.

Differences between VNet Peering and VPN Gateways

	VNet Peering	VPN Gateways
Cross-region support?	Yes – via Global VNet Peering	Yes
Cross-Azure Active Directory tenant support?	Yes	Yes
Cross-subscription support?	Yes	Yes
Cross-deployment model support?	Yes	Yes
Limits	You can keep up to 500 VNets with one VNet as seen in the documentation on Networking Limits.	Each VNet can only have one VPN Gateway. VPN Gateways depending on the SKU have type different number of tunnel supported.
Pricing	Ingress/Egress charged.	Gateway + Egress charged.
Encrypted?	Software level encryption is recommended	Yes, custom IPsec/IKE policy can be created and applied to new or existing connections.
Bandwidth limitations?	No bandwidth limitations.	Varies based on type of Gateway from 100 Mbps to 1.25Gps.
Private?	Yes, no Public IP endpoints. Routed through Microsoft backbone and is completely private. No public internet involved.	Public IP involved.
Transitive relationship	If VNet A is peered to VNet B, and VNet B is peered to VNet C, VNet A and VNet C cannot currently communicate. Spoke to spoke communication can be achieved via NVAs or Gateways in the hub VNet.	If VNet A, VNet B, and VNet C are connected via VPN Gateways and BGP is enabled in the VNet connections, transitivity works.
Typical customer scenarios	Data replication, database failover, and other scenarios needing frequent backups of large data.	Encryption specific scenarios that are not latency sensitive and do not need high throughout.
Initial setup time	It took me 24.38 seconds, but you should give it a shot!	30 mins to set it up

Continue reading

Accessing virtual machines behind Azure Firewall with Azure Bastion

Azure Virtual Network enables a flexible foundation for building advanced networking architectures. Managing heterogeneous environments with various types of filtering components, such as Azure Firewall or your favorite network virtual appliance (NVA), requires a little bit of planning.

Azure Bastion, which is currently in preview, is a fully managed platform as a service (PaaS) that provides secure and seamless remote desktop protocol (RDP) and secure shell (SSH) access to your virtual machines (VMs) directly through the Azure portal. Azure Bastion is provisioned directly in your virtual network, supporting all VMs attached without any exposure through public IP addresses.

When you deploy Azure Firewall, or any NVA, you invariably force tunnel all traffic from your subnets. Applying a 0.0.0.0/0 user-defined route can lead to asymmetric routing for ingress and egress traffic to your workloads in your virtual network.

While not trivial, you often find yourself creating and managing a growing set of network rules, including DS NAT, forwarding, and so on, for all your applications to resolve this. Although this can impact all your applications, RDP and SSH are the most common examples. In this scenario, the ingress traffic from the Internet may come directly to your virtual machine within your virtual network, but egress traffic will end up going to the NVA. Since most NVAs are stateful, it ends up dropping this traffic as it did not initially receive it.

Azure Bastion, allows for simplified set up of RDP/SSH to your workloads within virtual networks containing stateful NVAs or Azure Firewall with force tunneling enabled. In this blog, we will look at how to make that work seamlessly.

Having deployed both Azure Bastion and Azure Firewall in your virtual network, let us look at how you can configure Azure Bastion to work in this scenario.

Configuring Azure Bastion

When deploying Azure Firewall, or a virtual appliance, you may end up associating your RouteTable, which was created while deploying Azure Firewall, to all subnets in your virtual network. You may even be including the AzureBastionSubnet subnet as well. 

This applies a user-defined route to the AzureBastionSubnet subnet which directs all Azure Bastion traffic to Azure Firewall, thereby blocking traffic required for Azure Bastion. To avoid this, configuring Azure Bastion is very easy, but do not associate the RouteTable to AzureBastionSubnet subnet.

As you would have noticed above, myRouteTable is not associated with the AzureBastionSubnet, but with other subnets like Workload-SN.

The AzureBastionSubnet subnet is secure platform managed subnet, and no other Azure Resource can deploy in this subnet except Azure Bastion. All connections to Azure Bastion are enforced through the Azure Active Directory token-based authentication with 2FA, and all traffic is encrypted/over HTTPS. 

Azure Bastion is internally hardened and allows traffic only through port 443, saving you the task of applying additional network security groups (NSGs) or user-defined routes to the subnet.

With this, the RDP/SSH requests will land on Azure Bastion. Configured using the example above, the default route (0.0.0.0/0) does not apply to AzureBastionSubnet as it's not associated with this subnet. Based on the incoming RDP/SSH requests, Azure Bastion connects to your virtual machines in other subnets, like Workload-SN, which do have a default route associated. The return traffic from your virtual machine will go directly to Azure Bastion, instead of going to the NVA, in your virtual network as the return traffic is directed to a specific private IP in your virtual network. The specific private IP address in your virtual network makes it a more specific route and hence, takes precedence over the force-tunnel route to the NVA, making your RDP/SSH traffic work seamlessly with Azure Bastion when a NVA or Azure Firewall is deployed in your virtual network.

Continue reading

Enhancing the customer experience with the Azure Networking MSP partner program

We are always looking for ways to improve the customer experience and allow our partners to complement our offerings. In support of these efforts we are sharing the Azure Networking Managed Service Provider (MSP) program along with partners that deliver value added managed cloud network services to help enterprise customers connect, operationalize, and scale their mission critical applications running in Azure.

Azure Networking MSP Partner Program enables partners such as networking focused MSPs, network carriers, and systems integrators (SIs) to use their rich networking experience to offer cloud and hybrid networking services around Azure’s growing portfolio of Azure Networking products and services.

Azure’s Networking services are fundamental building blocks critical to cloud migration, optimal connectivity, and security of applications. New networking services such as Virtual WAN, ExpressRoute, Azure Firewall, and Azure Front Door further enrich this portfolio allowing customers to deploy richer applications in the cloud. The Networking MSP partners can help customers deploy and manage Azure Networking services.

Azure Networking MSPs

Azure MSPs play a critical role in enterprise cloud transformation by bringing their deep knowledge and real-world experience to help enterprise customers migrate to Azure. Azure MSPs and the Azure Expert MSP program make it easy for customers to discover and engage specialized MSPs.

Azure Networking MSPs are a specialized set of MSPs for addressing enterprise cloud networking needs and challenges across all aspects of cloud and hybrid networking. Their managed network services and offerings include various aspects of the application lifecycle including network architecture, planning, deployment, operations, maintenance, and optimization.

Azure Lighthouse - unblocking Azure Networking MSPs

Many enterprise customers, such as banks and financial institutions want partners who can help them with managing their Azure Networking subscriptions. However, the need for individual customer management for these subscriptions introduces a lot of manual work for these service providers.

Last week, we announced Azure Lighthouse, which is a unique set of capabilities on Azure, empowering service provider partners with a single control plane to view and manage Azure at scale across all their customers with higher automation and efficiency. We also talked about how Azure Lighthouse enables management at scale for service providers.

With Azure Lighthouse, Azure Networking MSPs can seamlessly onboard customers via managed services offers on the Azure marketplace or natively via ARM templates – empowering them to deliver a rich set of managed network experiences for their end-customers.

Azure Networking MSP partners

Azure Networking partners play a big role in the Azure networking ecosystem, delivering Virtual WAN CPEs and hybrid networking services such as ExpressRoute to enterprises that are building cloud infrastructures. We welcome the following Azure Networking MSP launch partners into our Azure Networking MSP partner ecosystem.

These partners have invested in people, best practices, operations and tools to build and harness deep Azure Networking knowledge and service capabilities. They’ve trained their staff on Azure and have partnered closely with us in Azure Networking through technical workshops and design reviews.

These partners are also early adopters of Azure Lighthouse, building and delivering a new generation of managed network experiences for their end customers. We encourage all worldwide networking MSPs, network carriers, and SIs that would like to join this program to reach out via ManagedVirtualWAN@microsoft.com to join the Azure Networking MSP program and bring your unique value and services to Azure customers.

Continue reading

Azure Firewall and network virtual appliances

Network security solutions can be delivered as appliances on premises, as network virtual appliances (NVAs) that run in the cloud or as a cloud native offering (known as firewall-as-a-service).

Customers often ask us how Azure Firewall is different from Network Virtual Appliances, whether it can coexist with these solutions, where it excels, what’s missing, and the TCO benefits expected. We answer these questions in this blog post.

Network virtual appliances (NVAs)

Third party networking offerings play a critical role in Azure, allowing you to use brands and solutions you already know, trust and have skills to manage. Most third-party networking offerings are delivered as NVAs today and provide a diverse set of capabilities such as firewalls, WAN optimizers, application delivery controllers, routers, load balancers, proxies, and more. These third party capabilities enable many hybrid solutions and are generally available through the Azure Marketplace.

Cloud native network security

A cloud native network security service (known as firewall-as-a-service) is highly available by design. It auto scales with usage, and you pay as you use it. Support is included at some level, and it has a published and committed SLA. It fits into DevOps model for deployment and uses cloud native monitoring tools.

What is Azure Firewall?

Azure Firewall is a cloud native network security service. It offers fully stateful network and application level traffic filtering for VNet resources, with built-in high availability and cloud scalability delivered as a service. You can protect your VNets by filtering outbound, inbound, spoke-to-spoke, VPN, and ExpressRoute traffic. Connectivity policy enforcement is supported across multiple VNets and Azure subscriptions. You can use Azure Monitor to centrally log all events. You can archive the logs to a storage account, stream events to your Event Hub, or send them to Log Analytics or your security information and event management (SIEM) product of your choice.

Is Azure Firewall a good fit for your organization security architecture?

Organizations have diverse security needs. In certain cases, even the same organization may have different security requirements for different environments. As mentioned above, third party offerings play a critical role in Azure. Today, most next-generation firewalls are offered as Network Virtual Appliances (NVA) and they provide a richer next-generation firewall feature set which is a must-have for specific environments/organizations.  In the future, we intend to enable chaining scenarios to allow you to use Azure Firewall for specific traffic types, with an option to send all or some traffic to a third party offering for further inspection. This third-party offering can be either a NVA or a cloud native solution.

Many Azure customers find the Azure Firewall feature set is a good fit and it provides some key advantages as a cloud native managed service:

◈ DevOps integration – easily deployed using Azure Portal, Templates, PowerShell, CLI, or REST.
Built in HA with cloud scale.
◈ Zero maintenance service model - no updates or upgrades.
◈ Azure specialization— for example, service tags, and FQDN tags.
◈ Significant total cost of ownership saving for most customers.

But for some customers third party solutions are a better fit.

The following table provides a high-level feature comparison for Azure Firewall vs. NVAs:

Figure 1: Azure Firewall versus Network Virtual Appliances – Feature comparison

Why Azure Firewall is cost effective

Azure Firewall pricing includes a fixed hourly cost ($1.25/firewall/hour) and a variable per GB processed cost to support auto scaling. Based on our observation, most customers save 30 percent – 50 percent in comparison to an NVA deployment model. We are announcing a price reduction, effective May 1, 2019, for the firewall per GB cost to $0.016/GB (-46.6 percent) to ensure that high throughput customers maintain cost effectiveness. There is no change to the fixed hourly cost.

The following table provides a conceptual TCO view for a NVA with full HA (active/active) deployment:

Cost	Azure Firewall	NVAs
Compute	$1.25/firewall/hour $0.016/GB processed (30%-50% cost saving)	Two plus VMs to meet peek requirements
Licensing		Per NVA vendor billing model
Standard Public Load Balancer		First five rules: $0.025/hour Additional rules: $0.01/rule/hour $0.005 per GB processed
Standard Internal Load Balancer		First five rules: $0.025/hour Additional rules: $0.01/rule/hour $0.005 per GB processed
Ongoing/Maintenance	Included	Customer responsibility
Support	Included in your Azure Support plan	Per NVA vendor billing model

Figure 2: Azure Firewall versus Network Virtual Appliances – Cost comparison

Continue reading

Azure Stack IaaS – part three

Foundation of Azure Stack IaaS

Remember back in the virtualization days when you had to pick a host for your virtual machine? Some of my business units could tell by the naming convention the make and manufacturer of the hardware. Using this knowledge, they’d fill up the better gear first, leaving the teams that didn’t know better with the oldest hosts.

Clouds take a different approach. Instead of hosts, VMs are placed into a pool of capacity. The physical infrastructure is abstract. The compute, storage, and networking resources consumed by the VM are defined through software.

Azure Stack is an instance of the Azure cloud that you can run in your own datacenter. Microsoft has taken the experience and technology from running one of the largest clouds in the world to design a solution you can host in your facility. This forms the foundation of Azure Stack’s infrastructure-as-service (IaaS).

Let’s explore some of the characteristics of the Azure Stack infrastructure that allows you to run cloud-native VMs directly in your facility.

Cloud inspired hardware

Microsoft employees can’t just purchase their favorite server and rack it into an Azure datacenter. The only servers that enter an Azure datacenter have been specifically built for Azure. Not only are the servers built for Azure, so are the networking devices, the racks, and the cabling. This extreme standardization allows the Azure team to operate an Azure datacenter with just a handful of employees. Because all the servers are standardized and can be uniformly operated and automated, adding additional capacity to a datacenter doesn’t require hiring more employees to operate them.

Other advantages of standardizing hardware configurations is the standardization leads to expected, repeatable results – not only for Microsoft and Azure, but for its customers. The hardware integration has been validated and is a known recipe. Servers, storage, networking, cabling layout, and more are all well-known and based on these recipes, the ordering, delivery, and integration of new hardware components. Servicing and eventual retirement are repeatable and scalable. The full end-to-end validation of these configurations is done once with quick checks in place when the capacity is delivered and installed.

These principles are applied to Azure Stack solutions as well. The configurations, their capabilities, and validation are all well-known and the result is a repeatable and supportable product. Microsoft, its partners, and most importantly the end customer benefit. While an Azure Stack customer is limited to the defined, partner solutions, they have been built with reasonable flexibility so the customer can choose the specific capabilities or capacities required. Please note, there is one exception – the Azure Stack Development Kit (ASDK) allows you to install Azure Stack on any hardware that meets the hardware requirements. The ASDK is for evaluation purposes and not supported as a production environment.

Azure Stack hardware partnerships

Microsoft has partnered and co-engineered solutions with a variety of hardware partners or OEMs. The benefit is that Azure Stack can meet you where your existing relationships exist. These relationships may be based on existing hardware purchasing agreements, geographic location, or support capabilities. Keeping in mind the principles of operating a solution in a well-defined manner, Microsoft has set minimum requirements for Azure Stack hardware solutions. Each of our partners can then choose from their portfolio the components, servers, and network switches that best meet your needs. This creates a well-defined variety that continues to be supportable and delivers the overall solution value.

Resiliency to failure

One of the principles we have taken from Microsoft’s experience in the enterprise and from Azure is overall solution resilience. The world of software and hardware is not perfect; things fail – cables go bad, software has bugs, power outages occur, and on and on. While we work to build better software and with our solution partners to continually improve, we must expect that things fail. Azure Stack solutions are not perfect, but have been constructed with the intent to overcome the common points of failure. For example, each copy of tenant/user data is stored on three separate storage devices in three separate servers. The physical network paths are redundant and provide better performance and resiliency to potential failure. The internal software of Azure Stack are services that coordinate across multiple instances. This type of end-to-end architectural design and implementation leads to a better end experience. Combining this approach to infrastructure resilience with the well-known and validated solutions approach described above provides for a better experience for the customer.

Hardened by default

When you run your IaaS VMs in Azure Stack you should know they are running on a secure foundation. It turns out that one of the reasons people select Azure Stack is because they have data and/or processes that are either regulated or defined in a contractual agreement. Azure Stack not only gives its owners control of their data and processes, it comes with an infrastructure which is secured by default. In fact, the underlying infrastructure is locked down in a way that neither the owner nor Microsoft can access it. If it ever needs to be accessed because of a support issue, both the owner and Microsoft combine their keys to obtain access to the system and for a limited time.

Azure leads the industry in security compliance, and security compliance is important for Azure Stack as well. In Azure, Microsoft fully manages the technology, people, and processes as well as its compliance responsibilities. Things are different with Azure Stack. While the technology is provided by Microsoft, the people and processes are managed by the operator. To help operators jump-start the certification process, Azure Stack has gone through a set of formal assessments by a third party-independent auditing firm to document how the Azure Stack infrastructure meets the applicable controls from several major compliance standards. The documentation is an assessment of the technology not a certification of Azure Stack due to the standards including several personnel-related and process-related controls, but they help you get started. The technology assessments include the following standards:

◈ PCI-DSS – Addresses the payment card industry
◈ CSA Cloud Control Matrix –  A comprehensive mapping across multiple standards, including FedRAMP Moderate, ISO27001, HIPAA, HITRUST, ITAR, NIST SP800-53, and others
◈ FedRAMP High – For government customers

If you face compliance mandates or internal processes that demand that you originate and manage your cloud data encryption keys, and even for Azure Stack, the CipherTrust Cloud Key Manager (CCKM) from Thales works with Azure and Azure Stack “Bring Your Own Key” (BYOK) API’s to enable such key control. CipherTrust Cloud Key Manager creates Azure-compatible keys from a FIPS 140-2 source. You can then can upload, manage, and revoke, if needed, to and from Azure Key Vaults running in Azure Stack or Azure, all from a single pane of glass.

For instance, you could create a salary app on Azure Stack, generate data encryption keys with CipherTrust Cloud Key Manager, and then set a policy to enable use of those keys in the Key Vault on Azure Stack only during the last week of the month when the app is computing the salaries. Among many other benefits, CCKM provides reduced time exposure for the keys, remote backup, a secure location for storing the keys, and the decoupling of management of the keys from the app itself. Not to mention automated key versioning. CCKM supports both Azure Active Directory (AAD) and Active Directory Federation Services (ADFS) deployments.

Get started by reviewing your options

As noted earlier, Azure Stack is sold as an integrated hardware system, with software pre-installed on the validated hardware. It typically comes in a standard server rack. You choose where your system will be located. You can host it in your data center or perhaps you want to run it in a service provider’s facility.

With the Azure Stack running in your location of choice, you also have a choice of who operates the Azure Stack infrastructure. An Azure Stack operator is responsible for giving access to the Azure Stack, keeping the software and firmware up-to-date, providing the content in the marketplace, monitoring the system health, and diagnosing issues. Azure Stack provides automation, documentation, and training for all of these processes so that someone from your organization can operate Azure Stack. We also provide trained partner experts who can operate your Azure Stack either in their facility or yours.

Here is an overview of your options when you acquire your Azure Stack:

A system you manage	A managed service
◈ Typically on-premises ◈ You control management and ops ◈ Buy Azure Stack from Microsoft ◈ Buy hardware from the vendor ◈ Call Microsoft for support	◈ Typically at service provider premises ◈ Service is managed for you ◈ Buy service from service provider ◈ Service includes hardware and software ◈ Call the service provider for support

Tuning your IaaS VMs for a cloud infrastructure

Once you have your Azure Stack up and running and you begin to plan your first IaaS VM deployments, you need to think about these VMs as cloud deployments, not virtualization deployments. IaaS VMs run best when they take advantage of the cloud infrastructure that they are running on. Many times, the way you tune a VM in your cloud infrastructure will be very different than the way you tuned VMs in your traditional virtualization environment. That said, you can always start with what you already have and improve those solutions through modern operations.

A great example of this is the use of multiple disks to get the needed IOps and throughput required of the application. As is the case in Azure, virtual machines placed in Azure Stack have limits applied for their disk activity. This limits the impact of one VM’s activity on another VM – aka noisy neighbor. While these limits are great for IaaS environments, it may take extra work to deploy workloads that get the appropriate resources needed, and in this example, it is IOps.

For optimization of SQL Server deployments, our documentation provides guidance on how to configure storage to obtain the needed performance. In this case, the approach is to attach multiple disks and stripe them to obtain the capacity and performance. When you use managed disks for your VMs, it allows the system to optimize where the physical data gets stored within your Azure Stack. Moving from virtualization environments to IaaS is reasonably straightforward and has its benefits, but requires a little bit of work on your first deployment. You can always use tools like Azure Monitor and the Virtual Machine solutions to better understand your workloads and gain insights on the performance of your VMs. When your VMs are not answering the performance requirements, you can also use the Azure Performance Diagnostics VM Extension for Windows to troubleshoot and identify potential bottlenecks.

The great thing about IaaS, and specifically Azure Stack, is the ability to easily reuse the deployment templates or artifacts to reduce the work for migration of similar workloads.

Infrastructure purpose built for running cloud-native VMs

Few organizations can claim that they have experience building one the largest cloud infrastructures in the world. When you buy an Azure Stack, you get the benefit of Microsoft’s Azure experience. Microsoft has partnered with the best OEMs to deliver a standardized configuration so that you don’t have to worry about these details. The infrastructure of Azure Stack is purpose-built to get the best for your IaaS VMs – keeping them safe, secure, and performant.

Continue reading

How to develop secure applications using Azure Cosmos DB

Before we begin to discuss how to develop secure applications using Azure Cosmos DB, we should also highlight some of the different layers of security that Azure Cosmos DB offers. The following image illustrates these various layers of security:

Azure Cosmos DB is a ring zero Azure service, this means it will be available in any new Azure data center as soon as it goes online and must keep all its compliance certificates current.

The first layer of Azure provides physical safety of data centers and continuous protections from DDoS attacks. Azure has dedicated teams to continuously monitor the security issues. All Azure services run a common security agent to collect anomalous activity. Production resources are patched regularly and all the secrets, certificates, or passwords have a defined lifetime. These certificates or secrets should be rotated after they expire. All the production ports in Azure Cosmos DB are scanned and penetration tested regularly. The source code is scanned for security issues and they require two approvers before integrating into the product. 

Access to Azure is restricted. Operations staff or developers working for Azure Cosmos DB cannot access any production from their machines. All production is accessed via dedicated secure access workstations (SAWs). These workstations are watertight, there is no outside access from these machines unless it is through Azure. Engineers get just in time (JIT) approval for all production access. All access is monitored and every activity of an engineer is monitored by an escort.  All production deployments require multiple approvals including test sign off and approvers.

Azure has a very strict access policy by Microsoft personnel. It is almost impossible for any Microsoft employee to access the production system if he or she is not authorized. I am mentioning these details because customers often ask if their data is secure in Azure and whether Azure Cosmos DB engineering teams can access their data.

Security offered by IP firewall

Using an IP firewall is the first layer of protection to secure your database. Azure Cosmos DB supports policy driven IP-based access controls for inbound firewall support. This model is like the firewall rules of a traditional database system and provides an additional level of security to the Azure Cosmos DB account. With this model, you can now configure an Azure Cosmos DB account to be accessible only from an approved set of machines and/or cloud services. Access to Azure Cosmos DB resources from these approved sets of machines and services still require the caller to present a valid authorization token.

The IP access control policy can be set in the Azure portal, or programmatically through Azure CLI, Azure PowerShell, or the REST API by updating the “ipRangeFilter” property.

Security offered by virtual network

Virtual network is the next layer that secures Azure Cosmos DB accounts. You can configure your Azure Cosmos DB account to allow access only from a specific subnet of Azure Virtual Networks. By enabling a service endpoint for Azure Cosmos DB from a virtual network and its subnet, traffic is ensured an optimal and secure route to Azure Cosmos DB.

Once an Azure Cosmos DB account is configured with a virtual network service endpoint, it can be accessed only from the specified subnet, and the public or Internet access is removed. To learn in detail about service endpoints, refer to the Azure “Virtual Network Service Endpoints” overview article. 

You can filter network traffic to and from Azure resources in an Azure Virtual Network with a network security group. A network security group contains security rules that allow or deny inbound network traffic and outbound network traffic from several types of Azure resources. 

Access control with keys

The security layers discussed so far come as part of Azure Cosmos DB and don’t require much other than configuring the firewall and virtual network. Now, let’s discuss what you can do as an application developer. All access to Azure Cosmos DB is controlled by two keys, a master key and read-only key. Master key, as its name implies is a master key and can do all operations on Azure Cosmos DB. A read-only key enables you to read the data, but no other actions are possible with this key.

Developers may worry about keys, wondering if someone could steal them and access their data. The rest of this post will discuss the architecture you should adopt to build a secure application on Azure Cosmos DB.

Architecture for building a secure application

First, no application should have access keys in the config file or in code. Keys should always be kept in Key Vault. Your application or users can have managed identities registered with Key Vault and they can get the keys at the run time.

Second, application should not try to access the database directly. They should have a middle tier web service to access data. This approach provides the following benefits:

◈ Frontend application can call REST service to get the data. Frontend does not need to understand different API’s of Azure Cosmos DB. Azure Cosmos DB is a multi-model database, data can be kept in different formats, but the frontend applications can be abstracted away from all the details of data access.

◈ Generally you may have many applications, in different formats (PC, phone, and web) which will be accessing the same data. Not all application must repeat the data retrieval logic.

◈ You can provide throttling if needed to any rogue application at the web service layer before they can harm the database.

◈ You can audit at the finer granularity all the access to database.

◈ And then of course, there are different benefits of three tier architecture e.g. abstraction from data layer, opportunity for caching, different teams can work on different layer parallelly, scaling different layers separately, ease of maintenance etc.

Moreover, the biggest benefit is that you only have one service which needs to access keys. As you rotate keys only one service will need to refresh them, rather than hundreds or thousands of clients.

As shown above, you can have an application, which is authenticated by AAD and calls the REST web service. The application shown above can be tens of thousands of instance of a web site. REST API will get the caller identity in the call. Here is an opportunity for this web service to apply some business rules for the call. This web service can consult its custom rules to see which user has permission for which Azure Cosmos DB collection. Once the business rule layer validates the call, Data Service can get the keys from Key-Vault to access the data from Azure Cosmos DB.

If you are following Azure Cosmos DB closely, you may wonder about resource tokens. You should prefer the above architecture rather than implementing a service which can dispense the resource tokens and then a client can use those resource tokens to access Azure Cosmos DB directly.

I will assert having a data service is a better design than letting multiple client apps directly access the database. Resource token is a right solution for any small application, but it has some of its own limitations. Resource token does not work for all the API’s. It works only for SQL API. You still have to create a web service, which will dispense the token as per your business rules, so why don’t that service go one-step ahead and dispense the data itself.

Yes, you are adding a hop for the application, and it may be adding few milliseconds, but this is a better alternative to Resource token.

Azure Cosmos DB keeps all your data encrypted at rest and on wire.  With virtual network, IP Filtering, Key-Vault you can built a very secure application with Azure Cosmos DB.

Continue reading

VNet service endpoints for Azure database services for MySQL and PostgreSQL in preview

We recently made Azure database services for MySQL and PostgreSQL generally available. These services offer the community versions of MySQL and PostgreSQL with built-in high availability, a 99.99% availability SLA, elastic scaling for performance, and industry leading security and compliance on Azure. Since general availability, we have continued to bring new features and capabilities like increased storage and availability across more regions worldwide.

We are excited to announce the public preview of Virtual Network (VNet) service endpoints for Azure Database for MySQL and PostgreSQL in all regions where the service is available. Visit region expansion for MySQL and PostgreSQL for service availability. VNet service endpoints enable you to isolate connectivity to your logical server from only a given subnet or set of subnets within your virtual network. The traffic to Azure Database for MySQL and/or PostgreSQL from your VNet always stays within the Azure backbone network. Preference for this direct route is over any specific ones that route Internet traffic through virtual appliances or on-premises.

There is no additional billing for virtual network access through service endpoints. The current pricing model for Azure Database for MySQL and PostgreSQL applies as is.

Using firewall rules and VNet service endpoints together

Turning on VNet service endpoints does not override firewall rules that you have provisioned on your Azure Database for MySQL or PostgreSQL. Both continue to be applicable.

VNet service endpoints don’t extend to on-premises. To allow access from on-premises, firewall rules can be used to limit connectivity only to your public (NAT) IPs.

To enable VNet protection, visit these articles for Azure Database for MySQL and PostgreSQL.

Turning on service endpoints for servers with pre-existing firewall rules

When you connect to your server with service endpoints turned on, the source IP of database connections switches to the private IP space of your VNet. Configuration is via the “Microsoft.Sql” shared service tag for all Azure Databases including Azure Database for MySQL, PostgreSQL, Azure SQL Database Managed Instance, and Azure SQL Data Warehouse. If at present, your server or database firewall rules allow specific Azure public IPs, then the connectivity breaks until you allow the given VNet/subnet by specifying it in the VNet firewall rules. To ensure connectivity, you can preemptively specify VNet firewall rules before turning on service endpoints by using IgnoreMissingServiceEndpoint flag.

Support for App Service Environment

As part of the public preview, we support service endpoints for App Service Environment (ASE) subnets deployed into your VNets.

Continue reading

Application Security Groups now generally available in all Azure regions

We are pleased to announce the general availability of Application Security Groups (ASG) in all Azure regions. This feature provides security micro-segmentation for your virtual networks in Azure.

Network security micro segmentation

ASGs enable you to define fine-grained network security policies based on workloads, centralized on applications, instead of explicit IP addresses. Provides the capability to group VMs with monikers and secure applications by filtering traffic from trusted segments of your network.

Implementing granular security traffic controls improves isolation of workloads and protects them individually. If a breach occurs, this technique limits the potential impact of lateral exploration of your networks from hackers.

Security definition simplified

With ASGs, filtering traffic based on applications patterns is simplified, using the following steps:

◈ Define your application groups, provide a moniker descriptive name that fits your architecture. You can use it for applications, workload types, systems, tiers, environments or any role.

◈ Define a single collection of rules using ASGs and Network Security Groups (NSG), you can apply a single NSG to your entire virtual network on all subnets. A single NSG gives you full visibility on your traffic policies, and a single place for management.

◈ Scale at your own pace. When you deploy VMs, make them members of the appropriate ASGs. If your VM is running multiple workloads, just assign multiple ASGs. Access is granted based on your workloads. No need to worry about security definition again. More importantly, you can implement a zero-trust model, limiting access to the application flows that are explicitly permitted.

Single network security policy

ASGs introduce the ability to deploy multiple applications within the same subnet, and isolate traffic based on ASGs. With ASGs you can reduce the number of NSGs in your subscription. In some cases, you can use a single NSG for multiple subnets of your virtual network. ASGs enable you to centralize your configuration, providing the following benefits in dynamic environments:

◈ Centralized NSG view: All traffic policies in a single place. It’s easy to operate and manage changes. If you need to allow a new port to or from a group of VMs, you can make a change to a single rule.

◈ Centralized logging: In combination with NSG flow logs, a single configuration for logs has multiple advantages for traffic analysis.

◈ Enforce policies: If you need to deny specific traffic, you can add a security rule with high priority and enforce administrative rules.

Filtering east-west traffic

With ASGs, you can isolate multiple workloads and provide additional levels of protection for your virtual network.

In the following illustration, multiple applications are deployed into the same virtual network. Based on the security rules described, workloads are isolated from each other. If a VM from one of the applications is compromised, lateral exploration is limited, minimizing the potential impact of an attacker.

In this example, let’s assume one of the web server VMs from application1 is compromised, the rest of the application will continue to be protected, even access to critical workloads like database servers will still be unreachable. This implementation provides multiple extra layers of security to your network, making this intrusion less harmful and easy to react on such events.

Filtering north-south traffic

In combination with additional features on NSG, you can also isolate your workloads from on premises and azure services in different scenarios.

In the following illustration, a relatively complex environment is configured for multiple workload types within a virtual network. By describing their security rules, applications have the correct set of policies applied on each VM. Similar to the previous example, if one of your branches is compromised, exploration within the virtual network is limited therefore minimizing the potential impact of an intruder.

In this example, let’s assume someone on one of your branches connected using VPN, compromise a workstation and has access to your network. Normally only a subset of your network is required for this branch, by isolating the rest of your network; all other applications will continue to be protected and unreachable. ASGs another layers of security to your entire network.

Another interesting scenario, assuming you have detected a breach on one of your web servers, a good idea would be to isolate the VM for investigation. With ASGs, you can easily assign a special group predefined for quarantine VMs on your first security policy. These VMs lose access providing an additional benefit to help you react and mitigate this treats.

Continue reading

Azure DNS Private Zones now available in public preview

We are pleased to announce the public preview of DNS Private Zones in all Azure Public cloud regions. This capability provides secure and reliable name resolution for your virtual networks in Azure. Private Zones was announced as a managed preview in fall of last year.

No more custom DNS server burden

Private Zones obviates the need to setup and manage custom DNS servers. You can bring DNS zones to your virtual network as you lift-and-shift applications to the Azure cloud, or if you are building Cloud-Native applications. You also have the flexibility to use custom domain names, such as your company’s domain name.

Name resolution across virtual networks and across regions

Private zones provide name resolution both within a virtual network and across virtual networks. You can have private zones not only span across virtual networks in the same region, but also across regions and subscriptions. This feature is available in all Azure Public cloud regions.

Split-horizon support

You can configure zones with a split-horizon view, allowing for a private and a public DNS zone to share the same name. This is a common scenario when you want to validate your workloads in a local test environment, before rolling out in production for broader consumption. To realize this scenario, simply configure the same DNS zone as both a public zone and private zone in Azure DNS. Now for clients in a virtual network attached to the zone, Azure will return the DNS response from the private zone, and for clients on the internet, Azure will return the DNS response from the public zone. Since name resolution is confined to configured virtual networks, you can prevent DNS exfiltration.

Dynamic DNS Registration

We are introducing two concepts to DNS zones with this update; Registration virtual networks and Resolution virtual networks. When you designate a virtual network as a Registration virtual network at the time of creating a private zone or later when you update the zone, Azure will dynamically register DNS A records in the private zone for the virtual machines within this virtual network and will keep track of virtual machine additions or removals within the virtual network to keep your private zone updated. This is without any work on your part.

You can also designate up to 10 virtual networks as Resolution virtual networks when creating or updating a private zone. Forward DNS queries will resolve against the private zone records from any of these virtual networks. There is no dependency or requirement that the virtual networks be peered for DNS resolution to work across virtual networks.

Azure DNS Private Zones also supports Reverse DNS queries for the private IP address space of the Registration virtual network.

Continue reading