Azure cloud business value for retail and consumer goods explained

For brick and mortar retailers, the world has been overturned. Online retailers have been demolishing their market share and icons of commerce are struggling. But what helped online retailers can help the offline. The cloud can also be used by brick and mortar retailers. In fact, the brick and mortar experience, transformed with cloud technology, can be a real advantage in competition with online only.

Reasons for retailers and consumer brands to move to the cloud

Cloud technologies are enabling new capabilities and those new powers are disrupting the business models of traditional retailers and sellers of consumer goods. The cloud is at the heart of digital transformation.

◈ It is changing the way technology is implemented and managed.
◈ It offers the benefit of massive scale, increased business speed, and organizational agility.
◈ It makes possible economic benefits related to variable expense, maintenance and deployment.
◈ It enables seamless consumer experiences between offline and online.
◈ It encourages differentiated experiences that wow customers.

Now you have the key to competing in today’s landscape. For these reasons, it is no longer a question of “if,” but “when” and “how” to move to the cloud for most brands.

Business value of the cloud

Born-in-the-cloud retailers are entering the marketplace by solving long-standing consumer challenges in new and innovative ways. Modern technology capabilities allow them to accelerate benefits to both the consumer and business objectives. These new experiences raise the bar on what’s possible. They elevate consumers’ expectations by delivering relevancy and convenience, often at a fraction of the ecosystem footprint of long-standing retailers.

Each organization’s journey to the cloud will be unique. There will be a variety of reasons and benefits that should be acknowledged. However, here are the four major categories for cloud business value: cost, agility, performance, and new sources of value.

Evolved cost structure and transparency

Innovation doesn’t stop because of an organization’s budgeting cycle. Your internal processes should not impact your speed and agility to deliver improved experiences to your consumers.  If it does, as a leader you should add those processes to your list of things to evolve.

The cloud enables and encourages a continuous planning approach. It allows you to reap the full benefits of the cloud despite the traditional annual budgeting cycles. The dominant conversation related to cost becomes the shift from CapEx to OpEx. This fundamentally changes how organizations budget and pay for technology. Since fixed costs associated with shared infrastructure are distributed, the cloud enables greater visibility into the true cost of individual applications. The shift to variable expense offers the organization the ability to begin executing more quickly. And the organization becomes more agile through a fail-fast approach, especially given the lower barrier to initiatives. This enables you to experiment and deliver new concepts to customers. And for some brands, the ability to continually test and learn before committing to significant investments is extremely valuable. Especially when determining the relevancy of the offer and viability of the concept.

Improved agility, speed and productivity

Developing and deploying via on-premises infrastructures (datacenters) can take weeks to months. The cloud provides greater agility and speed-to-consumer. Development teams can be more productive and can quickly develop services that reach global markets. Azure offers near-instant provisioning, allowing projects to move quickly without the need to over-provision resources. As an added bonus, infrastructure planning costs disappear.

The flexibility of the cloud enables organizations to deploy new approaches more effectively. It lets you deliver value to customers and productivity to the organization. Profits accrue with the adoption of agile software development methodologies, DevOps, CI/CD, and modern SOA and PaaS-based architectures.

Azure cloud, made to order

Azure is designed with the developer in mind. Applications can be built with the language of choice, including Node.js, Java, and .NET. Development tools are available for PC or MAC. Visual Studio and Visual Studio Code are premier environments with built-in features for Azure. For example, mobile app development is accelerated by integrating the development lifecycle with Visual Studio App Center. Features include automated builds, and testing for cross-platform, hybrid, and native apps on iOS and Android.

Most compliant

Azure’s infrastructure has been developed to support global demand. Azure is available in 54 global Azure regions, more than any cloud provider. Azure has 70+ compliance offerings—the largest portfolio in the industry. Azure meets a broad set of international and industry-specific compliance standards, such as General Data Protection Regulation (GDPR), as well as country-specific standards, including Australia IRAP, UK G-Cloud, and Singapore MTCS. 

Security matters

Security is essential to you and your customers. Here is a short list of how Azure offers improvements in reliability and security over on-premises infrastructure.

◈ The Azure Security Center spans on-premises and cloud workloads. From a single dashboard, you can monitor and manage all of your resources.

◈ The Azure Advisor is a free service that gives you the best advice based on the most current data. Azure Active Directory helps you to manage user identities and create intelligence-driven access policies to secure your resources.

◈ Site Recovery gives you some assurance that you can recover from a disaster.

◈ Individual services have security features. For example, see the security features of Azure SQL Database.

Possibilities to wow customers

The cloud enables unlimited computing scale and storage while removing boundaries. This freedom is a distinct advantage over on-premises infrastructure. This opens a wealth of new opportunities. It frees your organization’s creatives. They can imagine, prototype, and deliver new experiences that wow customers, leading to new business model opportunities.

These cloud capabilities, plus the availability of data and digital networks, provide an opportunity for modern technologies such as artificial intelligence, IoT, machine learning, and AR/VR to thrive. These technologies enable you to innovate and experiment. This leads to competitive advantages, many of which are only available in the cloud, and that are cost-prohibitive if implemented on-premises.

This is where it gets exciting for retail and consumer goods brands who are focused on delivering new and/or improved digital experiences. The cloud opens possibilities as new data signals are captured and used to provide insights fuelled with artificial intelligence.

Continue reading

Orchestrating production-grade workloads with Azure Kubernetes Service

In this blog post, I will dig into the top scenarios that Azure customers are building on Azure Kubernetes Service. After that, we will blow out the candles and have some cake.

Lift and shift to containers

Organizations typically want to move to the cloud quickly and it is often not possible to re-write applications to take full advantage of cloud-native features right from the beginning. Containerizing applications makes it much simpler to modernize your apps and move to the cloud in a frictionless manner while adding benefits such as CI/CD automation, auto-scale, security, monitoring, and much more. It also allows for simplified management at the data layer by taking advantage of Azure’s PaaS based databases such as Azure CosmosDB or Azure’s managed PostgreSQL Service.

For example, I worked with a customer in the manufacturing industry who had many legacy Java applications sprawled throughout a high cost datacenter. They were often unable to scale these applications to meet customer demand and updates were cumbersome and unreliable. With Azure Kubernetes Service and containers, they were able to host many of these applications in a single managed service. This led to a much higher reliability and the ability to ship new capabilities much more frequently.

The following diagram visually illustrates a typical lift and shift approach involving AKS.

Microservices based cloud native applications

Microservices bring super powers to many applications and include benefits such as:

◈ Independent deployments

◈ Improved scale and resource utilization per service

◈ Smaller, focused developer teams

◈ Focus code around business capabilities

Containers are the ideal technology to deliver microservices based applications. Kubernetes provides the much-needed orchestration layer to help organizations manage these distributed microservices apps at scale.

What Azure uniquely brings to the table is its native integration with developer tools, and its flexibility to plug into the best tools and services coming out of the Kubernetes ecosystem. It offers the comprehensive, yet simple, end-to-end experience for seamless Kubernetes lifecycle management on Azure. Since microservices are polyglot in nature for language, processes, and tooling, Azure’s focus on developer and ops productivity attracts companies like Siemens, Varian, and Tryg to run microservices at scale with AKS. My customers running AKS also found the below capabilities helpful for development, deployment, and management of their microservices-based applications:

◈ Azure Dev Spaces support to iteratively develop, test, and debug microservices targeting at AKS clusters.

◈ Automating external access to services with HTTP application routing.

◈ Using ACI Connector, a Virtual Kubelet implementation, to allow for fast scaling of services with Azure Container Instances.

◈ Simplifying CI/CD with Azure DevOps Projects and open source tools such as Helm, Draft, and Brigade, all backed by a secure, private Docker repository in Azure Container Registry.

◈ Supporting Service Mesh solutions such as Istio or Linkerd to easily add complex network routing, logging, TLS security, and distributed tracing to distributed apps.

The image below shows how some of the elements called out above fit in the overall scenario.

This blog has a good run down of developing microservices with AKS. If you are looking to get more hands-on and build microservices with AKS.

IoT Edge deployments

IoT solutions such as SmartCity, ConnectedCar, and ConnectedHealth have enabled many diverse applications connecting billions of devices to the cloud. With advances in computing power, these IoT devices are becoming more and more powerful. Of course, IoT application development poses some challenges as well. For instance, crafting, maintaining and updating a robust IoT solution is time-consuming. Such solutions also face a higher degree of difficulty when it comes to maintaining cohesive security in a distributed environment. Device incompatibility with existing infrastructure, and challenges in scaling further compound IoT solution development.

Azure provides a robust set of capabilities to address these IoT challenges. More specifically, Azure IoT Edge was created to help customers run custom business logic and cloud analytics on edge devices so that the focus of the devices can be on business insights instead of data management.

At Azure, we are seeing customers utilize the power of AKS to bring containers and orchestration to help manage this IoT Edge layer. Customers can combine AKS with the IoT Edge connector, a Virtual Kubelet implementation, to help provide:

◈ Consistency between cloud and edge software configuration.

◈ Applying identical deployments across multiple IoT hubs.

With AKS and the IoT Edge connector, the configuration can be defined in a Kubernetes manifest and then simply and reliably deployed to all IoT devices at the edge with a single command. The simplicity of a single manifest to manage all IoT Hubs helps customers deliver and manage IoT applications at scale. For example, consider the challenges involved in deploying and managing devices across different regions. AKS, along with the IoT Hub and the IoT Edge connector, make these deployments simple. The graphic below illustrates this IoT scenario involving AKS and the IoT Edge connector.

Machine learning at scale

Though machine learning is immensely powerful, using it in practice is by no means easy. Machine learning in practice often involves training and hosting models which tend to require the data scientist to reproduce the code to work in different environments and be deployed at different scales. Additionally, once the model is running in production with large scale clusters, lifecycle management becomes increasingly difficult. Configuration and deployment are often left to data scientists which results in their time being consumed by infrastructure setup instead of data science itself.

AKS can help address these challenges faced with training and hosting ML models and the lifecycle management workflows.

◈ For training, AKS can help ensure GPU resources, designed for compute-intensive, high-scale workloads, are available on demand and scaled down when not needed. This becomes critical when a group of data scientists are all working on various projects and require resources on very diverse schedules. This also allows for faster training cycles by enabling strategies such as distributed training and hyperparameter optimization.

◈ For hosting, AKS brings DevOps capabilities to machine learning models. These models can be upgraded more easily using the rolling upgrades capability, and strategies such as blue/green or canary deployments can be easily applied.

◈ Using containers also brings much higher consistency across test, dev, and production environments. Also, self-healing capabilities dramatically improve reliability of the execution.

A possible scenario involving AKS for machine learning models is shown in the image below.

Continue reading

Azure App Service now supports Java SE on Linux

Lately there have been a whole lot of changes to Java and its vibrant communities. Now shared between Oracle for Java SE and Eclipse Foundation for Jakarta EE (formerly Java EE), Java continues to be the leading programming language by developers and enterprises. As a matter of fact, it is now well-positioned to thrive in the cloud considering how modern application development is trending with over 12 million Java developers worldwide, and digital transformation being top of mind for many IT organizations.

With the sheer volume of Java apps in existence and soon to be developed, Java developers will benefit greatly from cloud services that will enable fast and secure application development while saving time and cost. Couple this with a vast geographic region coverage, it is a cloud solution every developer should experience.

Today, Microsoft is pleased to announce that Azure App Service now supports Java SE 8 based applications on Linux, now available in public preview. This and subsequent time released versions will be supported for an extended period, as well as upcoming LTS versions. Java web apps can now be built and deployed on a highly scalable, self-patching web hosting service where bug fixes and security updates will be maintained by Microsoft. Additional performance features include scaling to support millions of users with multiple instances, applications, and regions in a dynamic scaling intelligent configuration.

Java Web Apps benefits

Let Azure App Service do all the heavy lifting commonly associated with enterprise-grade infrastructure so developers can focus on productivity. Take advantage of Microsoft Azure’s battle tested infrastructure platform spanning from global security compliance to DevOps capabilities. Developer productivity benefits do not stop there. Java web apps provide the following benefits:

◈ Fully managed enterprise platform – Log aggregation, email notifications, and Azure portal alerts. Version updates will soon include auto-patching.

◈ Performance monitoring – Integrate with the Application Performance Management (APM) product of your choice to monitor and manage applications.

◈ Global scale with high availability – 99.95% uptime with low latency, auto-scaling, or manual-scaling (up or out), anywhere in Microsoft’s global datacenters.

◈ Security and compliance – App Service is ISO, SOC, PCI, and GDPR compliant.
Authentication and authorization – Built-in authentication with Azure Active Directory, governance with Roll-Based Access Control (RBAC) to manage IP address restrictions.

◈ Build automation and CI/CD Support – Maven, Jenkins, and Visual Studio Team Services support will be available in the general availability release.

There are three ways of deploying Java Web Apps on Azure. You can create it from the Azure portal, use a template, or create and deploy from Maven. In this post, we will cover how to deploy a Spring Boot app using Maven.

Get started with Maven and Spring

To get started, clone your favorite Java Web app or use this sample: bash-3.2$ git clone

Add the Maven plugin for Azure Web Apps to the app project POM file and set server port to 80.

<build>
   <plugins>
      <plugin>
         <groupId>com.microsoft.azure</groupId>
         <artifactId>azure-webapp-maven-plugin</artifactId>
         <version>1.2.0</version>
         <configuration>

            <!-- Web App information -->
            <resourceGroup>${RESOURCE_GROUP}</resourceGroup>
            <appName>${WEBAPP_NAME}</appName>
            <region>${REGION}</region>
            <pricingTier>S1</pricingTier>

            <!-- Java Runtime Stack for Web App on Linux -->
            <linuxRuntime>jre8</linuxRuntime>

            <deploymentType>ftp</deploymentType>
            <!-- Resources to be deployed to your Web App -->
            <resources>
               <resource>
                  <directory>${project.basedir}/target</directory>
                  <targetPath>/</targetPath>
                  <includes>
                     <include>app.jar</include>
                  </includes>
               </resource>
            </resources>
            <appSettings>
               <property>
                  <name>JAVA_OPTS</name>
                  <value>-Djava.security.egd=file:/dev/./urandom</value>
               </property>
            </appSettings>
         </configuration>
     </plugin>
   </plugins>
   <finalName>app</finalName>
</build> ​

Build, package, and deploy using Maven – like you would normally do.

bash-3.2$ mvn package azure-webapp:deploy
[INFO] Scanning for projects...
[INFO]
[INFO] ----------------------------------------------------------------------
[INFO] Building petclinic 2.0.0
[INFO] ----------------------------------------------------------------------
[INFO]
...
...
[INFO] --- azure-webapp-maven-plugin:1.2.0:deploy (default-cli) @ spring-petclinic ---
[INFO] Start deploying to Web App myjavase-07262018aa...
[INFO] Authenticate with Azure CLI 2.0
[INFO] Target Web App doesn't exist. Creating a new one...
[INFO] Creating App Service Plan 'ServicePlan1af9c8f0-3f71-43a8'...
[INFO] Successfully created App Service Plan.
[INFO] Successfully created Web App.
...
...

[INFO] Finished uploading directory: /Users/selvasingh/GitHub/selvasingh/spring-petclinic/target/azure-webapps/myjavase-07262018aa --> /site/wwwroot
[INFO] Successfully uploaded files to FTP server: waws-prod-bay-081.ftp.azurewebsites.windows.net
[INFO] Starting Web App after deploying artifacts...
[INFO] Successfully started Web App.
[INFO] Successfully deployed Web App at https://myjavase-07262018aa.azurewebsites.net
[INFO] ----------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ----------------------------------------------------------------------
[INFO] Total time: 03:06 min
[INFO] Finished at: 2018-07-13T18:03:22-07:00
[INFO] Final Memory: 139M/987M
[INFO] ----------------------------------------------------------------------

Open the site in your favorite browser.

And here it is, your first Java web app on Azure App Service. Nice job!

More friendly Java tools on the way

We understand that development tools are not one size fit all. We are making sure we continue to enrich our tool suite with tools that enhance productivity and efficiency. In the meantime, expect to see support for Gradle and Jenkins next.

Continue reading

New recommendations in Azure Advisor

Azure Advisor is a free service that analyzes your Azure usage and provides recommendations on how you can optimize your Azure resources to reduce costs, boost performance, strengthen security, and improve reliability.

We are excited to announce that we have added several new Azure Advisor recommendations to help you get the most out of your Azure subscriptions.

Buy Reserved Instances to save over pay-as-you-go costs

Azure Reserved Instances (RIs) allow you to reserve virtual machines (VMs) in advance on a one or three-year term and save up to 80 percent versus pay-as-you go rates. RIs are ideal for workloads with predictable, consistent traffic.

Azure Advisor will analyze your last 30 days of VM usage and recommend purchasing RIs when it may provide cost savings. Advisor will show you the regions and VM sizes where you could save money and give you an estimate of your potential savings from purchasing RIs if your usage remains consistent with the previous 30 days.

Create Azure Service Health alerts

Azure Service Health is a free service that provides personalized guidance and support when Azure service issues might affect you. You can create Service Health alerts for any region or service so that you and your teams stay informed via the Azure portal, email, text message, or webhook notification when business-critical resources could be impacted.

Azure Advisor will identify your subscriptions that do not have Service Health alerts configured and recommend that you set up alerts on those subscriptions.

Upgrade to a support plan that includes technical support

Azure technical support plans give you access to Azure experts when you need assistance. Azure offers a range of support options to best fit your needs, whether you’re a developer just starting your cloud journey or a large organization deploying business-critical applications.

Azure Advisor will identify subscriptions with a high amount of monthly Azure spend that are likely running strategic workloads and recommend upgrading your support plan to include technical support.

Configure your Traffic Manager profiles for optimal performance and availability

Azure Traffic Manager allows you to control the distribution of user traffic for service endpoints in different datacenters and optimize for performance and availability. Azure Advisor has added new recommendations to solve common configuration issues with Traffic Manager profiles.

Reduce DNS Time to Live

Time to Live (TTL) settings on your Traffic Manager profile allow you to specify how quickly to switch endpoints if a given endpoint stops responding to queries. Reducing the TTL value means that clients will be routed to functioning endpoints faster.

Azure Advisor will identify Traffic Manager profiles with a longer TTL configured and will recommend configuring the TTL to either 20 seconds or 60 seconds depending on whether the profile is configured for Fast Failover.

Add or move one endpoint to another Azure region

If all endpoints in a Traffic Manager profile configured for proximity routing are in the same region, users from other regions may experience connection delays. Adding or moving an endpoint to another region will improve overall performance and provide better availability if all endpoints in one region fail.

Azure Advisor will identify Traffic Manager profiles configured for proximity routing where all the endpoints are in the same region and recommend that you either add or move an endpoint to another Azure region.

Add an endpoint configured to “All (World)”

If a Traffic Manager profile is configured for geographic routing, then traffic is routed to endpoints based on defined regions. If a region fails, there is no pre-defined failover. Having an endpoint where the Regional Grouping is configured to “All (World)” will avoid traffic being dropped and improve service availability.

Azure Advisor will identify Traffic Manager profiles configured for geographic routing where there is no endpoint configured to have the Regional Grouping as “All (World)” and recommend making that configuration change.

Add at least one more endpoint, preferably in another region

Traffic Manager profiles with more than one endpoint experience higher availability if any given endpoint fails. Placing these endpoints in different regions further improves service reliability.

Azure Advisor will identify Traffic Manager profiles where there is only one endpoint and recommend adding at least one more endpoint in another region.

Continue reading

Foretell and prevent downtime with predictive maintenance

The story of predictive maintenance (PdM) starts back in the 1990s. Technologies began to arrive that sense the world in new ways: ultrasound, infrared, thermal, vibration, to name a few. However, until recently the technology has not been available to make predictive maintenance a reality. But now, with advances in cloud storage, machine learning, edge computing, and the Internet of Things — predictive maintenance looms as the next step for the manufacturing industry.

What is predictive maintenance?

There are three strategies for machine maintenance:

◈ Reactive — the “don’t fix what isn’t broken” approach. This means you extract the maximum possible lifetime from a machine. However, costs balloon with unexpected downtime and collateral damage from failures.

◈ Preventative — service on a fixed schedule based on the regularity of previous failures. You maximize up-time by fixing machines before they fail. The downside is that components may have life left, and there is still a chance that they will fail before the scheduled maintenance.

◈ Predictive — where we use data about previous breakdowns to model when failures are about to occur, and intervene just as sensors detect the same conditions. Until recently this has not been a realistic option, as modeling did not exist, and real-time processing power was too expensive. But Azure solves that problem.

The figure below shows how the three strategies differ.

It is clear that predictive maintenance is superior by far to the other methods. It allows you to maximize uptime while getting the most value out of your machinery. Also, using machine learning, the model can continuously be refined; over time, you will experience fewer failures.

How Azure can help

To use machine learning for a PdM solution, there are three requirements: a machine learning model, data used to train the model, and a data ingestion mechanism to gather the training data. Once you have an ingestion point, you need to collect data from a normally operating machine until the machine fails. You then can then characterize the received data as normal, failing, and failed. This data is used to train the model, which means the model is successively adjusted until it can predict failure with some certainty.

To accurately assess the state of a system which leads to failure, as much data as possible needs to be collected. In other words, start collecting your data now. The more comprehensive the data used to train the model, the more accurate the analysis will be.

Azure options for data ingestion

Azure offers these options for data ingestion: Event Hubs, Azure IoT Hub, and Kafka on HDInsight. (For more information, see Choosing a real-time message ingestion technology in Azure.) Ingesting data from distributed systems is often not a sustainable approach. As more systems talk to each other, the system builds a Ο(n2) complexity. A much better architecture is to have all your systems talking to a central hub. This can be implemented efficiently using the Azure IoT Hub. All your systems can speak to the IoT Hub, and the hub feeds the data into Azure. Loading the data into Azure Data Factory is a great option, as it can move and transform data as part of an automated pipeline.

Once the data has been ingested, it is then used to train your machine learning model. Azure options for doing this include Azure Machine Learning Studio, Azure Databricks, Data Science Virtual Machine, and Azure Batch AI — to name a few. The choice depends on the complexity of your problem, the experience of your team, and the size of the data to be processed.

Once the model has been trained and is ready for use, the results can be presented. This means building a mechanism to predict future failures and generating notifications for action. The workflow looks something like this:

In a working system, the results are presented to the maintenance team in real-time, along with recommendations for action. The team can decide the best course of action.

Overall, Azure can provide your predictive maintenance solution with the following:

◈ Scalability, as storage and processing power, is easily scalable.

◈ Availability and resilience, through the fact that you can provision resources as needed.

◈ Management, through a variety of options including ARM, PowerShell, and management APIs.

◈ Security, via IoT Hub keys, encryption and much more.

◈ Cost-effectiveness, as resources can be provisioned and discarded, as necessary.

Continue reading

Blockchain as a tool for anti-fraud

Healthcare costs are skyrocketing. In 2016, healthcare costs in the US are estimated at nearly 18 percent of the GDP! Healthcare is becoming less affordable worldwide, and a serious chasm is widening between those that can afford healthcare and those that cannot. There are many factors driving the high cost of healthcare, one of them is fraud. In healthcare, there are several types of fraud including prescription fraud, medical identity fraud, financial fraud, and occupational fraud. The National Health Care Anti-Fraud Association estimates conservatively that health care fraud costs the US about $68 billion annually, which is about three percent of the US total $2.26 trillion in overall healthcare spending. There are two root vulnerabilities in healthcare organizations: insufficient protection of data integrity, and a lack of transparency.

Insufficient protection of data integrity enables fraudulent modification of records

Cybersecurity involves safeguarding the confidentiality, availability, and integrity of data. Often cybersecurity is mistakenly equated with protecting just the confidentiality of data to prevent unauthorized access. However, equally important is protecting the availability of data. That is, you must secure timely and reliable access to data, as well as the integrity of the data. You must ensure records are accurate, complete, and up-to-date. Protecting the integrity of information is particularly important for anti-fraud. Insufficient protection paves the way for the alteration or deletion of records for personal gain.

Insufficient transparency enables fraud to proceed undetected

Many types of fraud involve the creation of new fraudulent records. This can occur within a single organization and many times even within just a single system or silo within a healthcare organization. When this happens with limited access and visibility, then it is easier for fraudsters to conceal their activity. They can conduct fraudulent activities for longer before detection. The longer fraud proceeds undetected, the larger the business impact and cost.

Blockchain immutability protects the integrity of records

Chained hashcodes on blockchains make it practically impossible to alter or delete records on the blockchain. Furthermore, records on blockchains can also include pointers to off-chain data. Together with hashcodes, the pointers can be used to verify the integrity of such off-chain data. And that enables blockchains also to protect the integrity of off-chain data. Protecting the integrity of both on and off-chain records essentially blocks the fraudulent deletion or alteration of records for personal gain.

Blockchain transparency enables improved detection of fraud

Blockchains provide near real-time transparency of electronic records appended to the blockchain. These new records are visible across the consortium of healthcare organizations connected to the blockchain. This improves the ability of such consortiums to detect fraudulent records. The mere presence of such transparency and the prospect of detection alone will significantly deter fraud. And once such fraud is discovered, organizations can stop, remediate losses, and minimize business impacts and costs.

Blockchain advancing artificial intelligence for anti-fraud

Artificial Intelligence (AI) and Machine Learning (ML) have significant potential as tools for anti-fraud. They enable anti-fraud teams to process vast quantities of data in near real-time and detect fraudulent patterns that can then be investigated further by a team of anti-fraud experts. As new patterns of fraud are identified, new AI/ML models can be built and integrated into future anti-fraud inference suites, enabling a high degree of automation in anti-fraud. However, AI/ML are very data hungry. When they are powered by data from a single organization and often only a single silo of data within that organization, this stunts the quality of the models that can be built. Further, the quality of inference that can be done using such stunted models often results in a high inference error rate making many anti-fraud use cases infeasible. Blockchains enable collaboration on AI/ML for anti-fraud across a consortium of healthcare organizations. This allows collaboration on training data, the creation of shared models, inference results, and the validation of those results. As discussed in Accelerating AI and ML in Healthcare Using Blockchain.

Getting started with Blockchain

If you would like to get started with prototyping a blockchain for your anti-fraud initiative. The workbench is a powerful platform for rapid prototyping of your blockchain. Once created, you can also deploy it as an Ethereum Blockchain on the Microsoft Azure cloud. The Azure Blockchain Workbench will also be adding support for both the Hyperledger Fabric and R3 Corda Blockchain platforms going forward. The Azure Blockchain Workbench enables you to accelerate your prototyping, technical POCs and pilots, enabling you to focus on anti-fraud results, and business value rather than blockchain technologies and deployment complexities.

Getting started with AI

Bootstrap your AI anti-fraud initiative with the Azure Security and Compliance Blueprint - HIPAA/HITRUST Health Data and AI. Rather than starting from scratch, you can accelerate your AI initiative by downloading, configuring, running, and customizing this AI blueprint for your anti-fraud use case. This blueprint also provides a wealth of information on how to protect the privacy, security, and compliance of your cloud-based AI solution.

Collaboration

I post regularly about new developments in healthcare, anti-fraud, AI, blockchain, and cloud computing on social media.

Continue reading

Location and Maps in Azure IoT Central powered by Azure Maps

Azure IoT Central brings the simplicity of SaaS for IoT with built-in support for IoT best practices and world class security and scalability with no cloud expertise required. We have been constantly adding features and true to the promise of SaaS applications, you can just start using new features right away to build production-grade applications without worrying about managing infrastructure.

This blog post is part of a series of blog posts you will start seeing for new features in Azure IoT Central in the upcoming weeks.

Azure IoT Central now leverages Azure Maps. A portfolio of geospatial functionalities natively integrated into Azure to enable users with fresh mapping data necessary to provide geographic context to their location aware IoT applications. We received several interests from public preview customers to leverage geospatial services for various use cases ranging from simply localizing their devices, validating location information, spatially referencing device locations on a map, to geofencing use cases around their devices. As any other property in Azure IoT Central, location metadata can be persisted on the cloud and updated either by the device itself (device properties) or the user (application properties). By integrating with Azure Maps, user can now give geographic context to their location property and map any latitude and longitude of a street address, or simply latitude and longitude coordinates.

For this first release, Azure IoT Central customers can configure a location property in their device template and pass addresses or coordinates as values. This feature uses the Azure Maps Search Service, to find addresses and places from around the world. Azure Maps supports address level geocoding in 38 regions, cascading to house numbers, street-level, and city level geocoding for other regions of the world.

After configuring a location property, in the Device dashboard, an Azure IoT Central customer will also be able to add map tiles. For this, IoT Central uses the Azure Maps JavaScript Control Services to allow layering the configured location property atop of Azure’s Maps right in the Device Dashboards. User can interact with the map tiles.

Similarly, Azure IoT Central users can add a location map tile in the Device Sets dashboard which will display the configured location for all the devices in the set on a map.

In addition, if you are a device developer get started with the MXChip IoT DevKit device using the Azure IoT Central Sample DevKits which will contain an example of device location property and maps, all powered by Azure Maps!

Continue reading

Spoken Language Identification in Video Indexer

We are excited to share that Video Indexer has a new capability, Spoken Language Identification (LID)!

A common ask from our customers has been to enable indexing of videos or batches of videos, without manually providing their language. This is especially important for batch uploads. To support this, we have introduced automatic spoken language identification to Video Indexer. The identified language is used to invoke the appropriate speech-to-text model.

LID is based on state of the art Deep Learning applied on the audio. LID currently supports eight languages including English, Chinese, French, German, Italian, Japanese, Spanish, and Russian. It works with high accuracy for high-to-mid quality recordings. We are working on adding more languages to the list, so stay tuned.

Let’s learn more about LID in Video Indexer.

Using LID in Video Indexer

To use the LID capability, you have two options. If you use the portal, you can now select Auto detect in the language selection combo box when uploading a video.

If you use the API to upload a video, use auto as the language parameter value.

Attribute sourceLanguage in the video index JSON under root/videos/insights is assigned the detected language, and attribute sourceLanguageConfidence:

Note to the user

◈ The model behind LID works best with clear recordings: broadcast materials and enterprise materials such as podcasts, lectures, tutorials, etc.

◈ The model may be confused by: noisy recordings, low-quality recordings, highly variant acoustics, and heavy accents.

◈ When the model cannot yield a result with high confidence, VI will fall back to English.

Behind the curtains

Many cognitive tasks, including spoken language identification, are easy for humans but still very challenging for computers. One way to approach this type of tasks, is to mimic the human brain. The initial idea of an artificial neural network was proposed more than 70 years ago. The state of the art in the field is called Deep Learning and is being successfully used for different tasks in Speech and Language Understanding, Computer Vision and even outperforming humans in some tasks, such as diagnosis of skin cancer.

In VI, we harness the power of Deep Learning for Spoken Language Identification. We train the network by presenting it with a huge number of speech examples, coming from different speakers and having diverse acoustic conditions.

The figure below shows how we represent speech to the network. This representation turns voice into an image called a spectrogram. A spectrogram gives us a sense of how complex acoustics are, as 30 seconds of speech can easily require 300,000 pixels!

Legend: Phonetic visualization of a speech smaple. Top: Waveform representation of the recorded audio. Bottom: Spectrogram representation.

The network learns from such huge amounts of data by looking for patterns in the spectrograms, which differentiate between languages. A good pattern is a pattern that is typical to one language. Deep learning starts with a random guess on what these patterns are. Then, using the examples from each language, improves the guess in the right direction.

Continue reading

Announcing public preview of Azure Virtual WAN and Azure Firewall

Networking trends such as SDWAN (Software-Defined Wide Area Network) can improve performance by using path selection polices at the branch offices to send Internet-bound traffic directly to the cloud eliminating the backhaul to select breakout points. This traffic can quickly reach Microsoft’s global backbone network with intelligent routing to provide the best network experience. However, having all branches directly accessing the Internet introduces new challenges such as managing branch connectivity and uniformly enforcing network and security polices at scale. Further complicating network policy management across all the branch offices is the trend of more employees working remotely with ever stricter security, privacy, and compliance requirements polices that vary by country/region.

Network security plays an important role in protecting users, data and applications. Cloud developers and IT teams struggle to stay ahead of security attacks. Cloud native network security solutions better fit the modern dev ops model of building and deploying applications while taking advantage of the economic and scale benefits of the cloud. Customers need turnkey solutions that are easy to deploy, use, and manage that offer high availability and automatically scale.

To help customers with these massive modernization efforts, we are announcing Azure Virtual WAN to simplify large-scale branch connectivity, and Azure Firewall to enforce your network security polices while taking advantage of the scale and simplicity provided by the cloud.

Azure Virtual WAN

The new Azure Virtual WAN service provides optimized, automated and global scale branch connectivity. Virtual WAN brings the ability to seamlessly connect your branches to Azure with SDWAN & VPN devices (i.e. Customer Premises Equipment or CPE) with built in ease of use and automated connectivity and configuration management.

Figure 1: Connect SDWAN and VPN devices to Hubs that comprise an Azure Virtual WAN

Virtual WAN provides a better networking experience by taking advantage of Microsoft’s global network. Traffic from your branches enters Microsoft’s network at the Microsoft edge site closest to a given branch office. We have over 130 edge sites or Points of Presence (PoPs). Once your traffic is in the Microsoft global network, it terminates in a virtual hub. An Azure Virtual WAN is composed of multiple virtual hubs. You can create your hubs in different Azure regions. Azure has more global regions than any other public cloud provider bringing your virtual hubs close to your branches around the world.

Here is a simple example with a virtual hub in West Europe (Netherlands) and another in North Europe (Ireland). These two hubs are part of a customer’s Azure Virtual WAN. Branch offices connect to the closest virtual hub for the very best performance.

We are launching Azure Virtual WAN Preview with Citrix and Riverbed providing a fully automated branch connectivity experience. Our continued commitment to customers is to create more options with a new and fast-growing SDWAN and VPN partner ecosystem.  Solutions from additional partners such as Checkpoint, Nokia Nuage, Palo Alto and Silverpeak will be available in the coming months. I encourage you to join the preview and provide feedback on both service functionality, performance as well as ecosystem and partners.

We have been working closely with customers dealing with the challenges of branch connectivity at a global scale.

Public preview capabilities

◈ Virtual WAN and virtual hubs: You can create a virtual WAN and then deploy virtual hubs in any Azure public region. This allows your hubs to be close to your branch offices. The hubs are where network traffic initially terminates before heading to another branch office or an Azure Virtual Network (VNet).

◈ Connectivity automation: It is difficult to manually establish and manage a large number of VPN tunnels. Azure Virtual WAN brings together your preferred CPE be it SD-WAN controller or VPN device to automate the branch provisioning, configuration management and connectivity setup enabling you to easily deploy and manage your Virtual WAN. 

◈ Automated VNet configuration: The automated VNet configuration allows you to easily connect your VNet to your hub so users in a branch office can access their Azure resources. 

◈ Troubleshooting and monitoring: The platform monitors your on-premises connections providing a unified experience to manage your Virtual WAN along with your Azure resources.

Azure Firewall

The new Azure Firewall service offers fully stateful native firewall capabilities for Virtual Network resources, with built-in high availability and the ability to scale automatically.  Customers can create and enforce connectivity policies using application and network level filtering rules. Connectivity policies can be enforced across multiple subscriptions and virtual networks. The Azure Firewall service is fully integrated with the Azure platform, portal UI and services.

Public preview capabilities

◈ Outbound FQDN filtering: Keep data within your infrastructure and prevent outbound Internet traffic and data exfiltration by limiting outbound HTTP/S traffic to a customer specified list of Fully Qualified Domain Names (FQDN).

◈ Network traffic filtering rules: Gain visibility and increase control across multiple subscriptions by centrally creating, enforcing and managing your stateful filtering rules by source and destination address, port and protocol.

◈ Outbound SNAT support: Enable outside communication from other security devices and appliances using Source Network Address Translation (SNAT). SNAT support provides address translation between your VNet and Public IP, while easily integrating with existing security perimeter and sharing of policies.

◈ Azure Monitor logging: All events are integrated with Azure Monitor, giving you a single shared interface for your logging and analytics needs. The integration secures logging of all blocked/accepted incidents and further allows you to both archive logs to an Azure storage account, stream events to your Event Hub, or send them to Log Analytics for additional insights.

Azure Firewall – A perfect fit with your existing security

Azure Firewall has been built to enhance and strengthen your current Azure security posture, seamlessly complimenting existing  Azure security services.

◈ Network Security Group (NSG) and Azure Firewall are complementary, and together provide   better defense and in-depth network security. NSGs provides distributed network layer traffic filtering to limit traffic to resources within virtual networks. Azure Firewall is a fully stateful centralized network firewall as-a-service, providing network and application level protection across virtual networks.

◈ Application Gateway WAF provides centralized inbound protection for web applications (L7). Azure Firewall provides outbound network level protection(L3-L4) for all ports and protocols and application level protection (L7) for outbound HTTP/S.

◈ Azure DDoS Protection leverages the scale and elasticity of Microsoft’s global network to bring massive DDoS mitigation capacity in every Azure region. Microsoft’s DDoS Protection service protects your Azure applications by scrubbing traffic at the Azure network edge before it can impact your service's availability.

◈ Service Endpoints: For secure access to PaaS services, we recommend Service Endpoints that extend your virtual network private address space and the identity of your VNet to the Azure service. Azure Firewall customers can choose to enable service endpoints in the Azure Firewall subnet and disable it on the connected spoke VNETs therefore benefitting from both features – service endpoint security and central logging for all traffic.

◈ Network Virtual Appliances: Customers can have a mix of 3rd party NVAs and Azure Firewalls. We are working with our partners on multiple better together scenarios.

With the addition of Virtual WAN and Firewall to our broad portfolio of network services, we are again expanding what is possible with Azure. They both provide a strong testament to our goal of integrating broadly with the platform and your infrastructure, while at the same time being simple and easy to deploy and use.

Continue reading

Securing the connection between Power BI and Azure SQL Database

How can you connect to Azure SQL Database from the Power BI service in a secure fashion? The easiest way to limit access to the database is to select the “allow access to Azure Services” option (Figure1). This can be found in the database server options in the Azure portal. This allows Power BI to access your database. However, it also makes the database visible to any component deployed within Azure, such as a virtual machine. For many organizations this is not sufficient for their security and compliance requirements.

Figure 1: Setting the database access in the Azure Portal.

  The following is a list of suggestions that one may want to consider achieving the organizations security goals:

To start, use VNet service endpoints to further secure access. This feature was introduced at the start of 2018. This is easy to configure. In the Azure portal either create a new virtual network or edit an existing VNet and enable service endpoints for SQL in the VNet (Figure2).

Figure 2: Creating a service endpoint in the virtual network.

Once completed, the next task is to set up a virtual network rule on the database server. This allows us to restrict access to all the SQL databases on that database server to just a subnet within the virtual network. This might be a little too restrictive, so additionally you may also add specific ip addresses that can also have access. The example below (Figure3) illustrates the scenario where a vnet rule called newVnetRule1 restricts access to just objects within the subnet and in addition external access is granted to a machine using the ip address 80.90.100.110. The latter is useful if you also need to allow access from on-premise machines connecting to the database with Power BI desktop. You can simply restrict external access to your companies ip address range.

Figure 3: Adding a virtual network rule and a client ip to the database server.

By restricting access to the database server, we have also prevented the Power BI service from connecting to the database. The solution is to install the on premise data gateway on a virtual machine that resides within the subnet. There are two steps to this, firstly install and configure the gateway on the VM within the subnet. Afterwards use the Power BI portal to configure the gateway so that it is aware on the database you wish to connect to, and the security you want to apply. There is a good write-up of this process. The gateway can either communicate via TCP or HTTPS. The former is more efficient but will require ports 443 (default), 5671, 5672, 9350 thru 9354 to be opened for outgoing traffic whereas HTTPS will only require port 80. To use TCP, ensure that in the gateway configuration screens under networking that the Azure service bus connectivity mode is correctly selected (Figure 4). Note: the gateway does not require any inbound ports to be opened.

Figure 4: Setting the on-premise gateway to connect over TCP.

It is important to note that for the gateway connector we need to use at present is the SQL Server connector. This provides support for both basic and windows authentication. That means that not only can the gateway connect to the backend SQL Server database with a single username and password, but you can also use Windows authentication to pass through the current users’ credentials when issuing queries directly to the database. This provides a solution where the Power BI user’s access to data can be restricted to a data subset by the SQL Server DBA. We are using the same connector to access the Azure SQL Database. The only authentication method common to both databases is database authentication. Therefore, we are restricted to basic authentication when configuring the gateway and therefore user credentials cannot be passed onto the database via the gateway. This may change over time but at the time of writing this article this is the current restriction.

To avoid having a single point of failure in accessing the database through the gateway it is also possible to install multiple gateways in a cluster to provide resiliency. Simply create a second virtual machine and install a second gateway. During the installation you will be able to choose an option to cluster the gateways.

The final step is to go back and make the following changes to the virtual network: create a network security group that can be applied to the subnet. That security group will restrict access to resources on just the incoming ports that you wish to allow. In the above scenario with only SQL databases in the network we would restrict the ports to just the IP ports listed above. If the virtual network is being shared by other resources then they may require additional ports to be opened. A final point to be aware of is that we have deliberately prevented access to the database from other Azure services. As a consequence we will restrict the use of certain Azure SQL Database features.

Continue reading