Digital transformation with legacy systems simplified

Intelligent insurance means improving operations, enabling revenue growth, and creating engaging experiences—which is the result of digital transformation. The cloud has arrived with an array of technical capabilities that can equip an existing business to move into the future. However insurance carriers face a harder road to transform business processes and IT infrastructures. Traditional policy and claim management solutions lack both cloud-era agility, and the modularity required to react quickly to market forces. And legacy systems cannot be decommissioned unless new systems are fully operational and tested, meaning some overlap between old and new.

The Azure platform offers a wealth of services for partners to enhance, extend, and build industry solutions. Here we describe how one Microsoft partner uses Azure to solve a unique problem.

The need for efficient automation

The prevailing approach to upgrading enterprise software is to engage in large scale IT projects that may take years and significant cost to execute. Delaying may only increase the costs, especially with the burden of continuing (and increasing) compliance. But more importantly, delay results in a significant opportunity cost. Due to competition, insurers are under pressure to pursue lower costs overall, and especially in claim handling. New insurance technology also forces the need for new distribution models and to automate internal workflows and supply chains.

A platform built for transformation

The name of Codafication’s solution is Unity (not to be confused with the Unity game engine platform). Codafication calls Unity an ecosystem Platform-as-a-Service (ePaaS). It enables insurance carriers to accelerate their digital transformation through secure, bi-directional data integration with core and legacy systems. At the same time, the platform enables Codafication’s subscribers to use new cloud-native apps and services. The increase in connectivity means customers, staff and supply chains can integrate more easily and with greater efficiencies.

Unity seeks to address the changing expectations of insured customers without disruption to core policy and claim management functions within the enterprise. Codafication stresses a modular approach to implementing Unity. Their website provides a catalog of components such as project management, supply chain and resource planning, and financial control (and more).

In this graphic, potential inputs for the system include a wide variety of processes, from legacy core systems (expected) to robotic processes (a surprise). The output is equally versatile—dashboards and portals along with data lake and IoT workflow apps.

Insurers can take an iterative and modular approach to solving high value challenges rapidly. Unity provides all the tools required to accelerate digital transformation. Other noteworthy features include:

◈ Custom extensions: use any programming language supported by Docker, in combination with Unity SDKs, to build custom frontend and backend solutions.

◈ Off-the-shelf apps: plug in applications and services (from Codafication) designed for the insurance industry.

◈ Scalability: cloud-native technology, underpinned by Kubernetes, can be hosted in the cloud or in a multi-cloud scenario, with a mix of Docker, serverless and on-premises options.

◈ GraphQL API: leverage the power of a graph database to unlock data silos and find relationships between data stores from legacy systems. Integrate with cloud vendors, AI services and best-in-breed services through a single, secure, scalable and dynamic API.

◈ Integrative technologies: create powerful custom IoT workflows with logic hooks, web hooks and real-time data subscriptions.

Benefits

◈ Through Unity, organizations can interconnect everything and relate data on the fly. Developers can leverage legacy core systems, middleware, and robotics using a microservice architecture driven by a powerful service mesh and extensible framework.

◈ Teams can leverage this infrastructure to deliver (in parallel) solutions into the platform and into the hands of their users. Insurance carriers will find new use cases (like data science uses, and AI) and develop apps rapidly, to deliver projects faster, for less cost and less risk.

◈ Projects can be secured and reused across the infrastructure. This accelerates digital transformation projects without disrupting existing architecture and is the primary step to implementing modern cloud native technologies, such as AI and IoT.

◈ The ‘modernize now, decommission later’ approach to core legacy systems lets an insurer compete and remain relevant against competitors while providing a longer runway for decommissioning aging legacy systems.

Continue reading

Best practices for queries used in log alert rules

Queries can start with a table name like SecurityEvent and Perf, or with “search” and “union” operators that can provide a multi-table/multi-column search experience. These operators are useful during data exploration and for searching terms in the entire data model. However, these operators are not efficient for productization in alerts. Log alert rule queries in Log Analytics and Application Insights should always start with a table to define a clear scope for the query execution. It improves both query performance and the relevance of the results.

Note that using cross-resource queries in log alert rules is not considered inefficient although “union” operator is used. The “union” in cross-resource queries is scoped to specific resources and tables as shown in this example, while the query scope for “union *” is the entire data model.

Union

app('Contoso-app1').requests,

app('Contoso-app2').requests,

workspace('Contoso-workspace1').Perf

After data exploration and query authoring, you may want to create a log alert using this query. These examples show how you can modify your queries and avoid “search” and “union *” operators.

Example 1

You want to create a log alert on the following query.

search ObjectName == 'Memory' and (CounterName == '% Committed Bytes In Use' or CounterName == '% Used Memory') and TimeGenerated > ago(5m)

| summarize Avg_Memory_Usage =avg(CounterValue) by Computer

| where Avg_Memory_Usage between(90 .. 95)

| count

To author a valid alert query without the use of “search” operator, follow these steps:

1. Identify the table that the properties are hosted in.

search ObjectName == 'Memory' and (CounterName == '% Committed Bytes In Use' or CounterName == '% Used Memory')

| summarize by $table

The result indicates that these properties belong to the Perf table.

2. Since the properties used in the query are from the Perf table, the query should start with it and scope the query execution to that table.

Perf

| where ObjectName == 'Memory' and (CounterName == '% Committed Bytes In Use' or CounterName == '% Used Memory') and TimeGenerated > ago(5m)

| summarize Avg_Memory_Usage=avg(CounterValue) by Computer

| where Avg_Memory_Usage between(90 .. 95)

| count

Example 2

You want to create a log alert on the following query.

search (ObjectName == 'Processor' and CounterName == '% Idle Time' and InstanceName == '_Total')

| where Computer !in ((union * | where CounterName == '% Processor Utility' | summarize by Computer)) | summarize Avg_Idle_Time = avg(CounterValue) by Computer, CounterPath | where Avg_Idle_Time < 5 | count

To modify the query, follow these steps:

1. Since the query makes use of both “search” and “union *” operators, you need to identify the tables hosting the properties in two stages.

search (ObjectName == 'Processor' and CounterName == '% Idle Time' and InstanceName == '_Total')

| summarize by $table

The properties of the first part of the query belong to the Perf table.

Note, the “withsource = table” command adds a column that designates the table name that hosts the property.

union withsource = table * | where CounterName == '% Processor Utility'

| summarize by table

The property in the second part of the query also belongs to the Perf table.

2. Since the properties used in the query are from the Perf table, both outer and inner queries shoud start with it and scope the query execution to this table.

Perf

| where ObjectName == 'Processor' and CounterName == '% Idle Time' and InstanceName == '_Total'

| where Computer !in ((Perf | where CounterName == '% Processor Utility' | summarize by Computer))

| summarize Avg_Idle_Time = avg(CounterValue) by Computer, CounterPath

| where Avg_Idle_Time < 5

| count

Continue reading

Deploying WordPress application using Visual Studio Team Services and Azure – Part One

This post is the first part of two blog posts, describing how to setup a CI/CD pipeline using Visual Studio Team Services (VSTS) for deploying a dockerized custom WordPress website working with Azure WebApp for Containers and Azure Database for MySQL.

The Motivation

The main motivation for building a WordPress CI/CD pipeline is the fact that WordPress is limited in supporting dynamic configuration to allow easy modification between different environments. Some values are hardcoded in the WordPress MySQL database. This limitation causing a time consuming task which limits our ability to deploy fast and more frequently.

The Idea

We will have four environments: local, dev, test and production. The local environment is for the developer that will run the docker images locally, commit the required changes, and will push the code to the master branch once they completed their work. The push action will initiate a CI process, which will build and push a new docker image to our Azure Container Registry. The base image of this docker image will be the WordPress image from the docker hub. As part of the dockerfile, a copy action will be executed for copying the new content into the new docker image.

After the CI process completion a CD process will start automatically, using Azure Database for MySQL as the WordPress DB each environment will have a separate database. For updating the hardcoded values DB, we will perform a DB export from a previous environment into a SQL script file. Execute find & replace will restore the new SQL file into the next environment DB. Also part of this process we will use Azure Application Insights WordPress plugin for logging and monitoring purposes.

Prerequisites

• Create an Azure account with the following services:
• One instance of Azure Container Registry.
• One instance of Azure Database for MySQL with 5 empty DBs.
• Three instances of Azure WebApp for Containers of each environment: dev, test, production, with one slot-staging.
• Four instances of Application Insights for each environment: local, dev, test, production
• Open a VSTS account with docker Integration extension installed from Visual Studio Marketplace. If you don’t have a Visual Studio Team services account yet, open one now.

It’s possible to create all the above Azure resources using ARM deployment task in VSTS.

Code structure

Create a new VSTS project and upload the code to master branch of this project.

The code repository structure looks like this:

◈ Html folder – sample WordPress files.

◈ Db folder – sample WordPress script DB file that we need to restore into Azure Database for MySQL (referred in this blog as WordPress5000).

◈ Application-insights folder – contains Application Insights plugin folder.

◈ Dockerfile – for building the docker image.

We can run the following docker command on our local machine to run the sample WordPress locally. It will connect to MySQL DB on Azure, we might need to add our IP to the firewall rule of this instance.

docker run -e DB_ENV_HOST=[your mysql db url]:[your mysql port number] -e DB_ENV_USER=[your mysql db user name] -e DB_ENV_PASSWORD=[your mysql db password] -e DB_ENV_NAME=[your mysql database name] -p 5000:80 -d [your docker image name]

VSTS – Build Phase

Now, we are going to create a new build definition. Select the relevant source repository and choose empty template as your baseline for build process. Choose Hosted Linux Preview as agent queue:

Add two new docker tasks.

The first docker task is Build an image with the following values:

◈ Container Registry Type – Azure Container Registry.

◈ Azure subscription – select the relevant Azure subscription.

◈ Azure Container Registry – select the relevant Azure Container Registry.

◈ Action – Build an image.

◈ Docker File – select the dockerfile from the repository.

◈ Check the Use Default Build Context option.

◈ Image Name – [image name, all letters should be lowercase]: $(Build.BuildId).

◈ Check the Qualify Image Name option.

The second docker task is Push an image choose the following:

◈ Container Registry Type – Azure Container Registry.

◈ Azure subscription – select the relevant Azure subscription.

◈ Azure Container Registry – select the relevant Azure Container Registry.

◈ Action – Push an image.

◈ Image Name – same name as in first task.

◈ Check the Qualify Image Name option.

Under Triggers tab - enable continuous integration:

Now, we need to verify our build.

We need to make a small change in one of the project files and push the new version into the master branch, a new build process should start. That’s it for now, in this post we saw how easy is to create a CI process using VSTS and docker integration for creating a new dockerized custom WordPress image and push it into Azure Container Registry. Stay tuned for part 2 where we will continue with our journey for creating a complete CI/CD pipeline.

Continue reading

Enhancements in Application Insights Profiler and Snapshot Debugger

We are pleased to announce a series of improvements on Application Insights Profiler and Snapshot Debugger. Profiler identifies the line of code that slowed down the web app performance under load. Snapshot Debugger captures runtime exception call stack and local variables to identify the issue in code. To ensure users can easily and conveniently use the tools, we delivered the following new features for Profiler and Snapshot Debugger:

Application Insights enablement with Profiler and Snapshot Debugger

With the newly enhanced Application Insights enablement experience, Profiler and Snapshot Debugger are default options to be turned on with Application Insights.

◈ Enabling Snapshot Debugger without redeploy your web app: For ASP.NET core web app, snapshot debugger is a simple, default option when enabling App Insights. It used to require modifying the project to install NuGet and add exception tracking code. Now it’s done via an ASP.NET core hosting light up through an App Setting, no redeploy will be required. ASP.NET support will be available very soon.

◈ Enabling Profiler with Application Insights in one step: Enabling Profiler used to be done in a separate Profiler Configuration pane, which requires extra steps. This is no longer needed.

Profiler

◈ On-demand profiler: Triggering a profiler run session on your web app anytime as needed. Before, Profiler would run randomly 5% of the time, which could miss capturing critical traces. With the new on-demand profiler feature, this problem is solved as users can capture traces anytime as needed.

◈ Profiler for ASP.NET core on Linux: Profiler now works on App Services Linux ASP.NET core 2.0 docker images. More platforms will be supported in the future.

Snapshot Debugger

◈ Snapshot healthy check: Smartly diagnose why web app runtime exceptions do not have associated snapshot. Easily and quickly troubleshooting snapshot debugger with more insights and visibility.

Enable Profiler and Snapshot Debugger is now easier than ever

We enhanced the App Insights enablement experience for App Services. Suppose you have deployed a Web Application to an App Services resource. Later, you notice your web app is being slow or throwing exceptions. You would want to enable App Insights on your Web App to monitor and diagnose what’s going on. Of course, you don’t want to redeploy the web app just to enable monitoring service.

With the new enablement experience, you can easily find the entry point to enable App Insights under Settings | Application Insights. The added section Code level diagnostics is on by default to enable Profiler and Snapshot Debugger for diagnosing slow app performance and runtime exceptions.

Profiler can be enabled easily like this because Profiler agent is installed in the new App Insights site extension and enabled by an App Setting. Snapshot Debugger is enabled through ASP.NET core hosting light up, the runtime will include an assembly if an environment variable is set.

The UI for the new enablement experience allows everything to be configured in one step:

The following App Settings are added to App Services for enabling Profiler and Snapshot Debugger:

Capture Interesting Profiler Traces On-Demand

We are excited to introduce the new on-demand triggering profiler feature. To make sure critical traces are not missed, you can go to the Profiler configuration pane and click on Profile Now button to start the profiler as needed. You can trigger profiler run when you are in the following situations:

◈ You want to get started using profiler by capturing the first traces to test everything is working.

◈ You want to efficiently and reliably capture traces during a load test run.

◈ You need to promptly capture traces for performance issues going on now.

In addition, you get more visibility into how profiler has been running from the Profiler run history list.

Investigate performance for ASP.NET Core on Linux using Profiler

Leveraging the Event Pipe technology, we can now capture traces for ASP.NET core web app running inside a Linux container hosted on App Services. The profiler runs in-proc in ASP.NET core to capture traces, which introduces less overhead. The current preview release is for evaluation purposes only.

Snapshot Health Check for Quickly Understand and Solve Issues

To address one of our top customer feedback for sometimes they cannot see snapshots for exceptions, we built a new feature to smartly help users diagnose reasons for missing snapshots. The service does health check on Snapshot Debugger based on user input. When missing snapshot, instead of not showing anything on the End-to-End trace viewer blade, we will show a link to help user troubleshoot what’s going on. We hope this can quickly help our customers to root cause and fix issues. We always strive to enable our customers’ success.

Continue reading

Deploying WordPress Application using Visual Studio Team Services and Azure – Part two

This post is the second part of two blog posts describing how to setup a CI/CD pipeline using Visual Studio Team Services (VSTS) for deploying a Dockerized custom WordPress website working with Azure WebApp for Containers and Azure Database for MySQL. In Deploying WordPress application using Visual Studio Team Services and Azure - part 1 we described how to build a Continuous Integration (CI) process using VSTS, while in this part we are going to focus on the Continuous Delivery (CD) part by using the VSTS Release management.

Prerequisites for this part

◈ MySQL client installed on Azure Virtual Machine (apt-get install mysql-client-5.7).
◈ Allow the connectivity between the Azure VM machine to Azure Databases for MySQL
◈ Installing this VSTS Route Traffic extension

Visual Studio Team Services – Release phase

I recommend on saving after completing each of the following steps:

First, we need to create a new empty release definition, go to releases, click on the + icon and choose create release definition, select empty process as the template.

Let’s start by adding artifacts to our pipeline. Click on add in the Artifacts section (step 1), a right blade will be presented, choose the following:

◈ Source type – Build

◈ Project – The relevant project name

◈ Source – The name of the build definition we created

◈ Default version – Latest

◈ Source alias – Keep the default

After clicking on Add, click on the trigger icon under the Artifacts section (step 2), a right blade will be shown, choose to enable the continuous deployment trigger.

We need to move to configure the environments section, click on pre-deployment conditions icon (step 3), select After release trigger and close the right blade.

Next step is adding tasks to the development pipeline, either click on the link (step 4) or click on tasks.

Development Pipeline

Under tasks tab, click on agent phase and select hosted Linux preview as agent queue. Add the following tasks by clicking the + icon:

◈ Three SSH command tasks

◈ One Azure App Service deploy task

◈ One Azure App Service Manage task

Now, we need to add variables that we going to use during the development CD process. Click on the variables tab and start adding from the below list, select development as the scope:

◈ $(destappinsight) –Application Insight Instrumentation Key of Dev environment

◈ $(desturl) – App Service URL of Dev environment

◈ $(migrationfile) – temp file name when executing DB backup and restore operation

◈ $(mysqldestdb) – DB name of Dev environment

◈ $(mysqlhost) – Server name of Azure Database for MySQL

◈ $(mysqlpass) – Password for Azure Database for MySQL

◈ $(mysqlport) – The Port for Azure Database for MySQL

◈ $(mysqlsourcedb) – DB name of Local environment

◈ $(mysqluser) – User name for Azure Database for MySQL

◈ $(resultfile) - temp file name when executing DB backup and restore operation

◈ $(sourceappinsight) - Application Insight Instrumentation Key of Local environment

◈ $(sourceurl) – Local environment URL

It’s possible to use a more secure solution for storing sensitive values, read more about using Azure KeyVault to store sensitive values and use them in VSTS.

Before returning to the tasks tab, we need to add a new SSH endpoint (Settings/Services/New Service Endpoint/SSH). Fill your Azure Virtual Machine details.

Now, let’s go back to the tasks tab and start editing the tasks.

The 1st SSH task is backup DB to file with the following values:

SSH endpoint – select the relevant SSH endpoint

Run – Commands

Commands –

mysqldump -P $(mysqlport) -h $(mysqlhost)  -u $(mysqluser)  -p$(mysqlpass) $(mysqlsourcedb) > $(resultfile)  

The 2nd SSH task is replace values with the following values:

SSH endpoint – select the relevant SSH endpoint

Run – Commands

Commands –

sed 's/$(sourceurl)/$(desturl)/g;s/$(sourceappinsight)/$(destappinsight)/g' $(resultfile) > $(migrationfile)

The 3rd SSH task is restore DB from migrated file with the following values:

SSH endpoint – select the relevant SSH endpoint

Run – Commands

Commands –

mysql -h $(mysqlhost) -u $(mysqluser)  -p$(mysqlpass) $(mysqldestdb) < $(migrationfile)

The 4th task is Azure App Service Deploy version 3 with the following values

Azure subscription – select the relevant Azure subscription

App type – Linux Web App

App Service name – select Dev environment App Service

Image Source – Container Registry

Registry or Namespace – Azure Container Registry login server value

Image – The Docker image name from CI process

Tag - $(Build.BuildId)

App settings –

-DB_ENV_NAME $(mysqldestdb) -DB_ENV_USER $(mysqluser) -DB_ENV_PASSWORD $(mysqlpass) -DB_ENV_HOST $(mysqlhost)

The 5th task is Azure App Service Manage, restart Azure App Service with the following values:

Azure subscription – select the relevant Azure subscription

Action – Restart App Service

App Service name - select the App Service of Dev environment

We have completed building the development CD pipeline.

Test Pipeline

Go back to pipeline tab, highlight development environment and choose to clone the environment (step 5), call the new environment test.

As Pre-deployment conditions of test environment (step 6), select after environment trigger, enable the pre-deployment approvals option, choose a member of your team as a approver to initiate the deployment process for test environment.

After closing the blade, click on the link below to view environment tasks (step 7).

The tasks tab will be presented, no need to update the first three SSH tasks.

The 4th task – update the App Service name to the correct App Service for test environment.

The 5th task – update again the App Service name.

Go to variables tab, filter according to test scope and set the variables values to test environment. We have now completed building the test CD pipeline.

Production Pipeline

Our goal is to have a production rollout without any downtime to achieve that we will use the slot mechanism and routing capabilities that Azure App Services are offering.

To create the production environment, repeat the same steps for creating the test environment (steps 8, 9 and 10).

Go to the variables tab, filter according to production scope and set the values to production environment. Set the value of $(mysqldestdb) to stage DB. In addition, add a new variable $(mysqlproddb) and set the value to production DB.

Go back to tasks tab, update the App Service name for the 4th and 5th tasks, this time check the deploy to slot option and choose ‘staging’ slot which we created for the production App Service.

Add additional tasks by clicking the + icon:

◈ One Route Traffic task - Select Production App Service, stage slot and route 100% of the traffic (see below screenshot)

◈ Two SSH command tasks – same configuration as other SSH tasks just different command

1st task Command:

mysqldump -P $(mysqlport) -h $(mysqlhost)  -u $(mysqluser)  -p$(mysqlpass) $(mysqldestdb) > $(resultfile)

2nd task command:

mysql -h $(mysqlhost) -u $(mysqluser)  -p$(mysqlpass) $(mysqlproddb) < $(resultfile)

◈ One Azure App Service deploy task

Same as 4th task but this time without checking the slot option

◈ One Azure App Service Manage task

Same as 5th task but this time without checking the slot option

◈ One Route Traffic task

Select Production App Service, stage slot and route 0% of the traffic

We have completed building the CD for Production. See the result:

Continue reading

New app usage monitoring capabilities in Application Insights

Our goal with Azure Monitoring tools is to provide full-stack monitoring for your applications. The top of this “stack” isn’t the client-side of your app, it’s your users themselves. Understanding user behavior is critical for making the right changes to your apps to drive the metrics your business cares about.

Recent improvements to the usage analytics tools in Application Insights can help your team better understand overall usage, dive deep into the impact of performance on customer experience, and give more visibility into user flows.

A faster, more insightful experience for Users, Sessions, and Events

Understanding application usage is critical to making smart investments with your development team. An application can be fast, reliable, and highly available, but if it doesn’t have many users, it’s not contributing value to your business.

The Users, Sessions, and Events tools in Application Insights make it easy to answer the most basic usage analytics question, “How much does my application and each of its features get used?”

We've re-built the Users, Sessions, and Events tools to make them even more responsive. A new sidebar of daily and monthly usage metrics help you spot growth and retention trends. Clicking on each metric gives you more detail, like a custom workbook for analyzing monthly active users (MAU). Also, the new “Meet your users” cards put you in the shoes of some of your customers, letting you follow their journeys step-by-step in a timeline.

Introducing the Impact tool

Are slow page loads the cause of user engagement problems in your app?

The new Impact tool in Application Insights makes it easy to find out. Just by choosing a page in your app and a user action on that page, the Impact tool graphs conversion rates by page load time. This makes it easy to spot if performance really does cause your users to churn.

The Impact tool can analyze more than just performance impact. It can look for correlations between any property or measurement in your telemetry and conversion rates. So you can see how conversion varies by country, device type, and more.

On our team, the Impact tool has uncovered several places where slow page load time was strongly correlated with decreased conversion rates. Better yet, the Impact tool quantified the page load time we should aim for, the slowest page load time that still had high conversion rates.

More capabilities for User Flows

Now the User Flows tool can analyze what users did before they visited some page or custom event in your site, in addition to what they did afterward. New “Session Started” nodes show you where a node was the first in a user session so you can spot how users are entering your site.

A new Split By option allows you to create more detailed User Flows visualizations by segmenting nodes by property values. For example, let’s say your team is collecting a custom event name with an overly generic name like “Button Clicked”. You can better understand user behavior by separating out which button was clicked by splitting by a “Button Name” custom dimension. Then in the visualization, you’ll see nodes to the effect of “Button Clicked where Button Name = Save,” “Button Clicked where Button Name = Edit,” and so on.

We’ve made a few smaller improvements to User Flows, too. The visualization now better adapts to smaller screen sizes. A “Tour” button gives you a step-by-step look at how to get more out of the User Flows tool. Also, on-node hide and reveal controls make it easier to control the density of information on the visualization.

Continue reading