Monitor Spring Boot applications end-to-end using Dynatrace

Today, we are announcing the integration of the Dynatrace Software Intelligence Platform in Azure Spring Cloud.

Over the past 18 months, we worked with many enterprise customers to learn about the scenarios they face. Many of these customers have thousands of Spring Boot applications running in on-premises data centers. As they migrate these applications to the cloud, they need to instrument them for application performance monitoring (APM) using solutions their developers are familiar with and have been using for years. In addition, they must ensure continuity for desktop and mobile applications that are already pre-instrumented for end-to-end monitoring using agents like Dynatrace OneAgent, which automatically discovers and maps all applications, microservices, and infrastructure as well as any dependencies in dynamic hybrid, multi-cloud environments. With the integration of Dynatrace OneAgent in Azure Spring Cloud, you can continue your journey and easily instrument your Spring Boot applications with Dynatrace.

Continue your Dynatrace journey. Most organizations that deploy Spring Boot applications today share a similar goal: maximize the benefits of running Spring Boot applications at virtually any scale, using automation and APM. While Azure Spring Cloud excels at abstracting away much of the toil associated with managing containerized workloads, the challenge of monitoring and maintaining the performance and health of these applications, or of troubleshooting issues when they occur, can be daunting—especially as organizations deploy these applications at massive scale. To help you succeed and continue your Dynatrace journey, we integrated and upgraded your ability to instrument, monitor, and deliver observability using Dynatrace OneAgent across your Azure Spring Cloud instances. That begins with setting up instrumentation quickly and easily. Then you can analyze the performance and health of your applications, JVMs, transactions, and more.

“For Liantis, true hybrid monitoring across both our on-premises and cloud-based Spring Boot microservices is key, but we also require simple and straightforward implementation—which is in line with the true Azure Spring Cloud philosophy of abstracting complexity. Doing so allows Liantis to spend more time on developing innovative applications, rather than building and operating infrastructure, which enables us to deliver true value for our customers and employees. Building on our in-house expertise with both Spring and Dynatrace technology, combined with our previous investments, the Dynatrace integration with Azure Spring Cloud was the obvious choice for Liantis.”—Nicolas Van Kerschaver, CIO, Liantis

“Being able to scale is critical for today’s digital business, as organizations have made the shift to cloud-native workloads and microservices. While cloud-native technologies and microservices have tremendous advantages, dynamic environments bring complexity that makes it difficult to understand the relationships and dependencies across an organization’s cloud ecosystem. Dynatrace’s strategic partnership with Microsoft allows us to extend the impact of our automatic and intelligent observability even further to accelerate digital transformation.  Through the Dynatrace integration with Azure Spring Cloud, we are enabling full visibility into application data for Spring Boot applications, which means more time innovating and a better product for end-users.”—Eric Horsman, Global Director of Strategic Alliances, Dynatrace

“At Microsoft, we are committed to helping our customers modernize their applications and innovate faster than ever before. By integrating a software intelligence solution like Dynatrace with Azure Spring Cloud, we can enable our customers with easy implementation of end-to-end observability, including automatic and continuous root-cause analysis, for their Spring Boot applications.”—Julia Liuson, Corporate Vice President, Developer Division, Microsoft

Watch the video above about how you can accelerate the transformation of Spring Boot applications with Microsoft Azure and Dynatrace.

Instrument your Spring Boot applications. Run a "provisioning" automation pipeline for a complete hands-off experience to instrument and monitor any new applications that you create and deploy—using Terraform or ARM Template. Or you can run it on-demand using the Azure CLI for greater flexibility and control.

az spring-cloud app update --name customers-service \

         --env DT_TENANT=<your-tenant> DT_TENANTTOKEN=<your-tenant-token> \

         DT_CONNECTION_POINT=<your-connection-point>

Automatic discovery and mapping of applications and their dependencies. To maintain real-time awareness in dynamic environments, Dynatrace automatically discovers and maps application components (including application servers, frameworks, and microservices), databases, messaging and eventing systems, and their relationships. In the view shown below, the Dynatrace Portal shows all the Spring Boot applications running in a production workload.

Figure 1: Shows all the Spring Boot applications running in a production workload

End-to-end observability of Spring Boot applications’ complete HTTP/S transactional behavior to understand the effect on business outcomes and user experiences. In the example view below, Dynatrace provides developers with all the transaction traces implemented in code without any code change to applications.

Figure 2: Shows transaction traces implemented in code without any code change to applications

Endpoint monitoring, API monitoring, DB calls monitoring, end-user experience monitoring. Dynatrace captures all the database queries initiated by your Spring Boot applications, including Azure database services. In the example view below, Dynatrace Portal shows all the active REST API operations within a production workload.

Figure 3: Shows all the active REST API operations within a production workload

In the example view below, the Dynatrace Portal shows all the database queries initiated by a production workload.

Figure 4: Shows all the database queries initiated by a production workload

Root-cause and impact analysis of application performance problems and business outcomes for faster, more reliable incident resolution. Dynatrace provides deep-code level visibility with end-to-end traces and the integration provides AI-assisted problem detection and automatic root-cause analysis allowing you to stay on top of your deployments and distinguish between healthy and unhealthy applications.

Figure 5: Shows results from stack trace analysis

Detect anomalies in your Spring Boot application instances. Dynatrace passes the collected data through an AI engine for automated root cause analysis, code level hotspot analysis, top database queries and exception analysis. In the example screenshot below, Dynatrace automatically identifies code modules that are CPU intensive so that you do not have to dig through the data.

Figure 6: Code modules that are CPU intensive so that you do not have to dig through the data

You can find all the top database queries initiated, how expensive these queries are, and how many times these queries are called by applications. In the example screenshot below, Dynatrace shows top database queries initiated by a production workload.

Figure 7: Shows top database queries initiated by a production workload

All application code level exceptions are logged along with many details into the stack traces of where the exception occurred. In the example screenshot below, the Dynatrace portal shows the top exceptions thrown by a production workload.

Figure 8—shows the top exceptions thrown by a production workload.

The Dynatrace Software Intelligence Platform automatically baselines all the performance metrics of Spring Boot applications. When the response times of an application increase beyond the auto-detected baseline, the platform creates an alert with information like how many response times have been breached from baselines. In the example screenshot below, Dynatrace shows response time degradation for a few services in a production workload.

Figure 9: Shows response time degradation for a few services in a production workload

Dynatrace gives you insights on what caused these increases in response time, particularly the time taken to make a connection to a database service. In the example below, the Dynatrace portal calls out the time taken to make connections to a database.

Figure 10: Shows the time taken to make connections to a database

Dynatrace automatically detects all the failures. In the example below, Dynatrace signals an increase in failure rates to reach an external network.

Figure 11: Signals an increase in failure rates to reach an external network

Focus on delivering value to your end-users. Once instrumented, as you scale out to multiple Spring Boot application instances, any new application instances are automatically monitored for you. Dynatrace enables application developers to observe Spring Boot applications end-to-end. You spend less time managing the agent installation and maintenance and more energy on identifying and resolving incidents faster. Azure Spring Cloud service is on-point for periodically updating the Dynatrace OneAgent.

Build your solutions and monitor them today

Azure Spring Cloud is jointly built, operated, and supported by Microsoft and VMware. It is a fully managed service for Spring Boot applications that abstracts away the complexity of infrastructure and Spring Cloud middleware management, so you can focus on building your business logic and let Azure take care of dynamic scaling, patches, security, compliance, and high availability. With a few steps, you can provision Azure Spring Cloud, create applications, deploy, and scale Spring Boot applications and start monitoring in minutes. We will continue to bring more developer-friendly and enterprise-ready features to Azure Spring Cloud.

Source: microsoft.com

Continue reading

Announcing Azure Spring Cloud Enterprise—fully managed VMware Tanzu components and advanced configurability for Spring Boot apps

In 2019, Microsoft and Pivotal (now VMware) announced Azure Spring Cloud, a fully managed service for Spring Boot applications. We set out to solve many of the common challenges enterprise developers face when running Spring Boot applications at scale. The service manages dynamic scaling, security patching, out-of-the-box instrumentation for monitoring, and more so developers can focus on their apps. Since then, we’ve worked with many customers including Kroger, Swiss Re, Raley’s, and Digital Realty to help them adopt the service.

We also learned that some customers need more. Many are running thousands of Spring Boot applications on-premises and need advanced capabilities to accelerate their Spring modernization projects. Based on our learnings, we started worked on a new Azure Spring Cloud tier with commercially supported components to meet the needs of enterprise customers. Now, we are announcing the availability of Azure Spring Cloud Enterprise in preview.

Azure Spring Cloud Enterprise is a managed service for Spring that is optimized for the needs of enterprise developers. We have collaborated with VMware to combine the cloud platform expertise of Microsoft with the innovation of the VMware Tanzu portfolio. Azure Spring Cloud Enterprise adds commercial Tanzu components built specifically to address enterprise requirements around configuration, integration, flexibility, and support.

Figure 1: Example—Azure Spring Cloud pricing tier selection, including Enterprise tier

Commercial VMware Tanzu components

With Azure Spring Cloud Enterprise, customers can use the VMware Tanzu components they know and love on managed Azure infrastructure. Tanzu Build Service, Tanzu Application Configuration Service, and Tanzu Service Registry are available during preview. Customers will have the flexibility to select which Tanzu components they want during or after instance creation. Microsoft and VMware will continue to add more Tanzu components such as Tanzu Spring Cloud Gateway and Spring Cloud Data Flow* to the service, providing increased value to customers.

*The Azure Spring Cloud Enterprise roadmap is not confirmed and is subject to change.

Figure 2: Example—VMware Tanzu settings and component selection in Azure portal

Advanced configurability and flexibility

Large enterprises often have complex workflows and need additional configuration options for their environments and development processes. Tanzu Build Service supports customizable Buildpack configurations that automate container creation and governance. Developers also get the full benefits of the Azure platform with limitless scaling and global deployment options, as well as integration with Azure services. And customers can move their existing Spring workloads built on Tanzu components to Azure Spring Cloud Enterprise, using the service to provide on-demand Spring Cloud infrastructure.

Spring Runtime support

Azure Spring Cloud Enterprise includes VMware Spring Runtime support for Spring projects. This gives you access to Spring experts and access to special versions of popular Spring projects specifically designed for enterprise use. With expert assistance, you can unlock the full potential of the Spring ecosystem and jumpstart your Spring application development.

Fully integrated with Azure

Azure Spring Cloud Enterprise runs on Azure in a fully managed environment. You get all the benefits of Azure, and the experience is familiar and intuitive. Just create your instances using a provisioning tool of your choice—Azure portal, Azure CLI, Azure Resource Manager Template, or Terraform.

Figure 3: Example—Azure Spring Cloud Enterprise creation review

After you create your instance and deploy your applications, you can easily monitor with Application Insights or other application performance management (APM) tools of your choice. As with the standard tier, Azure Spring Cloud Enterprise comes with out-of-the-box support for aggregating logs, metrics, distributed app traces, and alerts.

Figure 4: Example—Application transactions visible through Application Insights “Application Map”

Source: microsoft.com

Continue reading

5 reasons to attend the Azure data governance digital event

Data is one of your most strategic assets. But to take full advantage of it, you need to know what data you have, where it came from, and what regulatory requirements affect it. In short, you need a solution that consolidates all of your data and simplifies your ability to analyze it.

Azure Purview is that solution. This unified data governance solution gives you a holistic, up-to-date map of all your on-premises, multicloud, and software as a service (SaaS) data. By working with your entire data estate, Azure Purview enables you to mine your data for richer, more reliable business insights.

To show you the latest capabilities of Azure Purview—and share some exciting product announcements—my team will be hosting an Azure data governance digital event on Tuesday, September 28, 2021, from 9:00 AM – 10:00 AM Pacific Time.

Register now for this free event to:

1. Turbocharge your data governance strategy. Learn how to create a unified map of your data landscape that includes automated data discovery, sensitive data classification, and end-to-end data lineage.

2. Hear major product announcements. Be among the first to get the latest news about Azure Purview and learn about the latest innovations.

3. See Azure Purview in action. Watch in-depth demos of features like Azure Purview Data Map and Data Catalog and get an end-to-end look at the product. Plus:

◉ See how Azure Purview works with Azure Synapse Analytics, Power BI, SQL Server, Azure SQL, Microsoft Information Protection, and the rest of your data estate to deliver fast, actionable business insights.

◉ Explore how to easily combine all of your data sources for a seamless, unified experience and deliver trustworthy data at any scale.

◉ See how to scan your on-premises, multicloud, and SaaS data at scale to create a comprehensive map of your entire data estate.

◉ Learn how to use data classifications to automatically identify classifications and types of sensitive information.

4. Get insights and best practices from product insiders. Hear from leaders including:

◉ Rohan Kumar, Corporate Vice President, Azure Data

◉ Mike Flasko, Partner Director of Product, Azure Data Governance

5. Get answers to your data governance questions. Ask Microsoft product experts your questions in the live chat.

Register for the Azure data governance digital event now

Join us on September 28 to hear more about Azure data governance, engage with leaders and product experts, and learn how to maximize the business value of your data—no matter where it resides.

We hope to see you there!

Maximize the Value of Your Data in the Cloud

Achieve unified data governance with Azure Purview

Tuesday, September 28, 2021

9:00 AM–10:00 AM Pacific Time

Register now

Source: microsoft.com

Continue reading

Innovate securely with Azure

Security is based on the inherent need for safety. Today, we see that need challenged more than ever. In the past year alone, we’ve witnessed an exponential increase in ransomware, supply chain attacks, phishing, and identity theft. These activities fundamentally threaten our human desire for security in situations such as ransomware attacks on hospitals or supply chain attacks on industrial environments.

This security challenge meets us on a variety of fronts. While attacks like Nobelium demonstrate the level of sophistication of organized nation-state actors, most attacks exploit far simpler vulnerabilities, that are often publicly documented with patches already available. Why is security so much more challenging today?

For defenders, the surface area to protect has never been larger. Security teams are often chronically understaffed. On top of that, they’re overwhelmed by the volume of security signals—much of which is noise—and often spending valuable resources on non-security work, like maintaining infrastructure.

Something needs to change. Security needs to be a foundational principle, permeating every phase of the development process, from the cloud platform itself to the DevOps lifecycle, to the security operations processes.

Azure: The only cloud platform built by a security vendor

Facing these challenges requires you to embed security into every layer of architecture. As the only major cloud platform built by a security vendor, Azure empowers you to do that. Microsoft has deep security expertise, serving 400,000 customers including 90 out of the Fortune 100, and achieving $10 billion in security revenue as a result. Microsoft’s security ecosystem includes products that are leaders in a total of five Gartner Magic Quadrants and seven Forrester Waves, plus:

◉ Microsoft employs more than 3,700 security experts and spends more than $1 billion on security every single year.

◉ The volume of security signals that Microsoft analyzes is staggering—more than 8 trillion signals every 24 hours.

◉ In 2020 alone Microsoft 365 Defender blocked 6 billion malware threats.

In a nutshell, we get security. It’s this extensive experience that informs our approach to Azure security—security that is built-in, modern, and holistic.

Security is not a destination, but a continuous journey. Well-funded nation-state attackers will always continue to innovate. That’s why it’s so important to choose a cloud vendor who is constantly monitoring for security threats, constantly raising the bar on the security of the platform, and constantly assessing best practices.

Built-in: Security integrated into the DevOps lifecycle

Protecting your cloud innovation requires security to be built into every stage of the lifecycle and every level of architecture. If it isn’t, then developers struggle to integrate security into the DevOps cycle and security analytics may be required to slow down innovation or assets go unprotected. That’s why Azure has security built-in at every layer of architecture—not only at the runtime level but also when writing code.

GitHub Advanced Security features, for example, empower developers to deliver more secure code with built-in security capabilities like code scanning for vulnerabilities, secret scanning to avoid putting secrets like keys and passwords in code repositories. Another key area of focus is dependency reviews so that developers can update vulnerable open-source dependencies before merging code. This is important because 94 percent of projects use open source code in some form (GitHub Octoverse 2020 report).

Easily discover and turn on security tools with controls that are built directly into the Azure platform. Controls built into resources like the virtual machine, SQL, storage, and container blades puts security within easy reach for users beyond security professionals. Tools like Azure Defender help security operations (SecOps) to work at scale and enable protection and monitoring for all cloud resources. Azure also offers is broad policy support, automation, and actionable best practices.

Zero trust principles enforce security at every level of the organization. Azure is built on top of these key principles: verify explicitly and assume breach. Azure has a consistent Azure Resource Manager (ARM) layer to manage resources. This layer combines with our identity capabilities to deliver multi-factor authentication and least privilege access. What’s more, you get an architecture literally designed for Zero Trust with Azure’s built-in networking capabilities—spanning micro-segmentation to firewalls.

Modern: Security fueled by AI and the scale of the cloud

When you’re leveraging the power of AI and the scale of the cloud, defenders can protect, detect, and respond at a pace that enables them to get ahead of threats. It’s here that Microsoft’s wholesale commitment to security truly shines. Azure’s security approach is also very modern, especially compared to the tools that customers are using on-premises.

The AI used in Azure security solutions is powered by threat intelligence from across Microsoft’s entire security portfolio, encompassing trillions of signals per day and a large diversity of signals from the Microsoft Cloud. This allows Azure solutions to prioritize the most important incidents to raise to the security team, drastically cutting down on noise and saving SecOps precious time.

In addition, cloud-scale means that you always have the capacity you need, without investing in infrastructure setup and maintenance.

Holistic: Secure your entire organization, including Azure, hybrid, and multi-cloud

The attacks we’ve seen in recent years have proven that the age of effective point solutions is long over. Relying on a patchwork of disparate security solutions not only makes it harder for security teams to do their jobs—forcing them to pivot between many different tools—it also introduces far too many gaps for attackers to slip between.

That’s why it’s so important that security is holistic. Azure security solutions don’t just help you protect Azure—they protect your whole organization, including multi-cloud and hybrid environments. This gives you a unified view of your entire environment and enables SecOps to be more efficient with fewer tools.

For example, at the development phase, GitHub Advanced Security helps secure code deployed on any cloud. SecOps can get a bird’s eye view of your entire organization, including other clouds and your non-Microsoft security ecosystem, with Azure Sentinel, Microsoft’s cloud-native SIEM. Or, take it to the next level with integrated SIEM and XDR with Azure Sentinel, Azure Defender, and Microsoft 365 Defender and get comprehensive coverage combined with a view of the more important incidents that need attention immediately.

Plus, manage your cloud security posture across Azure, AWS, Google Cloud, and on-premises within one user experience in Azure Defender.

Where do you start?

Azure’s built-in, modern, and holistic solutions drastically simplify the process of securing your estate. But where do you start? Security is a shared responsibility. As an Azure customer here are five steps that we advise you to take now, whether you’re a new customer or an existing user:

1. Turn on Azure Secure Score. Azure Secure Score, located in Azure Security Center, gives a numeric view of your Azure security posture.

2. Turn on multi-factor authentication. Identity is such an important threat vector, and multi-factor authentication significantly reduces risk.

3. Turn on Azure Defender for all cloud workloads. Azure Defender protects against threats like remote desktop protocol (RDP) brute-force attacks, SQL injections, attacks on storage accounts, and much more. You can turn on Azure Defender with just a few clicks.

4. Turn on Azure WAF and DDoS protection for every website. This will ensure your web applications are protected from malicious attacks and common web vulnerabilities.

5. Turn on Azure Firewall for every subscription to protect Azure virtual networks.

Ongoing, it’s important that you assign a team member or partner to raise your Azure secure score percentage and engage your security operations team to action important incidents. This goes a long way towards improving your cloud security posture and lowering security risk.

Source: microsoft.com

Continue reading

Genomics testing on the ISS with HPE Spaceborne Computer-2 and Azure

This morning Microsoft News published a story about the use of Azure, enabled by HPE’s Spaceborne Computer-2 on the International Space Station (ISS). The project was designed to overcome the limited bandwidth between ISS and Earth by validating the benefits of a computational workflow that spans edge and cloud. Under this workflow, examination of high-volume raw data is processed and performed on the ISS using the HPE Spaceborne Computer-2’s edge computing platform and a much smaller data set containing only “interesting bits” is sent to Earth, where cloud resources are used to perform compute-intensive analysis to determine what those interesting bits really mean.

The Azure Space team performed the software development needed for the entire experiment in just three days.

A brief background

The International Space Station (ISS), a microgravity and space environment research laboratory, has just observed 20 years of continuous human presence. New technology is delivered to it regularly, as needed to keep up with the research being performed. Computers used on the ISS have typically been custom-built with specialized hardware and programming models, needed to deliver the reliability needed in space. Unfortunately, the developer experience for targeting these custom spaceborne systems is complex, making programming slow and challenging compared to the commercial-off-the-shelf systems used by most developers today.

Installed in 2017, Spaceborne Computer-1, designed by HPE, validated that a modern, commercial-off-the-shelf computer could survive a launch into space, be installed by astronauts, and operate correctly on the ISS—without “flipping bits” due to increased radiation in space. Basically, it was a year-long test to see if the computer hardware used on Earth would function normally in space. Building on this success, HPE’s Spaceborne Computer-2, an edge computing platform with purposely designed features for harsh environments, was installed in April 2021 to deliver twice as much compute performance, and for the first time, artificial intelligence (AI) capabilities to advance space exploration and research by enabling the same programming models and developer experiences used on Earth.

In many ways, Spaceborne Computer-2, which is comprised of the HPE Edgeline EL4000 Converged Edge system and HPE ProLiant DL360 Gen10 server, is the ultimate edge computing device platform, putting a game-changing amount of compute at the edge of space. However, the real limiting factor is the bandwidth between the ISS and Earth. Although Spaceborne Computer-2 supports the maximum available network speeds, it only receives from NASA an allocation of two hours of communication bandwidth a week to transmit data to earth, with a maximum download speed of 250 kilobytes per second.

In some cases, working around limited bandwidth can be accomplished by HPE helping researchers to compress data on Spaceborne Computer-2 before sending it down to Earth. In other cases, the data can be fully analyzed in space without needing to use the slow downlink at all. But what about research that requires more compute or bandwidth than what Spaceborne Computer-2 can provide, or that can be allotted to a single experiment among many? To address such scenarios, HPE applied its vision for an “edge to cloud” experience, in which Spaceborne Computer-2 is used to perform preliminary analysis or filtering on large data sets, extract what’s interesting or unexpected, and then burst those results down to Earth and into the public cloud for full analysis.

The Azure Space experiment

The Azure Space team at Microsoft proposed an experiment that simulates how NASA might monitor astronaut health in the presence of increased radiation exposure, as exists outside of our protective atmosphere. Such exposure will only increase as astronauts venture beyond the ISS’s low-earth orbit into and beyond the Van Allen Belts.

The experiment assumes access to a gene sequencer onboard the ISS, which is used to regularly monitor blood samples from astronauts. However, gene sequencing can generate an incredible amount of data—far too much for a 2Mbps/sec downlink—and the output needs to be compared against a large clinical database that’s constantly being updated.

To overcome those limitations, the experiment uses HPE Spaceborne Computer-2 to perform the initial process of comparing extracted gene sequences against reference DNA segments and capture only the differences, or mutations, which are then downloaded to the HPE ground station.

On earth, the data is uploaded to Azure, where the Microsoft Genomics service does the computational “alignment” work—the process of matching the short base-pair gene sequence reads in the downloaded data (which are about 70 base pairs in length) against the full 3 giga-base-pair human genome, as required to determine where in the human genome each mutation is located and the type of change (deletion, addition, replication, or swap). Aligned reads are then checked against the National Institute for Health’s dbSNP database to determine what the health impacts of a given mutation might mean. Watch the video below to see Azure in action.

Development process and computational workflow

The entire experiment was coded by 10 volunteers from the Azure Space team and its parent organization, the Azure Special Capabilities, Infrastructure, and Innovation Team. All major software components (both ISS-based and Azure-based) were written in Python and bash using Visual Studio Code, GitHub, and the Python libraries for Azure Functions and Azure Blob Storage. David Weinstein, Principal Software Engineering Manager at Azure Space, led the three-day development effort—consisting of a one-day hackathon and two days of cleanup.

The following graphic shows the computational workflow. It starts on the ISS, on Spaceborne Computer-2, which runs Red Hat Linux 7.4.

In space

◉ A Linux container hosts a Python workload, which is packaged with data representing mutated DNA fragments and wild-type (meaning normal or non-mutated) human DNA segments. There are 80 lines of Python code, with a 30-line bash script to execute the experiment.

◉ The Python workload generates a configurable amount of DNA sequences (mimicking gene sequencer reads, about 70 nucleotides long) from the mutated DNA fragment.

◉ The Python workload uses awk and grep to compare generated reads against the wild-type human genome segments.

◉ If a perfect match cannot be found for a read, it’s assumed to be a potential mutation and is compressed into an output folder on the Spaceborne Computer-2 network-attached storage device.

◉ After the Python workload completes, the compressed output folder is sent to the HPE ground station on Earth via rsync.

On Earth

◉ The HPE ground station uploads the data it receives to Azure, writing it to Azure Blob Storage through azcopy.

◉ An event-driven, serverless function written in Python and hosted in Azure Functions monitors Blob Storage, retrieving newly received data and sending it to the Microsoft Genomics service via its REST API.

◉ The Microsoft Genomics service, hosted on Azure, invokes a gene sequencing pipeline to “align” each read and determine where, how well, and how unambiguously it matches the full reference human genome. (The Microsoft Genomics service is a cloud implementation of the open-source Burroughs-Wheeler Aligner and Genome Analysis Toolkit, which Microsoft tuned for the cloud.)

◉ Aligned reads are written back to Blob Storage in Variant Call Format (VCF), a standard for describing variations from a reference genome.

◉ A second serverless function hosted in Azure Functions retrieves the VCF records, using the determined location of each mutation to query the dbSNP database hosted by the National Institute of Health—as needed to determine the clinical significance of the mutation—and writes that information to a JSON file in Blob Storage.

◉ Power BI retrieves the data containing clinical significance of the mutated genes from Blob Storage and displays it in an easily explorable format.

The Aligner and Analyzer functions total about 220 lines of code, with the Azure services and SDKs handling all of the low-level “plumbing” for the experiment. The functions are automatically triggered by blob storage uploads and are configured to point to the right storage accounts—requiring just a small amount of code to parse the raw data and query Microsoft Genomics and the dbSNP database at runtime.

Develop and test

During development and test, developers didn’t have access to HPE Spaceborne Computer-2 or the HPE ground station, so they recreated those environments on Azure, relying on GitHub Codespaces to further increase their velocity. They packaged both the ISS and ground station environments into an Azure Resource Manager (ARM) template, which simulates the latency between the ISS and the ground station by deploying the Spaceborne Computer-2 environment to an Azure data center in Australia and the ground station environment to one in Virginia.

The results

On August 12, 2021, the 120MB payload containing the experiment developed by Azure Space was uploaded to the ISS and run on Spaceborne Computer-2. The experiment is configurable, so Azure Space was able to execute “test”, “small”, and “medium” scenarios, executed in that order.

Table 1 shows the results of the experiment in terms of processing times and data volumes:

	Test	Small	Medium
Raw data examined	500KB	6MB	150MB
Downloaded to Earth	4KB	40KB	900KB
Run time on ISS	20 seconds	2 minutes	1 hour
Download time from ISS	<1 second	2 seconds	17 seconds

The experiment’s successful completion—and the data collected through it—is proof of how an edge-to-cloud computing workflow can be used to support high-value use cases aboard the ISS that might otherwise be impossible due to compute and bandwidth constraints. Without preprocessing the simulated output of the gene sequencer on the ISS to filter out only the gene mutations, 150 times as much data would need to be downloaded to Earth. Thus, a 200GB raw full human genome read which would require over two years to download given bandwidth and downlink window constraints, could be filtered to 1.5GB—which can be transmitted in just over an hour. Microsoft expects planned tests to further increase this ratio.

Similarly, attempting to perform all of the processing that’s being done on Azure would require uploading a copy of the full reference human genome and a copy of the full dbSNP database. To complicate matters, the dbSNP database is constantly being updated and peer-reviewed by scientists across the globe, meaning that regular synchronization would be required to maintain a useful copy in space.

Build cloud applications productively, anywhere

From a software development perspective, the developer velocity with which Azure Space delivered the experiment is as impressive as its results—with all components delivered over a three-day period using serverless Azure Functions written in Python, and best-in-class developer tools such as Visual Studio Code and GitHub. To support the development of additional experiments by others, Weinstein’s team at Azure Space plans to publish the Resource Manager templates containing the simulated ISS and ground station environments they used for development and test.

Making such capabilities available to others is just one early step for Azure Space, a new vertical within Microsoft that was publicly announced about a year ago. Its twofold mission: to enable organizations who build, launch, and operate spacecraft and satellites and to “democratize the benefits of space” by enabling more opportunities for all actors, large and small, in much the same way that support for open source on Azure has helped democratize cloud computing. One such example is Azure Orbital, a ground station as-a-service that provides communication and control for satellite operators—including the ability to easily process satellite data at a cloud-scale.

Source: microsoft.com

Continue reading

Discover and assess ASP.NET apps at-scale with Azure Migrate

Azure Migrate is your central place in the Azure portal that provides a unified experience to discover, assess, and migrate your servers, databases, and web apps to Azure. With a range of options across IaaS, PaaS, CaaS, and Serverless, Azure provides best-in-class flexibility and choice of platforms for your migrated workloads. With Azure Migrate we are making it easier than ever to migrate your applications to the Azure platform that best suits your business requirements. Last week we announced multiple enhancements to the App Containerization tool to help customers looking to adopt containers to run their migrated workloads. Today, we are announcing the preview of at-scale, agentless discovery, and assessment of ASP.NET web apps to help you migrate ASP.NET web apps and run them natively on Azure App Service.

With this preview, you can now easily discover ASP.NET web apps running on Internet Information Services (IIS) servers in a VMware environment and assess them for migration to Azure App Service. Assessments will help you determine the migration readiness of the web apps, migration blockers and remediation guidance, recommended SKU, and cost of hosting your web apps in App Service.

Unified onboarding for servers, databases, and web apps

Azure Migrate appliance for VMware helps with discovery, assessment, software inventory, application dependency analysis, and migration.

◉ Deploy a new Azure Migrate on-premises appliance or upgrade your existing appliance to start discovering your ASP.NET web apps. You can also use the appliance to inventory installed software and perform agentless dependency analysis.

◉ You do not need to provide separate credentials for web apps discovery. Credentials provided for dependency analysis are sufficient for web apps discovery. Please note that the account used should have local admin privileges on the source server to use these features.

◉ You can provide multiple credentials (domain, non-domain, and SQL authentication) for discovery. Azure Migrate appliance will automatically map server and database credentials across the entire estate. Moreover, credentials are encrypted and stored on your appliance in your datacenter. Credentials are not sent to Microsoft.

Web apps discovery

◉ You can discover up to 20,000 web apps with one Azure Migrate appliance.

◉ Web apps discovery surfaces information such as web app name, web server type and version, framework, URL, binding port, and application pool.

◉ You can also use the agentless dependency analysis feature to identify application tiers or interdependent applications. This information is useful when you need to plan migration for interdependent servers.

Azure App Service assessment and cost planning

Azure Migrate assessments provide you rightsizing and migration readiness recommendations. Once you have discovered your entire estate, you can start creating assessments for Azure IaaS, Azure VMware Solution, and Azure SQL. Now, you will also be able to create at-scale assessments of your web apps to determine their readiness for migration to Azure App Service, and to get recommended SKU, and cost details.

◉ You can customize assessments as per your requirements with the ability to customize assessment properties like target Azure region, application isolation requirements, and reserved instance pricing.

◉ You’ll get best-fit recommendations for the App Service SKU that the web app should be migrated to.

◉ In addition to migration readiness information, blockers and issues are surfaced so that you can mitigate them as needed.

◉ You can also modify assessment inputs at any time or create multiple assessments for the same set of web apps to compare and identify the target Azure options that work best for you.

Source: microsoft.com

Continue reading

Optimize your private mobile network and accelerate innovation with hyperscale cloud

The power of 5G: faster speeds, lower latencies, improved cost efficiencies, rich capabilities

The evolution of 5G has empowered enterprise organizations to do more. With the support of high-speed, high-bandwidth connections, and low-latency computing, these enterprises have been enabled to fuel entirely new forms of content and experiences. The convergence of 5G, multi-access edge computing (MEC), and the cloud make it possible for operators, managed service providers, and enterprises to create private wireless networks that are ultra-fast, secure, and scalable and that can take advantage of powerful cloud applications for analytics.

The future is poised to include innovative, far-reaching, and highly sophisticated services—the beginnings of which we can already see developing in smart cities and in many Industry 4.0 applications. Because of a broad range of factors—rising volumes of data, increased need for security and mobility, and demand for real-time processing, to name a few—private mobile networks are increasingly being viewed as a critical enabler in enterprise digital transformation. With the unique features of 5G already providing the foundation for sophisticated new services, Microsoft believes that private mobile networks represent a new way forward for emerging enterprise applications.

Recently, we’ve outlined how Azure private MEC enables operators and systems integrators to deliver private cellular, enterprise workloads, and edge compute services with ease, offering a robust partner network while combining computing, networking, and services on hardware located at the edge. In this article, we outline how bringing the power of hyperscale cloud to operator-enabled private mobile networks can lower capital expenditures (CapEx) and operating expenses (OpEx), accelerate the pace of innovation, and open up new and exciting revenue opportunities.

Partnering for private mobile networks

As enterprises look to take advantage of edge computing, they will need the combined expertise of operators for 5G technologies, and their partners for cloud and edge computing. Enterprises will benefit by partnering with operators and Managed Services Providers (MSPs) when deploying and managing 5G networks. Operators bring extensive experience in managing sophisticated mobile networks and have the tools and applications for end-to-end management. MSPs offer application diversity through their well-established partner ecosystem, allowing enterprises the deployment flexibility of managed services or turnkey implementation.

The proliferation of industry-specific use cases means there is no “one-size-fits-all” solution. A private mobile network applied to a remote oil-drilling platform, for example, will not meet the needs of an automotive manufacturer. To meet the diverse connectivity requirements, networks must be built with flexibility and simplicity in how they are configured, deployed, and managed.

An ideal solution

The key components of an effective private mobile network combine hyperscale cloud, private multi-access edge compute (MEC), LTE/5G mobile core, and end-to-end orchestration and management abilities into one easily deployable solution. When functioning in an optimal capacity, private mobile networks provide:

◉ Managed connectivity—operators can leverage the ecosystem of partners to address any connectivity needs of the enterprises, taking advantage of a substantial opportunity to deliver greater value.

◉ Managed services—these address the RAN, core, and edge components and package them within a centralized environment in the cloud, offering a single viewpoint.

◉ Self-management options—portals and dashboards can be implemented to offer service customization and visibility with ease, delivering true value to the enterprise in the form of service-level assurance.

What Azure offers: An integrated approach

Our approach to private mobile networks provides operators and managed service providers with a completely integrated solution, from the mobile core to the edge to the cloud. Our solution architecture delivers unique advantages to both operators and enterprises, such as integration with multiple radio access technologies and an advanced edge computing platform that can host both the mobile core and O-RAN components (such as vCU and vDU) from within our partner ecosystem.

The solution architecture features several key components: Azure Stack Edge, Azure Network Function Manager, and Affirmed mobile core (4G/5G) network functions.

First, Azure Stack Edge enables operators and systems integrators to easily deliver ultra-low-latency networking, applications, and services at the enterprise edge. It provides a single point for processing mobile network data, and integration with Affirmed 4G/5G mobile core technology enables local, intelligent breakout of data processing and seamless data sharing for faster processing and lower bandwidth consumption. As a fully managed compute and connectivity solution, customers stay focused on developing new scenarios and revenue options.

Next, one must consider scalability as an essential need with private network deployment. Microsoft’s approach to provisioning, deployment orchestration, and management provides the speed, agility, and automation required to deploy and manage private mobile networks at scale, automating the lifecycle management of private network services in the process.

Azure Network Function Manager (NFM), a cloud-native orchestration service allows customers to deploy and provision network functions on Azure Stack Edge Pro, for a consistent hybrid experience using the Azure portal. The consistency comes from using a familiar platform like Azure Marketplace to pick from among a curated list of pre-validated offers (to ensure proper operation at the edge), and then Azure Portal to deploy network functions as managed applications.

Lastly, the Affirmed 5G Mobile Core is a fully virtualized, cloud-native solution that includes all standard 5G core network functions, with integrated virtualized network probes plus enhanced functionality such as Wi-Fi interworking and service automation.

The benefits of Microsoft’s private mobile network approach are clear

Microsoft’s fully integrated, cloud-native approach to private mobile networks has clear benefits to operators and MSPs. These range from providing a proven 5G mobile core architecture as the foundation for a carrier-grade network experience, to offering a cloud-managed solution that’s designed to meet their unique needs. Simplified administration, security, and operation come via automated lifecycle management, and service assurance that meets five-nines availability to support mission-critical applications. Operators and MSPs can now solve the critical infrastructure challenges involved with managing and deploying a private mobile network and monetize the enterprise opportunity.

Source: microsoft.com

Continue reading