Raise the bar on AI-powered app development with Azure Database for PostgreSQL

Known for its reliability and versatility, PostgreSQL is a popular and powerful open-source database system with a wide array of features. By harnessing the might of PostgreSQL in the cloud—with all the scalability and convenience you expect—comes Microsoft Azure Database for PostgreSQL. This fully managed service takes the hassle out of managing your PostgreSQL instances, allowing you to focus on what really matters: building amazing, AI-powered applications.

To better get you acquainted with how Azure Database for PostgreSQL empowers users to migrate their PostgreSQL databases and build intelligent apps, this blog will introduce a roster of new learning paths and events, including a pair of Cloud Skills Challenges. As if that’s not exciting enough, completing one of the challenges automatically enters you in a drawing for a great prize. So, let’s get going! 

Seamless database migration and app creation

Say goodbye to tedious maintenance tasks and hello to seamless deployments, automated patching, and built-in high availability. Azure Database for PostgreSQL is a fully managed service that simplifies the migration of existing PostgreSQL databases to the cloud. We handle the burdens of patching, backups, and scaling—allowing you to focus on your applications. 

Seamless compatibility with PostgreSQL minimizes code changes during the transition and caters to diverse needs and budgets. With migration tooling in Azure Database for PostgreSQL, transferring data and schemas to the cloud becomes a breeze. 

Beyond migration, Azure Database for PostgreSQL empowers the development of AI-powered applications. Its native support for the pgvector extension allows for efficient storage and querying of vector embeddings, essential for AI and machine learning tasks. The service seamlessly integrates with other Azure AI services, such as Azure Machine Learning, Azure OpenAI Service, Microsoft Azure AI Language, and Microsoft Azure AI Translator, providing developers with a rich toolkit for building intelligent applications.

Additionally, the service’s scalability ensures optimal performance as AI workloads grow, maintaining cost efficiency throughout the development process. Overall, Azure Database for PostgreSQL provides a comprehensive solution for both migrating to the cloud and building powerful AI applications. 

Here are some key features: 

• High availability: Up to 99.99% uptime guaranteed with zone-redundant high availability, automated maintenance, patching, and updates.
• Performance automation: Get analysis of your database workloads to identify opportunities to improve query performance with query store and index recommendations.
• Security: Includes Microsoft Defender for open-source relational databases to protect your data, and Azure IP Advantage, which is designed to protect businesses and developers who build on Azure from intellectual property risks.
• Azure AI extension: Generate and store vector embeddings, call Azure AI services, and build AI-powered apps directly within the database.
• Migration support: Tools to migrate Oracle Database to Azure Database for PostgreSQL are available, making the transition smoother.
• Cost-effective: Provides operational savings—up to 62% compared with on-premises—with comprehensive database monitoring and optimization tools, which can lead to a lower total cost of ownership. 

Learn at your own pace with curated lessons 

Now that you’ve gotten a primer on Azure Database for PostgreSQL, the next step is engaging with our curated learning paths. The collected modules in these two courses include readings, exercises, and knowledge checks.

◉ Build AI Apps with Azure Database for PostgreSQL

Designed for developers interested in harnessing AI within their PostgreSQL applications on Azure, this learning path explores how the Azure AI extension for Azure Database for PostgreSQL can be leveraged to incorporate AI capabilities into your apps.

By completing this learning path, you’ll gain a solid understanding of the Azure AI extension and its various functionalities. Discover how to evaluate different summarization techniques available through Azure AI services and the azure_ai extension, explore the differences between extractive, abstractive, and query-focused summarization, and apply generative AI summarization techniques to data within a PostgreSQL database. This hands-on experience will empower you to leverage Azure AI services and the azure_ai extension to build intelligent applications that can summarize complex content into concise and informative summaries. 

◉ Configure and migrate to Azure Database for PostgreSQL

This learning path supplies you with the essential skills needed to effectively work with Azure Database for PostgreSQL. It begins with a foundational understanding of PostgreSQL architecture and core concepts, before delving into practical aspects such as connecting to the database, executing queries, and ensuring robust security measures.

You’ll also learn how to create and manage databases, schemas, and tables, and how to leverage stored procedures and functions for code reusability. With insights into how Azure Database for PostgreSQL implements ACID transactions and write-ahead logging for data integrity and durability, you’ll gain confidence in configuring, managing, and migrating existing PostgreSQL databases to Azure.

Complete timed challenges to win Azure prizes

To go along with these learning paths, we’ve also assembled a pair of corresponding Azure Database for PostgreSQL Cloud Skills Challenges. While learning paths are usually self-paced, solitary activities, Cloud Skills Challenges are part interactive learning sprint, part good-natured tournament between you and thousands of your peers around the globe. They’re immersive, gamified learning experiences blending hands-on exercises, tutorials, and assessments to ensure a well-rounded learning experience. 

Connect with PostgreSQL experts at POSETTE 2024 conference

Hosted by Microsoft, POSETTE 2024 (formerly Citus Con), is an exciting developer event dedicated to all things PostgreSQL. The event is a unique opportunity to learn from experts, network with fellow Postgres enthusiasts, and delve into the latest innovations in database technology. 

As a key player in the PostgreSQL community, we’ll be showcasing our commitment to the open-source database system. Attendees can look forward to a session on the future of Azure Database for PostgreSQL, where our experts will share our vision for the service and its integration with other Azure offerings.

Running June 11 to 13, 2024, POSETTE—which stands for Postgres Open Source Ecosystem Talks, Training, and Education—is a free, virtual event featuring four unique livestreams. Registration is optional, and all scheduled talks will be online to watch immediately after the event ends. Don’t miss out on this chance to connect with the Microsoft team and learn how we’re advancing PostgreSQL in the cloud.

Take the next step on your Azure Database for PostgreSQL journey

Whether you’re a seasoned developer or just starting out, PostgreSQL and Azure Database for PostgreSQL is a dream team for building modern, scalable, and AI-powered apps. By offering robust migration tools and seamless integration with AI and machine learning services, Azure Database for PostgreSQL helps users efficiently migrate to the cloud and build sophisticated AI applications.

Get started today with our pair of learning paths and their respective Cloud Skills Challenges to be entered into a drawing for cool Azure prizes, then check out the POSETTE 2024 livestreams to learn more about everything you can do with the world’s most advanced open-source database.

Source: microsoft.com

Continue reading

Azure Data, AI, and Digital Applications: Harness the Power of Intelligent Apps

In today's rapidly evolving digital landscape, leveraging Azure Data, AI, and Digital Applications is essential for businesses aiming to stay ahead of the curve. By harnessing the power of intelligent apps, organizations can unlock unprecedented levels of efficiency, insight, and innovation. This comprehensive guide explores how Azure's suite of tools and services can transform your business operations and drive significant growth.

Understanding Azure's Data Capabilities

Azure offers a robust set of data services designed to help businesses store, manage, and analyze their data efficiently. These services are critical for building intelligent applications that can provide actionable insights and drive business value.

Azure SQL Database

Azure SQL Database is a fully managed relational database service that offers high performance, scalability, and security. It supports modern cloud applications and facilitates easy migration from on-premises databases. Key features include automated backups, built-in high availability, and advanced security features such as encryption and threat detection.

Azure Cosmos DB

For businesses requiring a globally distributed, multi-model database service, Azure Cosmos DB is an ideal choice. It offers seamless scalability and guarantees low latency, high availability, and consistency. Cosmos DB supports various data models, including document, key-value, graph, and column-family, making it versatile for different application needs.

Azure Data Lake Storage

Azure Data Lake Storage is designed for big data analytics, providing a scalable and secure data lake for high-performance analytics workloads. It integrates with Azure Data Services like Azure Databricks and Azure Synapse Analytics, enabling seamless data ingestion, processing, and analysis.

Harnessing the Power of AI with Azure

Artificial Intelligence (AI) is at the core of transforming business processes and applications. Azure provides a comprehensive suite of AI services that empower businesses to build, deploy, and manage AI models at scale.

Azure Machine Learning

Azure Machine Learning is a cloud-based service that allows data scientists and developers to build, train, and deploy machine learning models. It supports end-to-end machine learning lifecycle management, including data preparation, model training, deployment, and monitoring. With Azure Machine Learning, businesses can leverage automated machine learning (AutoML) to accelerate model development and improve accuracy.

Azure Cognitive Services

Azure Cognitive Services offers pre-built AI models that can be easily integrated into applications. These services include Vision, Speech, Language, and Decision APIs, enabling businesses to incorporate capabilities such as image and video analysis, speech recognition, language understanding, and anomaly detection into their apps without needing deep AI expertise.

Azure Bot Services

Azure Bot Services simplifies the process of building intelligent bots that can interact with users across various channels, including websites, mobile apps, and messaging platforms. These bots leverage natural language processing (NLP) to understand and respond to user queries, enhancing customer engagement and support.

Building Intelligent Applications with Azure

Intelligent applications leverage data and AI to deliver enhanced user experiences, improve decision-making, and drive operational efficiency. Azure provides a range of tools and frameworks to support the development of these applications.

Azure Logic Apps

Azure Logic Apps allows businesses to automate workflows and integrate various systems and services. With its visual designer, users can create workflows that connect cloud-based and on-premises applications, enabling seamless data flow and process automation.

Azure Functions

Azure Functions is a serverless computing service that enables developers to build and deploy event-driven applications. It supports various programming languages and can be triggered by numerous events, such as HTTP requests, database changes, or messages from other services. This flexibility makes it ideal for building microservices and automating backend processes.

Azure App Service

Azure App Service provides a fully managed platform for building, deploying, and scaling web apps and APIs. It supports multiple languages and frameworks, including .NET, Java, Node.js, and Python. With built-in DevOps capabilities, businesses can automate deployment and ensure continuous integration and delivery (CI/CD).

Integrating Data and AI for Maximum Impact

Combining data and AI enables businesses to create intelligent applications that deliver real-time insights and personalized experiences. Azure's integrated services facilitate this integration, ensuring seamless data flow and model deployment.

Azure Synapse Analytics

Azure Synapse Analytics is an analytics service that brings together big data and data warehousing. It provides a unified platform for data ingestion, preparation, management, and serving, allowing businesses to analyze large volumes of data in real-time. With built-in AI and machine learning capabilities, Azure Synapse Analytics enables predictive analytics and advanced data visualization.

Azure Databricks

Azure Databricks is an Apache Spark-based analytics platform optimized for Azure. It supports collaborative data engineering, data science, and machine learning workflows, enabling teams to work together efficiently. Azure Databricks integrates with other Azure services, such as Azure Data Lake Storage and Azure Machine Learning, to streamline the development of intelligent applications.

Azure Data Factory

Azure Data Factory is a data integration service that allows businesses to create, schedule, and orchestrate data workflows. It supports the ingestion of data from various sources, including on-premises databases, cloud-based storage, and SaaS applications. With its robust ETL capabilities, Azure Data Factory ensures that data is processed and transformed efficiently, ready for analysis and AI model training.

Enhancing Security and Compliance

Security and compliance are paramount when dealing with data and AI. Azure offers a comprehensive set of security features and compliance certifications to protect data and ensure regulatory adherence.

Azure Security Center

Azure Security Center provides advanced threat protection across hybrid cloud environments. It offers continuous assessment and security recommendations, helping businesses to strengthen their security posture. With integrated threat intelligence and AI-driven insights, Azure Security Center enables proactive threat detection and response.

Azure Active Directory

Azure Active Directory (Azure AD) is a cloud-based identity and access management service that ensures secure user authentication and authorization. It supports single sign-on (SSO), multi-factor authentication (MFA), and conditional access policies, enhancing security while simplifying user access to applications and data.

Compliance Certifications

Azure meets a wide range of industry and regional compliance standards, including GDPR, HIPAA, ISO 27001, and SOC 2. This extensive compliance portfolio ensures that businesses can confidently handle sensitive data and meet regulatory requirements.

Driving Innovation with Azure

By leveraging Azure's data, AI, and digital application capabilities, businesses can drive innovation and stay competitive in their respective industries. Azure's comprehensive ecosystem supports the development of intelligent applications that can transform customer experiences, optimize operations, and unlock new business opportunities.

Continue reading

Tackle large volumes of data with new solution from SLB for Azure Data Manager for Energy

As the energy industry continues to innovate, companies find themselves balancing the ever-growing demand for energy with the desire to work toward more efficient, sustainable operations. Decision makers spread across the globe rely on accurate data to improve confidence and minimize risk, so harnessing the power of data has become a central tenant of energy companies’ success as they push to evolve. However, different types of data and the variety of file types that energy companies manage daily make it difficult to access and analyze the data efficiently.

That is why energy companies around the world are using Microsoft technologies, including Azure Data Manager for Energy. This cloud-based OSDU® Data Platform maximizes customer performance by being truly interoperable across the exploration and production data landscape. It can integrate with industry datasets, applications, and other cloud services, which makes it easier for employees throughout an organization to access the data they need quickly and efficiently. These sources may include other data platforms for asset performance, the internet of things (IoT), or production, but also databases that may still be deployed on-premises before migration to the cloud can take place. Data siloes slow productivity and complicate workflows—optimizing access to high quality data is crucial for success.

The Microsoft partner ecosystem is a key component in how Microsoft delivers technology, services, and cloud-to-edge solutions for our customers. SLB—one of Microsoft’s strategic partners in the energy sector—designed the Enterprise Data Solution to work seamlessly with Azure Data Manager for Energy, easing data ingestion, management, and discoverability for domain applications.

Enterprise Data Solution from SLB facilitates day-to-day workflows

Designed to reduce the friction of dealing with large volumes of data traditionally stored in different silos, Enterprise Data Solution from SLB features a user-friendly, inclusive interface that facilitates the process of data ingestion, curation, and accessibility. Enterprise Data Solution from SLB allows customers to unlock data with a single click instead of running multiple steps to get the same results. This makes it easier for a variety of end users to ingest different data types and formats and access data faster. In addition, the curation of data along this pipeline is aided by several AI-enhanced tools, from more streamlined unstructured data ingestion, data quality control, contextualization, and data mastering, reducing the need for manual intervention.

Both traditional and new energy technical workflows are optimized when data and application interoperability are achieved. For example, handling large volumes of seismic data (access, ingestion, and streaming) can be dramatically improved with Enterprise Data Solution from SLB. The typical cost for handling these workloads is also reduced significantly, speeding up workflows and allowing users to incorporate as much diverse domain data as possible. Geoscientists and engineers can run more scenarios in different applications, compare results with their colleagues, and ultimately liberate the most accurate data quickly.

“Having the Enterprise Data Solution from SLB working with ADME is transformative. Workflow efficiency is vastly increased, and decision-making accelerated to levels that are impossible to achieve without it. Pairing the two technologies enables AI to work at full effect in the energy sector. AI plays a dual role, it enhances data quality, accessibility, and utilization, which in turn enables it to drive innovation across the entire planning to operations life cycle.”

—Jamie Cruise, Product Leader, SLB

Key features of Enterprise Data Solution from SLB

The key features of Enterprise Data Solution from SLB include:

Ingest: Collect, ingest, and stage data easily from desktop or other file sources for document transcription and parsing. For example, you can use a cloud-style file drop for changes to files and get automated translation into the correct schema. This is increasingly aided by AI to reduce human workload requirements.

Curate: Data quality is at the heart of many of the challenges in the industry and causes a lot of wasted time. The Enterprise Data Solution from SLB accelerates curation and refining of datasets into trusted data products, liberating them with streamlined tools for data quality so they’re ready for consumption. These tools include AI for quality control and mastering.

Discover: Ready-to-use data products enable quick data-asset analysis and screening online for quick-to-create collections and consumption from domain workflows. Locating and accessing quality data is critical for decision-making. Enterprise Data Solution from SLB enables workflows to access the right data for stakeholders to easily discover, visualize, and use.

Consume: The Enterprise Data Solution from SLB makes it easy for analysts and others to access trusted data products from within their familiar applications and workflows, whether that is the Delfi™ digital platform, Petrel™ subsurface software, or Techlog™ wellbore software. With an intuitive user interface that’s accessible from a web browser, full data lineage, and project versioning are always available in an optimized format. Data can be shared and consumed in workflows such as seismic processing, machine learning seismic interpretation, and machine learning property modeling. In addition, this data is now easily consumable in machine learning workbenches such as Delfi Data Science or tooling available in Microsoft Azure. This cuts time to decision and action, critical components for a smooth production workflow.

Unlock downstream advantages within the Microsoft ecosystem

Once data is in Azure Data Manager for Energy, it conforms to the OSDU® Technical Standard and is a reliable platform for other applications to consume and re-ingest data iteratively. The interoperability of Azure Data Manager for Energy improves accessibility, traceability, and validity of data, allowing domain users and data scientists to deliver business outcomes faster.

Enterprise Data Solution from SLB helps customers take full advantage of Azure Data Manager for Energy by making it more efficient to ingest large amounts of quality, trusted, and traceable data into the platform. Ultimately, Azure Data Manager for Energy’s interoperability empowers customers by harmonizing data that can be leveraged across the Microsoft ecosystem.

Once this data is in the platform, there are many opportunities to take advantage of Microsoft Azure OpenAI Service to drive additional insights and efficiencies further downstream in Microsoft 365 applications. For example, end users can extract tables directly from ingested documents and generate Open XML documents that are ready to use in Microsoft Excel, where they can be more easily visualized.

Source: microsoft.com

Continue reading

Azure Data Dynamics: Revolutionizing Data Solutions for Tomorrow

In the ever-evolving landscape of data management, Azure Data Dynamics emerges as a beacon of innovation and efficiency. As we navigate the intricate web of data solutions, it becomes imperative to comprehend the transformative power of Azure Data Dynamics and its pivotal role in shaping the future of data management.

Unraveling the Azure Data Dynamics Tapestry

Understanding the Core Principles

At the heart of Azure Data Dynamics lies a set of core principles designed to redefine how organizations interact with their data. Scalability, flexibility, and seamless integration are not just buzzwords here; they are the guiding principles that empower businesses to thrive in a data-centric era.

Unmatched Scalability for Growing Enterprises

For enterprises with ambitions of growth, scalability is not a luxury but a necessity. Azure Data Dynamics, with its cloud-based infrastructure, offers an unparalleled scalability that caters to the dynamic needs of a growing business. Whether you're handling terabytes or petabytes of data, Azure seamlessly adapts to your scale, ensuring performance remains optimal.

Flexibility Redefined: Tailoring Solutions to Fit Your Needs

One size does not fit all in the realm of data solutions. Azure Data Dynamics recognizes this, offering a level of flexibility that is unmatched in the industry. Through a modular approach, businesses can cherry-pick the components that align with their specific requirements, creating a tailor-made data management ecosystem.

Seamless Integration for a Unified Data Experience

The future of data solutions is not about isolated systems; it's about cohesion and unity. Azure Data Dynamics excels in providing seamless integration with existing infrastructures, ensuring a smooth transition without disrupting ongoing operations. This interoperability is not just a feature; it's a testament to the commitment of Azure to harmonize the data experience.

Azure Data Dynamics in Action

Real-Time Analytics: Navigating the Speed of Thought

In the fast-paced business environment, decisions need to be made swiftly and accurately. Azure Data Dynamics empowers organizations with the ability to conduct real-time analytics, unraveling insights at the speed of thought. Whether it's market trends, consumer behavior, or operational efficiency, the capability to analyze data in real-time provides a competitive edge.

AI-Driven Predictive Analytics: Anticipating the Future

Predicting future trends is no longer a luxury but a strategic necessity. Azure Data Dynamics incorporates state-of-the-art AI-driven predictive analytics to help businesses anticipate market shifts, customer preferences, and potential bottlenecks. By leveraging machine learning algorithms, organizations can make data-driven decisions that go beyond the realms of traditional analytics.

Advanced Security Measures: Safeguarding Your Data Fortress

In an era where data breaches are a constant threat, the security of sensitive information is paramount. Azure Data Dynamics takes security seriously, implementing advanced security measures to safeguard your data fortress. From encryption protocols to multi-factor authentication, Azure ensures that your data remains impervious to external threats.

Embracing the Future Today

As we delve into the intricacies of Azure Data Dynamics, it becomes evident that it is not just a solution for today but a roadmap for the future. The ability to scale, the flexibility to adapt, and the seamless integration with cutting-edge technologies make Azure Data Dynamics a catalyst for organizations aiming to thrive in the evolving landscape of data management.

Conclusion

In conclusion, Azure Data Dynamics emerges as a game-changer, redefining how businesses interact with their data. With an unwavering commitment to scalability, flexibility, and seamless integration, Azure Data Dynamics is not just a solution; it's a strategic investment in the future.

Continue reading

Unleashing the Power of Azure Data: A Comprehensive Guide

Introduction

In the ever-evolving landscape of technology, harnessing the capabilities of cloud computing has become paramount for businesses striving for efficiency and innovation. Among the multitude of cloud services available, Azure Data stands out as a powerhouse, offering a myriad of tools and solutions for data management and analytics.

Understanding Azure Data

What sets Azure Data Apart?

Azure Data, part of the robust Microsoft Azure ecosystem, distinguishes itself with its seamless integration of diverse data solutions. From storage to analytics, Azure Data provides a holistic approach to handling data, making it an ideal choice for enterprises seeking a comprehensive and scalable platform.

Key Components of Azure Data

Azure Data Lake Storage

Azure Data Lake Storage forms the cornerstone of data storage in the Azure ecosystem. With limitless scalability and the ability to handle both structured and unstructured data, it empowers businesses to store and process data efficiently.

Azure Synapse Analytics

Formerly known as SQL Data Warehouse, Azure Synapse Analytics takes data analytics to new heights. It seamlessly integrates with various data sources, enabling organizations to derive meaningful insights through advanced analytics and machine learning.

Azure Databricks

For those delving into big data and machine learning, Azure Databricks provides a collaborative environment for data scientists and engineers. Its integration with popular programming languages facilitates the creation of scalable and efficient data workflows.

Unlocking the Potential: Use Cases of Azure Data

Streamlining Data Management

Azure Data simplifies data management with its intuitive interfaces and robust features. Whether it's processing real-time data streams or efficiently managing large datasets, Azure Data emerges as the go-to solution for organizations aiming to streamline their data operations.

Accelerating Analytics

The integration of Azure Synapse Analytics allows businesses to accelerate their analytics processes. By harnessing the power of distributed computing, organizations can perform complex queries and gain valuable insights at unprecedented speeds, giving them a competitive edge in the market.

Enabling Predictive Insights with Azure Databricks

Azure Databricks facilitates the creation of predictive models, enabling organizations to harness the power of machine learning. From predicting customer behavior to optimizing supply chain processes, the possibilities are limitless with Azure Databricks.

Implementation Best Practices

Ensuring Security and Compliance

As data becomes a critical asset, ensuring its security is paramount. Azure Data employs robust security measures, including encryption, access controls, and compliance certifications, providing organizations with the confidence that their data is protected.

Scalability for Future Growth

One of the standout features of Azure Data is its scalability. As businesses grow, the platform effortlessly scales to accommodate increased data volumes and processing demands, ensuring that organizations can adapt to changing business landscapes seamlessly.

Conclusion

In conclusion, Azure Data emerges as a game-changer in the realm of cloud-based data solutions. Its comprehensive set of tools, seamless integration, and scalability make it the ideal choice for businesses looking to unlock the full potential of their data.

Continue reading

4 cloud cost optimization strategies with Microsoft Azure

We have seen many businesses make significant shifts toward cloud computing in the last decade. The Microsoft Azure public cloud offers many benefits to companies, such as increased flexibility, scalability, and availability of resources. However, with the increased usage of resources, implementing best practices in cloud efficiency is a necessity to validate spending and avoid waste.

What is cloud efficiency? It is the capacity to utilize cloud resources in the best possible way, and at the lowest possible cost while, at the same time, minimizing the waste of resources, and thus of energy and carbon emissions. It’s a combination of cost—how you handle and govern your cloud infrastructure, carbon—how you can keep carbon emissions at a minimum, and energy—how the application uses electricity, and how you can optimize these three areas to make the cheapest, more modern, efficient, and sustainable application. In this post, we will explore why you should immediately start your cloud cost management and governance process.

Cloud cost optimization is essential for companies as it directly impacts their bottom line and OPEX expenses. The cost of cloud computing can quickly add up, especially for businesses with a high volume of data or high traffic, and mission-critical applications.

Cloud cost optimization is what makes workloads more efficient, but what are its benefits?

◉ Understanding, measuring, optimizing, and tracking your cloud costs. Having full control of your monthly bill should be your primary goal.

◉ Reduce carbon emissions. Cloud computing consumes a significant amount of energy, and the increased usage of cloud resources has resulted in a substantial increase in carbon emissions. Cloud providers are taking steps to reduce their carbon footprint, but businesses can also play a significant role in reducing carbon emissions by optimizing their cloud resources.

◉ Improve the performance of applications. This can significantly impact user experience, as slow or unresponsive applications can lead to frustrated customers and lost revenue. By optimizing cloud resources, companies can ensure that their applications run smoothly, improving customer satisfaction, and decreasing cloud spend.

◉ Saving on your application’s cost in a systematic way can give you a budget for additional features, refactoring, and innovation.

The four main cloud cost optimization strategies are usually:

1. Right sizing

Right-sizing is probably the most important aspect of controlling cloud costs. The impact is not simply saving money—in many cases, there is a balance between performance and spending and, more specifically, between meeting your internal customer service-level agreements (SLAs) efficiently. You need to find this balance to keep both your application managers, financial operations (finops) team, and cloud team happy.

2. Clean-up

Another important part of cloud computing cost saving is cleanup operations. When dealing with many workloads or complex projects, lots of resources are created just as a transitional step and are often forgotten about and paid for. This is particularly valid during lift and shift migration where customers choose to initially match resources that were in a fixed, non-flexible environment, ending up with overallocated services. Cleaning up unused items—as a first approach—represents one of the short-term, quick wins for cost-saving. When inserted into a recurring process, this will also help you uncover any unassigned or unutilized infrastructure (with operational downfalls) and, in general, uncover gaps in your processes that might have a wider impact than costs. You should plan to periodically assess the evolution of your infrastructure for any resources that may have been left unassigned and add this to your technical debt management operations.

3. Azure reservations and savings plans

These are a 1- or 3-year commitment to specific Microsoft Azure services or compute use. In exchange for this, significant cloud computing cost savings are granted. This is a very important area of cost governance, as it can amount to very large savings, even though it has practically zero impact on the carbon footprint. We recommend using reservations and savings plans once the right-sizing and cleanup processes have successfully started and periodically track and adjust their usage to match up to 100 percent of your requirements.

4. Database and application tuning

We often see customers migrate applications that rely on legacy databases. Sometimes, even cloud-native applications are developed using old data handling patterns, mostly because companies have a history that needs to be retained and cannot be wiped out by switching to a new database. But a large, stratified database that was doing well in an on-premises environment, has immediate drawbacks in the cloud—queries may be slow and resource-intensive, and data is uselessly exchanged and in large quantities which all adds up to the monthly bill. Optimizing the database so that the application is leaner and faster will also save you money by downsizing the original infrastructure and using fewer data and networking resources.

Having fully optimized your databases can, sometimes, not be enough. Your freshly migrated application came from one of the cloud migration patterns—lift and shift, refactor, rearchitect, and rebuild. Their cloud efficiency is higher when applications are designed for the cloud, as they will utilize all the flexibility and scaling of infrastructure as a service (IaaS) and platform as a service (PaaS) services, with the result of higher performance and lower costs. Investing some of the savings from your cloud cost reduction exercise will not only improve your application performance but in the end improve your overall cloud resource optimization.

What can you do to kickstart your cloud computing efficiency today:

Start your recurrent cloud cost management meeting this week. Make sure to invite all the stakeholders—the cloud and finops teams, your finance controller, and anyone in your company who is dealing with cloud costs directly or indirectly.

Search for quick wins (cleaning up, downsizing, optimizing logs or backups, and more) so that this will fund the upcoming wave of cost-saving tasks and the refactoring and innovation of your applications.

In conclusion, cloud computing efficiency is a crucial element for any company that is operating in the cloud. By adopting cloud spend optimization practices, businesses can reduce their overall cloud spend and carbon emissions, improve the performance of their applications, and finance future elements of innovation.

Source: microsoft.com

Continue reading

Cegal and Microsoft break down data silos and offer open collaboration with Microsoft Energy Data Services

The vast amount of applications and data in energy companies across isolated environments is exposing inefficiencies in collaboration. Together with Cegal Cetegra, Microsoft Energy Data Services will accelerate your journey toward seamless access to the data and applications you need for your day-to-day work by providing an easy-to-deploy managed service fully supported by Microsoft.

Cegal has been successfully collaborating with Microsoft and partners to help evaluate the new Microsoft Energy Data Services preview program, an enterprise-grade OSDU Data Platform powered by the cloud.

With Microsoft Energy Data Services, energy companies can leverage new cloud-based data management and collaboration capabilities provided by Cegal and Microsoft. 

Microsoft Energy Data Services is a data platform fully supported by Microsoft, that enables efficient data management, standardization, liberation, and consumption in energy exploration. The solution is a hyperscale data ecosystem that leverages the capabilities of the OSDU Data Platform and Microsoft's secure and trustworthy cloud services with our partners’ extensive domain expertise.

Cegal and Microsoft create collaborative cloud-based applications on Microsoft Energy Data Services

As an ISV and a specialist systems integrator for the energy industry, Cegal has always seen great value in removing data silos to free organizations from dated constraints that can lead to lower productivity. Opening for universal access to one of the most critical assets in any organization—knowingly data—is an obvious path to innovation. Integrating proprietary IP in existing workflows, contextualizing data through new AI-based routines, and integrating best-of-breed applications to create new and innovative solutions are critical steps toward more efficient and productive operations.

To achieve this goal, Cegal and Microsoft closely collaborated over several months, during which multiple relevant use cases have been thoroughly assessed and tested on the Microsoft Energy Data Services platform. From operating the platform to developing new solutions on top of it, Cegal had the opportunity to put in context a wide range of scenarios, making sure the experience was as extensive as possible yet realistic for the energy industry.

Cegal recently released Cetegra, a cloud-based platform offering to its users a modern, collaborative environment, uniquely designed to cater to energy industry–specific needs in terms of digitalization and data management. Deployed through a fully scalable, pay-as-you-go model, Cetegra leverages the strengths of the Microsoft Cloud and will provide full support for the Microsoft Energy Data Services platform. Cetegra with Microsoft Energy Data Services delivers a one-stop shop for all types of data and applications tightly linked to OSDU, offering energy players a comprehensive integration of their applications portfolio, also allowing them to develop and test new apps within the Cetegra Innovation Space without impacting existing business operations.

With Microsoft Energy Data Services entering preview, Cegal looks forward to delivering operational support for the platform. As a global specialist in digitalization, capitalizing on years of experience within the energy sector, Cegal represents the partner of choice to support and guide energy players as they navigate their digital transformation journey. 

How to work with Cegal Solutions on Microsoft Energy Data Services

Microsoft Energy Data Services is an enterprise-grade, fully-managed, OSDU Data Platform for the energy industry that is efficient, standardized, easy to deploy, and scalable for data management—for ingesting, aggregating, storing, searching, and retrieving data. The platform will provide the scale, security, privacy, and compliance that are expected by our enterprise customers. The platform offers out-of-the-box compatibility with Cegal Cetegra, a cloud-based platform offering a modern, collaborative environment, with data contained in Microsoft Energy Data Services.

Source: microsoft.com

Continue reading

Gain deeper insights with Microsoft Intelligent Data Platform

Data is foundational to any digital transformation strategy, yet many organizations struggle to understand what data they have, how to extract insights from it, and how to govern it—according to a 2022 Evanta survey, over half of Chief Data Officers (CDOs) struggle with siloed operating models when it comes to data sharing and democratization. According to Harvard Business Review, organizations that have embraced their data as a strategic asset have been better positioned to drive strategic differentiation and grow their revenue, but the fragmentation that exists today between databases, analytics, and governance is a common barrier to success.

The Microsoft Intelligent Data Platform, empowers organizations to invest more time creating value rather than integrating and managing their data estate. It integrates best-in-class solutions across Microsoft’s technology stack—breaking down data siloes and enabling organizations to extract real-time insights with the data governance needed to run the business safely.

“Shifting from a legacy on-premises data warehouse to Azure Synapse, supported by Datometry, has allowed us to virtualize the vast majority of our code without needing to repoint it. We have gained speed, performance, and agility while reducing costs and taken a big step forward in modernizing our enterprise data storage and management.”—Charlotte Lock, Director of Data, Digital & Loyalty at Co-op.

Added security and analytics features for the Azure data portfolio

The Microsoft Intelligent Data Platform features everything already available in the Azure Data portfolio (Azure Data Factory, Azure Data Explorer, SQL Server 2022, Azure SQL, Cosmos DB, and more.) as well as new products and features, including SQL Server 2022, Azure Synapse Link for SQL, Microsoft Purview Data Estate Insights, and Datamart in Power BI:

◉ SQL Server 2022, currently in preview, is the most secure database of the last decade. And is now integrated with Microsoft Purview and Azure Synapse Link, allowing for richer insights and governance from data at scale. SQL Server 2022 also comes with new features including AWS S3 support, Azure Active Directory authentication, Query Store hints, as well as security improvements compared to SQL Server 2019.

◉ Azure Synapse Link for SQL, now in preview, offers real-time analytics for data stored in Azure Synapse Analytics and Azure SQL. It is an automated system that allows for replication of data from transactional databases (both SQL Server 2022 and Azure SQL Database) to a dedicated SQL pool in Azure Synapse Analytics. Azure Synapse Link features near real-time analytics, low-code/no-code solutions for replicating data, as well as minimal operational impact on source systems.

◉ Purview Data Estate Insights is an application that provides Chief Data Officers and other strategic leaders with a summary of their data estate and the risk associated with that data. Purview provides insights on data stewardship, inventory, curation, and governance through automatically generated reports which can be easily shared with stakeholders.

◉ Lastly, Datamart in Power BI allows analysts to access richer insights from their data sets through data marts. Datamarts are self-service analytic solutions that help to bridge the gap between business users through a simple and optionally no-code experience. With datamarts, you can easily ingest and prepare data, add business semantics to data, manage and govern data, as well as build and share reports.

Real-world applications for businesses through real-time data

Let’s explore one example of how the Microsoft Intelligent Data Platform helped navigate supply chain issues:

Many operations companies conduct daily batch runs, where they must manually track their inventory levels and input the data at least once a day. With this method, these organizations cannot accurately predict how much product to sell and must err on the side of selling less to avoid running out of inventory. In times when supply chains are uncertain, this means companies miss out on even more sales.

With the Microsoft Intelligent Data Platform, companies can get real-time information on current inventory levels, rather than a daily report. They can also extract AI-driven insights based on demand spikes, shipping delays, and factory status that predict how many units will be available in a week’s time. This information is supported by the upgraded SQL Server 2022 as well as Azure Synapse Link for SQL server, which allows for more on-premises data to be extended to the cloud, analyzed, and used for decision making.

But what about using data for customer-facing solutions? The Microsoft Intelligent Data platform leverages the CosmosDB platform, providing consumers with recommendations for the best product based on real-time availability of units, delivery time, and compatibility with their needs. Consumers also have access to a support number powered by Power Virtual Agents; through Conversational AI, consumers can get intelligent updates on their order status so they can get the information they need quickly.

Source: microsoft.com

Continue reading

Azure Data Explorer: Log and telemetry analytics benchmark

Azure Data Explorer (ADX), a component of Azure Synapse Analytics, is a highly scalable analytics service optimized for structured, semi-structured, and unstructured data. It provides users with an interactive query experience that unlocks insights from the ocean of ever-growing log and telemetry data. It is the perfect service to analyze high volumes of fresh and historical data in the cloud by using SQL or the Kusto Query Language (KQL), a powerful and user-friendly query language.

Azure Data Explorer is a key enabler for Microsoft’s own digital transformation. Virtually all Microsoft products and services use ADX in one way or another; this includes troubleshooting, diagnosis, monitoring, machine learning, and as a data platform for Azure services such as Azure Monitor, PlayFab, Sentinel, Microsoft 365 Defender, and many others. Microsoft’s customers and partners are using ADX for a large variety of scenarios from fleet management, manufacturing, security analytics solutions, package tracking and logistics, IoT device monitoring, financial transaction monitoring, and many other scenarios. Over the last years, the service has seen phenomenal growth and is now running on millions of Azure virtual machine cores.

Last year, the third generation of the Kusto engine (EngineV3) was released and is currently offered as a transparent, in-place upgrade to all users not already using the latest version. The new engine features a completely new implementation of the storage, cache, and query execution layers. As a result, performance has doubled or more in many mission-critical workloads.

Superior performance and cost-efficiency with Azure Data Explorer

To better help our users assess the performance of the new engine and cost advantages of ADX, we looked for an existing telemetry and logs benchmark that has the workload characteristics common to what we see with our users:

1. Telemetry tables that contain structured, semi-structured, and unstructured data types.

2. Records in the hundreds of billions to test massive scale.

3. Queries that represent common diagnostic and monitoring scenarios.

As we did not find an existing benchmark to meet these needs, we collaborated with and sponsored GigaOm to create and run one. The new logs and telemetry benchmark is publicly available in this GitHub repo. This repository includes a data generator to generate datasets of 1GB, 1TB, and 100TB, as well as a set of 19 queries and a test driver to execute the benchmark.

The results, now available in the GigaOm report, show that Azure Data Explorer provides superior performance at a significantly lower cost in both single and high-concurrency scenarios. For example, the following chart taken from the report displays the results of executing the benchmark while simulating 50 concurrent users: 

Source: microsoft.com

Continue reading

Introducing the Microsoft Intelligent Data Platform

We are moving to a world where every application needs to be intelligent and adaptive to real-time model learning. As businesses build modern data capabilities, they must make decisions at the speed of human thought. Developers are challenged by this, given the huge silos that exist between databases and analytic products, and the complexity of a fragmented data estate can hamper the speed of agility and innovation. Data engineers, data scientists, and business analysts struggle with the complexity of making data integration, data warehousing, machine learning operations (MLOps), and business intelligence (BI) work together. What is needed is a consistent data ecosystem.

To help address the fragmentation that exists today between databases, analytics and governance, and enable organizations to unlock these new capabilities, we shared several exciting announcements today at Microsoft Build that demonstrate our continued innovation and investment in the data products our customers have come to know and trust, which will enable our customers to achieve the kind of sustained agility that allows them to pivot and adapt in real-time, add layers of intelligence to their applications, unlock fast and predictive insights, and govern their data—wherever it resides.

Accelerate innovation

Today, we unveiled the new Microsoft Intelligent Data Platform, the leading cloud data platform that fully integrates databases, analytics, and governance. This seamless data platform empowers organizations to invest more time creating value rather than integrating and managing their data estate.

Furthering our mission of integration, Azure Synapse Link for SQL removes data movement barriers, providing a seamless data pipeline to Azure Synapse Analytics, and enables near-real-time analytics for SQL Server 2022 and Azure SQL Database. Once Azure Synapse Link transfers data to Azure Synapse Analytics, data can be used for advanced analytics with no performance impact on transactional workloads. Over 10 million Azure SQL databases globally can now leverage this capability.

Achieve agility

Through continued investments in databases and analytics, we are empowering customers to achieve agility in new ways.

Now in preview SQL Server 2022 is our most Azure-enabled release yet, with continued innovation across performance, security, and availability. By connecting SQL server to Azure through seamless disaster recovery to Azure SQL Managed Instance, SQL Server 2022 provides true resilience. The latest Azure Arc innovation is here with the Business Critical tier of Azure Arc-enabled SQL Managed Instance now generally available—helping customers run the most demanding mission-critical workloads in hybrid and multicloud environments. We continue to invest in bringing the best developer experience in any cloud with Azure SQL Database releasing new features to help simplify and expedite application development and reduce time to market for developers. In addition, the ledger feature in Azure SQL Database, now generally available, eliminates the additional cost and complexity of decentralized blockchain technology while providing the benefits of blockchain in a fully managed and familiar SQL environment.

New innovation in cloud-native NoSQL and open-source databases give developers the freedom to build on their terms. Azure Cosmos DB has an enhanced 30-day trial experience now generally available and has introduced new burst capacity and elasticity features in preview. Application traffic spikes don’t have to equal spikes in costs. New burst capacity and elasticity features ensure applications deliver high performance during peak times while still remaining cost-effective. The Azure Database for MySQL Flexible Server Memory-Optimized service tier is now the improved "Business Critical" tier for high performance transactional or analytical applications with a 1.5x performance improvement over Single Server with faster failover time to standby.

When it comes to analytics, Azure Synapse is simply unmatched with more meaningful integrations that enable existing Synapse customers to get even more value from their Microsoft 365 data. Microsoft Graph Data Connect empowers customers to securely export their Microsoft 365 data estate, and it’s only available on Microsoft Azure. This enables every customer to unlock new actionable business insights with the employee and customer collaboration data that comes from Microsoft 365.

Customers like KPMG use Microsoft Intelligent Data Platform integrations to power their KPMG Digital Gateway bringing together a wealth of tools to help customers tackle regulatory change, turn data into value, and streamline compliance and planning while enabling effective collaboration across tax, legal, and finance departments. KPMG Digital Gateway puts its investments in machine learning, data analytics, powerful visualizations, and AI technologies all in one place. Client data is provided once and leveraged across applications, saving time and money.

To continue creating simple frictionless experiences, we announced the preview of datamart in Power BI, a new Power BI Premium self-service capability that enables users to uncover actionable insights through their own data sets. This out-of-the-box feature empowers developers and business analysts to build a datamart that can be centrally governed and managed for workloads up to half a terabyte—accelerating time to insight while alleviating demands on IT. This new feature brings the power of data warehousing and puts it in the hands of individual Power BI developers and analysts, helping you uncover more insights and drive digital transformation at every level of a business.

Build on a trusted platform

Meeting data privacy and governance standards cannot be an afterthought. When governance is not deeply integrated where data lives, it is nearly impossible to meet regulatory requirements. 

Data governance has become top of mind for almost every organization where data is fluidly moving across hybrid and multicloud environments. This makes it increasingly important to map the lineage of data. Dynamic Lineage for Azure SQL Databases in Microsoft Purview is currently in preview to further enrich the Microsoft Purview Data Map with details from actual runs of SQL stored procedures in Azure SQL Databases for customers to govern their data across hybrid and multicloud environments. We are also excited to announce that the Microsoft Purview Data Estate Insights will be generally available in the coming months.

As we look beyond the horizon, machine learning and AI capabilities will be pivotal in harnessing the power of data in new ways. Azure Machine Learning now offers a Responsible AI dashboard in preview, making it possible for customers to implement it more easily by debugging machine learning models and making informed data-driven decisions. The dashboard brings together capabilities such as data explorer, model interpretability, error analysis, counterfactual, and causal inference analysis in a single view. In addition, Azure Machine Learning now offers a Responsible AI scorecard in preview to summarize model performance and insights so that all stakeholders can easily participate in compliance reviews.

The opportunity to accelerate innovation in your business and achieve agility across all your data is substantial. Now is the time to realize its limitless potential. We look forward to seeing what you can do with it.

Source: microsoft.com

Continue reading

Optimize your cloud investment with Azure Reservations

Continuous cost optimization can take place at all stages of an Azure workload’s lifecycle, but your Azure subscription provides a very effective benefit to further optimize your investment when you are ready to deploy that workload.

For cloud workloads with consistent resource usage, you can buy reserved instances at a significant discount and reduce your workload costs by up to 72 percent compared to pay-as-you-go prices. Azure Reservations can be obtained by committing to one-year or three-year plans for virtual machines, Azure Blob storage or Azure Data Lake Storage Gen2, SQL Database compute capacity, Azure Cosmos DB throughput, and other Azure resources.

When you can predict and commit to needed capacity, it gives us visibility into your resource requirements in advance, allowing us to be more efficient in our operations. We can then pass the savings on to you. This benefit applies to both Windows and Linux virtual machines (VMs).

In addition, you now can combine the cost savings of reserved instances with the added Azure Hybrid Benefit when running on-premises and Azure workloads to save up to 80 percent over pay-as-you-go pricing.

How to get your reservation

A reservation discount only applies to resources associated with Enterprise Agreement, Microsoft Customer Agreement, Cloud Solution Provider (CSP), or subscriptions with pay-as-you-go rates. These are billing discounts (paid upfront or monthly) and do not affect the runtime state of your resources. And do not worry, you will not pay any extra fees when you choose to pay monthly.

To determine which reservation to purchase, analyze your usage data in the Azure portal, or use reservation recommendations available in Azure Advisor (VMs only), the Cost Manage Power BI app, or the Reservation Recommendations REST API.

Reservation purchase recommendations are calculated by analyzing your hourly usage data over the last seven, 30, and 60 days.

Simple and flexible

You can purchase Azure Reserved VM Instances in three easy steps—just specify your Azure region, virtual machine type, and term (one year or three years)—that's it.

Here is how it works: Discounts are generally applied to the resource usage matching the attributes you select when you buy the reservation. Attributes include the scope where the matching VMs, SQL databases, Azure Cosmos DB, or other resources run. Attributes include the SKU, regions (where applicable), and scope. Reservation scope selects where the reservation savings apply. You can scope a reservation to a subscription or resource group. When you scope the reservation to a resource group, reservation discounts apply only to the resource group—not the entire subscription.

You can manage reservations for Azure resources including updating the scope to apply reservations to a different subscription, changing who can manage the reservation, splitting a reservation into smaller parts, or changing instance size. Enhanced data for reservation costs and usage is available for Enterprise Agreement (EA) and Microsoft Customer Agreement (MCA) usage in Azure Cost Management and Billing. Those same customers can view amortized cost data for reservations and use that data to chargeback the monetary value for a subscription, resource group, or resource.

Capacity on demand

The ability for you to access compute capacity with service-level agreements, and ahead of actual VM deployments, is important to ensure the availability of mission-critical applications running on Azure. On-demand capacity reservations, now in preview, enable you to reserve compute capacity for one or more virtual machine size(s) in an Azure region or availability zone for any length of time. You can create and cancel an on-demand capacity reservation at any time, no commitment is required.

You also can exchange a reservation for another reservation of the same type or refund a reservation, up to $50,000 USD in a 12-month rolling window if you no longer need it, or cancel a reserved instance at any time and return the remaining months to Microsoft.

Source: microsoft.com

Continue reading

Unleashing the true potential of 5G with cloud networks

For many, the intersection of 5G wireless and the Internet offers very little by way of new information, just from a quick, surface-level glance. However, things look very different when considered in light of the 5G New Radio (NR) standard.

For example, consider a new technology in the 5G NR called URLLC—which is short for ultra-reliable low-latency communications. Designed for mission-critical and latency-sensitive Internet services, URLLC is a configuration and technology feature that enables a new class of applications and services. These services include things such as remote control of robots; thin, tether-less augmented reality headsets, smart car collision avoidance systems, traffic control on roads, and entertainment services such as fast-action multi-player multi-region gaming.

The delay specification delivered by URLLC can be anywhere from one to four msec on-air latency, well within the required hard limit of the latency required by these applications. However, in many ways, this delay specification is simply not sufficient in light of the fact that many of these latency-sensitive applications only make sense in the context of the Internet. Interacting with users, devices, and cloud services over long distances forces one to think about the end-to-end latency characteristics that include the Internet.

Stretching the reach of low-latency applications

No one network can encompass all end-users and cloud services, so every network operator ultimately depends on the Internet at some point. Based on the locations of the source and destination, Internet latencies can vary anywhere from a single order to several orders of magnitude more than on-air 5G wireless latencies. This high latency can effectively remove all the benefits of the low-latency properties offered via the 5G network. As a result, the scope of the services that people enthusiastically look forward to gets reduced considerably.

Beyond URLLC, 5G traffic in general (voice and high-resolution video calls) will demand performance and reliability from both the wireless and Internet paths, even more so than traditional web and enterprise traffic.

Operators spend a lot of money to manage and maintain their networks and peering relationships, but so does Microsoft. The question then is, why are two massive industries doing the same thing? Because both parties move packets around, doesn’t it make more sense for them to collaborate?

Here, the well-managed, reliable, and performant Azure network should be thought of as the backbone that operators trust. With this shift in thinking will come all the advantages of innovation that IT companies like Microsoft are rapidly bringing in.

For example, large operators that run extensive national backbones can get tremendous benefit from extending their wide-area network (WAN) with Azure’s WAN. In this example, a 5G network that spans both will allow 5G devices to more effectively reach cloud services deployed on Azure datacenters. This includes first-party services such as, Xbox cloud gaming, as well as third-party services that Azure customers run. Smaller operators (or new operators) that do not have their own national backbones can save resources including time, human capital, and money, and instead leverage Azure’s extensive investment to build a unique 5G network on top of something that has already been proven reliable.

Azure’s planet-scale WAN

Azure maintains a massive WAN with significant capacity and one that is continuously growing. We have over 175,000 miles of lit fiber optic and undersea cable systems. This connectivity covers close to 200 network points of presence (PoPs) over 60 regions, across 140 countries.

Azure’s network is connected to many thousands of ISPs and other networks with significant peering capacity. Our global network is well-provisioned, with redundant fiber paths that can handle multiple simultaneous failures, it also has massive reserve capacity in unlit dark fiber. These optical fibers are fully owned or leased by Microsoft, and all traffic between and among Azure datacenters within a region or across regions is automatically encrypted at the physical layer.

This combination of redundant capacity to handle failures, dark capacity for significant growth, and research advancements being made in increasing transmission speeds means that we have a massive amount of spare capacity to serve 5G traffic to a broad array of new operators.

Challenges related to carrying 5G traffic over a cloud WAN

Figure 1 illustrates how packets can move from one client (in 5G terminology, a “User Equipment” (UE)) to another over the Azure cloud network, with extremely low latency.

Figure 1: Unified platform.

With 5G, there is an intense amount of pressure on traditional metrics that include latency, jitter, throughput, reachability, and loss, by which transport quality is typically judged. With on-air 5G latencies reaching close to the sub-ms range, wired transport latency will likely dominate end-to-end performance. However, with on-air 5G throughputs in the tens of Gbps range in Enhanced Mobile Broadband (eMBB) mode, just a small number of UEs could overwhelm a single peering link that today would typically have a capacity of tens to hundred Gbps.

Beefing up capacity at the peering surface area and on backbone links that support them remains an expensive endeavor. Finding efficient paths through the maze of WAN links is also crucial to achieving business success. Outages are a fact of life for any cloud network, but 5G experiences will be more sensitive to packet loss and reachability than traditional web and enterprise traffic.

Reliability will need to be increased. This will happen by guarding against peering outages and overload, and by verifying configuration changes. The performance will also need to be increased. This includes supporting streaming audio and video services by actively reducing jitter and access queuing delays. Additional factors such as cost, safety, reachability, and regulatory and business policies also must be addressed.

Further, there will be a need to orchestrate wired transport for 5G deployments. 5G networks will deploy many network functions in a distributed fashion. Rigorous needs around scale-out and fault tolerance will make these deployments more complex than typical enterprise application deployments. Orchestrating cloud network capabilities, such as Virtual Networks, Virtual Network Peering, Virtual WAN, and private endpoints, all while meeting performance and policy constraints (business and regulatory), represents a new challenge.

Making Azure WAN great for 5G traffic

For many years, Microsoft researchers and engineers have been working on a hybrid-global traffic orchestrator for routing network packets across Azure’s WAN. Our orchestrator takes control away from classic Internet protocols and instead moves that control into software that we build and control for 5G traffic. We place the 5G flows that demand high performance on low-latency, high bandwidth paths to and from the Internet. Network flows that are cost-sensitive are instead routed through cheaper paths.

In effect, we have developed a fast-forwarding mechanism to build a 5G overlay on our existing WAN, thereby supporting a variety of 5G network slices with different wired transport properties, while avoiding interference with the operation of the underlying enterprise cloud network.

We have also extended our state-of-the-art network verification capability to cover complex network topologies by modeling Virtual WAN, Virtual Networks, and other network function virtualizations (NFVs), as well as modeling reachability using formal methods. Using fast solvers, we can verify reachability constraints on customer topologies, at deployment time or when undergoing a config change.

We have applied machine learning to predict the impact of peering link outages and congestion mitigation strategies and use the data to improve the availability of the WAN peering surface area.

Our expertise in optimization algorithms has been shown to ultimately reduce cloud networking spend. Techniques like these will be invaluable in carving out 5G paths on the overlay that are cost-efficient, but still meet the performance needs of every network slice.

Figure 2: 5G WAN technical architecture. This is how we envision the different technical components will need to come together to serve 5G operators.

The significant upside for operators

To reiterate, Microsoft is heavily invested in running a well-managed, always-available global network. We have been incorporating multiple groundbreaking technologies, including scalable optimization, formal verification of routing policies, machine learning, and AI. We envision operators to not only be able to use our WAN to transfer 5G packets, with low latency, but also to benefit from multiple network services such as DDoS protection, firewalls, traffic accelerators, connection analytics, load balancers, and rate limiters, many of which we use in running existing Azure network workloads.

At Microsoft, we bring the full power of research and engineering leadership into our networks, rapidly incorporating innovation and new features to provide reliable, low-latency, low-cost service. In turn, this effort will open up the significant potential of next-generation services and applications as envisioned by the community at large. It is no understatement to say that collaboration between operators and Azure is key to unleashing the true power of 5G.

Source: microsoft.com

Continue reading