Announcing a new OpenAI feature for developers on Azure

We are thrilled to announce the launch of OpenAI’s latest model on Azure. This new model, officially named GPT-4o-2024-08-06, brings innovative features designed to elevate developer experiences on Azure. Specifically, the new model focuses on enhancing productivity through Structured Outputs, like JSON Schemas, for the new GPT-4o and GPT-4o mini models.

A focus on Structured Outputs

GPT-4o was first announced in May 2024, as OpenAI’s new multimodal model, followed by GPT-4o mini in July 2024. Today’s version is designed with a specific use case in mind: simplifying the process of generating well-defined, structured outputs from AI models. This feature is particularly valuable for developers who need to validate and format AI outputs into structures like JSON Schemas. Developers often face challenges validating and formatting AI outputs into well-defined structures like JSON Schemas.  

Structured Outputs addresses this by allowing developers to specify the desired output format directly from the AI model. This feature enables developers to define a JSON Schema for text outputs, simplifying the process of generating data payloads that can seamlessly integrate with other systems or enhance user experiences. 

Use cases for JSON

JSON Schema is essential for defining the structure and constraints of JSON documents, ensuring they follow specific formats with mandatory properties and value types. It enhances data understandability through semantic annotation and serves as a domain-specific language for optimized application requirements. Development teams use JSON Schema to maintain consistency across platforms, drive model-driven UI constraints, and automatically generate user interfaces. It aids in data serialization, security testing, and partial validation in technical scenarios. JSON Schema also supports automated testing, Schema inference, and machine-readable web profiles, improving data interoperability. It standardizes validation interfaces and reporting, handles external validation, and ensures data consistency within and across documents. It can also help with customer support and how to communicate in a timely manner. 

Two flavors of Structured Outputs

Structured Outputs is available in two forms: 

1. User-defined JSON Schema: This option allows developers to specify the exact JSON Schema they want the AI to follow, supported by both GPT-4o-2024-08-06 and GPT-4o-mini-2024-07-18.

2. More Accurate Tool Output (“Strict Mode”): This limited version lets developers define specific function signatures for tool use, supported by all models that support function calling, including GPT-3.5 Turbo, GPT-4, GPT-4 Turbo, and GPT-4o models from June 2023 onwards. 

Technical guidance on using Structured Outputs

To help you get started with Structured Outputs, we recommend the following approach. 

Getting started with Structured Outputs 

1. Define Your JSON Schema: Determine the structure you want your AI outputs to follow. This can include required fields, data types, and other constraints. 

2. Configure the AI model: Use the Structured Outputs feature to specify your JSON Schema within the API call. This ensures that the AI output adheres to your defined structure. 

3. Integration and testing: Integrate the output into your application or system, and test thoroughly to ensure compliance with your JSON Schema. 

Example use case: Customer support automation

Imagine you’re developing a customer support chatbot that needs to generate responses in a specific format for logging and analytics. By using Structured Outputs, you can define a JSON Schema that includes fields like responseText, intent, confidenceScore, and timestamp. This ensures that every response generated by the chatbot is formatted correctly, making it easier to log, analyze, and act upon. 

Example API call

Here’s an example API call to illustrate how to use Structured Outputs:

{

  "model": "gpt-4o-2024-08-06",

  "prompt": "Generate a customer support response",

  "structured_output": {

    "schema": {

      "type": "object",

      "properties": {

        "responseText": { "type": "string" },

        "intent": { "type": "string" },

        "confidenceScore": { "type": "number" },

        "timestamp": { "type": "string", "format": "date-time" }

      },

      "required": ["responseText", "intent", "confidenceScore", "timestamp"]

    }

  }

}

Pricing

We will make pricing for this feature available soon. Please bookmark the Azure OpenAI Service pricing page. 

Learn more about the future of AI

We’ve been rolling out several new models recently, and we understand it can be a lot to keep up with. This flurry of activity is all about empowering developer innovation. Each new model brings unique capabilities and enhancements, helping you build even more powerful and versatile applications. 

The launch of this new model feature for GPT-4o and GPT-4o mini marks a significant milestone in our ongoing efforts to push the boundaries of AI capabilities. We’re excited to see how developers will leverage these new features to create innovative and impactful applications.

Source: microsoft.com

Continue reading

10 ways to impact business velocity through Azure OpenAI Service

The phrase, “time is money,” is commonly attributed to Benjamin Franklin, who first used it in his essay “Advice to a Young Tradesman,” published in 1748. Franklin was addressing the economic value of time, a concept increasingly relevant when discussing AI’s impact on business today. AI is adept at processing and analyzing troves of data much faster than a human brain—enabling quicker, more informed decision-making. Leaders who embrace AI now and take action to understand it, experiment with it, and envision how it can solve hard problems are going to run companies that thrive in an AI world. From automating routine tasks to providing deep insights through data analysis, AI technologies are enabling businesses to make quicker, more informed decisions, driving growth and competitive advantage.

10 ways AI can turbocharge business efficiency

1. Automating repetitive tasks: AI can handle mundane and repetitive tasks such as data entry, scheduling, and email sorting.
2. Real-time data analysis: AI algorithms can analyze vast amounts of data in real-time, providing immediate insights and allowing businesses to make faster, data-driven decisions.
3. Predictive analytics: AI can forecast trends and behaviors based on historical data, enabling companies to anticipate market changes and customer needs more rapidly.
4. Customer support chatbots: AI-powered chatbots provide instant customer service, addressing inquiries and resolving issues without human intervention.
5. Supply chain optimization: AI can predict demand, optimize inventory levels, and streamline logistics.
6. Fraud detection: AI systems can quickly detect and respond to fraudulent activities by analyzing transaction patterns and identifying anomalies in real-time.
7. Personalized marketing: AI can tailor marketing campaigns to individual preferences and behaviors, increasing engagement and conversion rates more swiftly.
8. Enhanced recruitment processes: AI can screen resumes, conduct initial interviews, and identify the best candidates faster than traditional methods.
9. Process automation: Robotic Process Automation (RPA) driven by AI can execute business processes faster and with fewer errors, from financial transactions to regulatory compliance.
10. Product development: AI accelerates product development cycles by simulating different design scenarios, optimizing prototypes, and predicting performance outcomes.

Below, we look at three Microsoft customers who used Azure OpenAI Service to accelerate the speed at which they do business.

◉ Akbank empowers staff to search through 10,000 records in seconds with Azure OpenAI Service

Akbank, one of Türkiye’s largest banks, has significantly improved its customer support operations by integrating Azure OpenAI Service. Whereas their customer representatives previously had to search through a hefty 10,000-article knowledge base in hopes of finding correct responses, they now interact with an AI chatbot that generates correct answers 90% of the time. This integration saves three minutes per interaction, thus enhancing both the quality and accuracy of the support provided. Akbank has also incorporated proactive suggestions into the chatbot, enabling staff to get faster responses and continually improve customer support.

◉ Serving customers 78% faster: VOCALLS’ voicebots supercharge call handing with Azure OpenAI

VOCALLS, a Prague and London-based telecommunications company, leverages Microsoft Azure AI technologies to support its customer service with AI-powered voicebots. Specializing in conversational AI solutions, VOCALLS automates over 50 million interactions annually, improving customer experiences for companies like Estafeta. Estafeta, a logistics pioneer in Latin America, saw a 78% reduction in average handling time and a 120% increase in answered calls after deploying VOCALLS’ voicebot, Beatriz. This voicebot provides immediate support, eliminating wait times and boosting customer satisfaction scores.

◉ RepsMate empowers businesses to create super agents by leveraging Azure’s AI superpowers

As a member of the Microsoft for Startups program, RepsMate has leveraged Microsoft’s networks, support, and the Azure Marketplace to gain traction in Eastern Europe. RepsMate’s solution, driven by AI and data analysis, has led to significant efficiency gains, reducing average handling times by 12%, decreasing chat durations by 20 to 30%, and increasing first-call resolution rates by 5 to 10%. Additionally, RepsMate has automated up to 25% of interactions with predefined answers, enhancing both speed and accuracy. Additionally, their strategic use of Microsoft’s full suite of technologies, has allowed RepsMate to train large datasets faster and avoid unnecessary costs, further enhancing efficiency.

A faster more efficient future

Examples like those of Akbank, VOCALLS, and RepsMate demonstrate the impact of AI on business speed and productivity. By integrating AI solutions like Microsoft Azure OpenAI Service, companies can achieve faster decision-making, optimize their processes, and better support customer experiences. As businesses continue to adopt and innovate with AI, they’re in a better position to meet the demands of a rapidly evolving market.

Our commitment to responsible AI

Organizations across industries are leveraging Microsoft Azure OpenAI Service and Copilot services and capabilities to drive growth, increase productivity, and create value-added experiences. From advancing medical breakthroughs to streamlining manufacturing operations, our customers trust that their data is protected by robust privacy protections and data governance practices. As our customers continue to expand their use of our AI solutions, they can be confident that their valuable data is safeguarded by industry-leading data governance and privacy practices in the most trusted cloud on the market today.

At Microsoft, we have a long-standing practice of protecting our customers’ information. Our approach to responsible AI is built on a foundation of privacy, and we remain dedicated to upholding core values of privacy, security, and safety in all our generative AI products and solutions.

Source: microsoft.com

Continue reading

Leverage AI to simplify CSRD reporting

Organizations around the world are navigating complex reporting frameworks to meet sustainability goals. For companies working in—or with—the European Union, the Corporate Sustainability Reporting Directive (CSRD) is a sweeping set of requirements to provide non-financial public disclosures on environmental, social, and governance (ESG) topics. CSRD rules began rolling out in 2024, requiring over 11,000 companies to disclose non-financial information. An increase of an additional 50,000 companies that are incorporated, listed, or doing business in the European Union is estimated. As companies are working to comply, allocating the time and resources is a challenge. Microsoft is developing solutions to address the diverse reporting needs of our customers and investing in our partners to create a variety of options that organizations can engage.

To address this growing need, Microsoft Cloud for Sustainability data solutions in Microsoft Fabric (preview) can help organizations take sustainability data in any format, organize, and normalize the data for sustainability regulatory reporting. This quantitative data reporting approach is complemented by a joint solution from Accenture and Avanade that leverages generative AI to provide qualitative insights. This enables organizations to manage workflows associated with multiple sustainability reporting frameworks globally, including CSRD, Global Report Initiative (GRI), and the International Sustainability Standards Board (ISSB). Organizations can optimize both quantitative data from Microsoft Cloud for Sustainability solutions and qualitative data with Accenture and Avanade’s generative AI-powered solution. 

The solution integrates seamlessly with Microsoft Sustainability Manager, offering a comprehensive platform for managing sustainability key performance indicators (KPIs) across different frameworks. The solution’s collaborative features, generative AI-enhanced insights, and streamlined data integration can help organizations simplify compliance-related processes. Meanwhile, the improved richness of its sustainability reporting enables organizations to take more effective actions to achieve their ESG targets.

Navigate the complexities of reporting frameworks

All ESG reporting frameworks carry their own complexities. However, the CSRD has been top of mind in 2024 for organizations in and outside of the European Union as they begin to gather, analyze, and report the required data. This is no simple task—the CSRD encompasses 12 standards and 82 reporting requirements, which amounts to approximately 500 KPIs and over 10,000 underlying data points. In addition to managing this large array of ESG data, companies face other significant challenges associated with CSRD compliance and data management including reporting on the entire value chain versus only on their own organization. The evolving nature of sustainability criteria and metrics further complicates the reporting process. 

Navigating the intricacies of a multitude of reporting frameworks and intricate regulations necessitates extensive data gathering and assimilation. The process of ESG reporting often engages various departments and resources within an organization, introducing its own set of risks and costs. The more manual a process, the more opportunities there are for errors to occur, and the complexities of each reporting framework require time, diligence, and accuracy. A proficient solution can help simplify the process and effectively aid in the generation of accurate reports with fewer resources required. 

Accenture and Avanade’s generative AI-powered solution allows users to select from different reporting frameworks and adapts to the specific requirements of the chosen framework, displaying relevant categories and reporting structures. Users can access the breadth and depth of their data and translate it into the necessary reporting frameworks. This flexibility is crucial for organizations that are subject to multiple reporting obligations or need to adhere to international standards beyond local mandates. 

Streamlining data management using Microsoft Azure OpenAI Service

Accenture and Avanade’s solution addresses the dual challenges of qualitative and quantitative data in sustainability reporting, utilizing Azure OpenAI for enhancing its reporting capabilities, especially for qualitative data input. Using Azure OpenAI to create an AI assistant, Accenture and Avanade’s solution offers a unified platform for sustainability reporting, simplifying the process of compiling CSRD reports, making it easier for users to interact with data and generate reports quickly.

◉ Qualitative data management: Leveraging Azure OpenAI integration, Accenture and Avanade’s solution assists in drafting responses to qualitative questions, such as detailing a company’s sustainability policies, practices, and goals. This AI assistant ensures that responses are not only in keeping with reporting standards but also reflective of best practices and forward-thinking sustainability strategies.

◉ Quantitative data management: The integration of Microsoft Sustainability Manager allows for the automatic import of calculated quantitative metrics. This means that insights or recommendations provided by Accenture and Avanade’s solution is informed by the data in the user’s ESG platform. The AI assistant enables identification of trends and patterns in both qualitative and quantitative data sources, aiding in a more holistic analysis. 

An enhanced collaborative workflow

One of the biggest pain points that companies have related to ESG reporting is managing the approval workflow with multiple process and approval steps. It’s important to have a trail of accountability, which depending on organization size, can exist across several departments and users. Accenture and Avanade’s solution enables organizations to assign responsibility and accountability, thereby streamlining the process of preparing the report and its associated approval processes. 

Importantly, team members can work on the same sections of the report simultaneously and the platform maintains comprehensive audit trails of all changes made to the report. This transparency is vital for accountability, enabling team leaders to monitor progress and ensure that all contributions align. 

Source: microsoft.com

Continue reading

6 findings from IoT Signals report: Manufacturers prepare their shop floor for AI

Manufacturers are embracing AI to deliver a new level of automation, optimization, and innovation. To unlock the full potential of AI on the shop floor, organizations are testing and investigating technologies and paradigms that empower them to leverage their data more effectively.

Microsoft, in partnership with IoT Analytics market research firm, conducted a global survey of manufacturers to gain insight into how they are seizing the AI opportunity while navigating key industry challenges. We asked manufacturers about their current priorities and future visions, their adoption of modern technologies and paradigms, and the benefits they expect from those technologies 

In this report, we share the key findings from the survey, to show how manufacturing enterprises are preparing their shopfloors for AI to make them secure, scalable, and automated and how they are adopting advanced technologies such as centralized device management, software containerization at the edge, and unified industrial data operations to accelerate that process.

Six findings from manufacturers preparing their shop floor for AI

1. Scale matters the most in the era of AI

Scalability was the main concern for 72% of survey respondents, who highlighted this paradigm as crucial for their factory’s future. Scalability came first, followed by automation and serviceability. These paradigms ensure that factories can efficiently expand with demand, optimize with minimal manual decision making, and maintain high uptime through easy troubleshooting and maintenance. 

What does scale look like for industrial environments? 

Manufacturers face the challenges of keeping up with the changing demands of the market, the regulations, and the competition. They also recognize the potential of AI to transform their operations, optimize their processes, and enhance their products. But they don’t have the luxury of spending months or years on deploying and scaling solutions across their plants. Manufacturers need a faster way to move, a smarter way to manage, and a more flexible way to adapt. That’s why we have introduced a new approach—the adaptive cloud approach. 

2. Cybersecurity and data management are top of mind right now 

Security risks and data handling difficulties pose serious problems, with 58% of respondents seeing cybersecurity as a severe issue and 49% seeing data management as a severe issue. These concerns are motivating customers to improve network security and ensure data is reliable and accessible for decision-making. 

What does security look like for industrial environments? 

Security and data protection are critical for the manufacturing sector, as the sector faces increasing regulatory standards and cyber threats. Manufacturers need to secure existing devices, and plan during device refresh to choose devices that meet industry security standards, will enable them to more easily comply with regulatory standards, and provide security to defend from the latest security threats.

3. Device management is critical for security and data handling 

Device management’s value is evolving beyond updates and device health monitoring to also address security risks and data flow management. The survey data supported this trend, with 68% of respondents noting that the security monitoring aspect of device management was very or extremely important to their organization and 59% of respondents highlighting data management as the second most important aspect of device management. 

Why is centralized device management important? 

Centralized device management is vital for ensuring the performance and security of operations in a factory setting. It helps to keep devices secure and functioning optimally, which contributes to the overall efficiency and productivity of a manufacturing environment. Effective management also enables better oversight and control over the factory processes, improving operational reliability and supporting scalability and adaptability in a dynamic industrial landscape.

4. Containerized workloads are coming to the shop floor 

The adoption of containerized software on the shop floor is rising, with 85% of survey respondents already utilizing this technology. This shift towards containerization at the edge signifies a move to improve operational efficiency, system stability, and security. 55% of respondents indicated that containerized software could significantly or extremely mitigate reliability and uptime challenges, while 53% indicated it could do the same for cybersecurity challenges.

What is containerized software? 

Software containerization enables consistent and repeatable development and deployment of solutions across different environments, in the cloud and in factory. Containerization of OT software is essential for the AI-powered factory of the future, as it enables seamless technology deployment in scalable, serviceable, and automated factories. Kubernetes automates the scaling and management of containerized applications, saving time and resources for manufacturers.

5. Industrial data operations optimize OT data management

Companies want to combine information technology (IT) and operational technology (OT) systems for context driven decision making. 52% of respondents indicated that having a combined IT and OT data platform was very or extremely important for their company. Industrial data operations enhance the integration of IT and OT data by improving data flow, quality and value; therefore, 87% of companies have already adopted industrial data operations technology in some form or are planning to do so.

What are industrial data operations? 

Industrial data operations delivers data in a reliable, real-time manner for optimizing factories and plants. Industrial data operations manages and unifies data from various sources, facilitates seamless integration of information, and ensures data is accessible and usable for decision-making purposes. Industrial data operations helps break down data silos and improve predictive insights through an exchange and integration between shop floor and cloud environments.

6. Respondents are investing in underlying data architecture for AI 

According to the study, manufacturers plan to invest in AI-powered factories of the future within the next two years. On average, respondents expected their organizations to increase their investments in software for orchestrating edge AI by 11%. This investment shows that they recognize the need to overcome technical and skill gaps to fully exploit AI’s capabilities in future manufacturing processes. 

How to invest in underlying architecture for AI? 

Microsoft recommends adopting advanced technology frameworks such as centralized device management, software containerization at the edge, and unified industrial data operations to accelerate industrial transformation and prepare for AI. Azure’s adaptive cloud approach embraces all three advanced technology frameworks. 

Accelerate industrial transformation in manufacturing

A comprehensive survey of manufacturers’ priorities, challenges, and plans for adopting new technologies, such as these, in their factories to prepare for AI. The report shows that manufacturers are looking for solutions that can help them secure, scale, and automate. Microsoft Azure is responding to these needs with its adaptive cloud approach, which offers a flexible and scalable platform for managing devices, applications, and integrated data across the edge and the cloud.

Source: microsoft.com

Continue reading

Azure OpenAI Service: Transforming legal practices with generative AI solutions

In today’s fast-paced legal environment, the ability to efficiently manage and analyze vast amounts of data is crucial. And the field of law is no exception. In fact, a 2023 research paper claimed that of all the industries affected by new AI, the legal field was most exposed. AI is increasingly being leveraged to meet this need, enhancing the capabilities of legal professionals, and improving outcomes for clients. A recent Thomson Reuters survey reported that 82% of law firm lawyers said they believe that ChatGPT and generative AI could be readily applied to legal work. Another research report, by economists at Goldman Sachs, estimated that 44% of legal work could be automated.

Over the past several years, the data landscape has exploded, presenting legal teams with the challenge of managing not only increasing volumes of data but also a variety of new data types. Traditional sources like emails and documents are now accompanied by more complex sources such as collaboration platforms, chat data, text messages, and video recordings. Given the potential relevance of all this information in legal matters, modern legal teams must be prepared to identify, collect, and analyze vast amounts of data—often under tight deadlines. A number of law firms and legal service providers are using AI to streamline processes, reduce risks, and increase efficiency. Notably, companies like Icertis, Relativity, and Clifford Chance are pioneering the integration of AI into their workflows, demonstrating the transformative power of Azure AI Services in the legal field.

Key benefits and applications at work

The following AI applications can help teams throughout the legal field manage contracts more efficiently, reduce risk, ensure compliance, and drive better legal business:

• Enhanced document review: Uses natural language processing to analyze documents, providing relevant insights for legal cases.
• Accelerated e-discovery: Quickly identifies, collects, and analyzes large volumes of data from various sources.
• Improved efficiency: Reduces the time and resources needed for document review. 
• Identification of key information: Uncovers critical terms and conditions buried within documents.
• Risk management: Assist legal users to consider problematic terms and ensure compliance.
• Cognitive translation: Implements AI-driven translation to improve communication across languages.
• Accessible contracts: Natural language processing capabilities help users navigate and understand complex legal language.
• Enhanced decision-making: Provides insights for more informed strategic decisions.

Below we look at three companies who have adopted Azure OpenAI Service to support their legal practices, illustrating the profound impact and potential of these technologies in reshaping the industry.

◉ Revolutionizing contract life cycle management with generative AI and Azure

Contracts can be crucial in business, but managing them often remains disjointed across departments, and may lead to inefficiencies and risks. Icertis, used by 30% of Fortune 100 companies, has partnered with Microsoft to enhance contract management using AI. Their platform, Icertis Contract Intelligence (ICI), now incorporates generative AI through ICI Copilots, which streamlines contract reviews and uncovers hidden terms, thereby reducing risks and increasing efficiency. Icertis natively interoperates with Microsoft 365, Dynamics 365, and Azure AI to empower users to create, view, manage, and collaborate on contracts in the tools and applications they use every day. This collaboration helps customers achieve considerable time savings and better risk management. The partnership emphasizes continuous innovation and customer value, enhancing contract management capabilities and solidifying both companies’ market positions.

◉ Relativity and Microsoft partner to deliver generative AI to litigators

In the realm of e-discovery, efficiently organizing and analyzing vast amounts of unstructured data is critical. Relativity, partnering with Microsoft, provides AI-powered solutions to address this challenge. Relativity leverages Microsoft Azure and GPT-4 to enhance document review processes. They developed Relativity aiR for Review on top of Azure OpenAI Service to deliver a streamlined experience directly in RelativityOne. Relativity aiR for Review, uses natural language processing to analyze documents and assists with legal reviews and investigations. This partnership promotes security, interoperability, and global reach—enabling legal teams to manage growing data volumes and diverse data types effectively.

The collaboration focuses on responsible AI, aiming to deliver innovative, secure, and efficient tools for legal professionals. Relativity and Microsoft’s joint efforts aim to continue advancing AI capabilities in e-discovery and offer tools to assist with complex data challenges.

◉ Clifford Chance trailblazes Microsoft responsible AI to improve services for its legal clients

Clifford Chance, a leading British multinational law firm, leveraged advanced technologies like Azure, Azure OpenAI Service, and Microsoft Copilot for Microsoft 365 to enhance their legal services. Early implementations, such as cognitive translation, have quickly become some of their fastest-growing products, significantly improving their ability to handle sensitive, text-based information efficiently. Additionally, the firm benefits from intelligent recap and enhanced data security features through Microsoft Teams Premium, resulting in substantial cost savings and improved protection of client data. By integrating AI-powered solutions, Clifford Chance aims to free up their professionals’ time for strategic tasks and provide innovative, efficient services to their clients. The firm is excited about the potential of large language models (LLMs) and other AI tools to further transform their operations, making them more responsive and effective in a highly competitive and time-pressured environment.

Azure OpenAI Service: Impact

The integration of Azure AI Services is assisting legal professionals in managing data, conducting reviews, and offering services. Companies like Icertis, Relativity, and Clifford Chance are leveraging AI to boost operational efficiency and foster a more innovative and responsive legal system. As AI technologies continue to advance, their impact on the legal industry will grow, driving greater efficiencies and yielding better outcomes for all stakeholders.

Our commitment to responsible AI

Organizations across industries are leveraging Azure OpenAI Service and copilot services and capabilities to drive growth, increase productivity, and create value-added experiences. From advancing medical breakthroughs to streamlining manufacturing operations, our customers trust that their data is protected by robust privacy protections and data governance practices. As our customers continue to expand their use of our AI solutions, they can be confident that their valuable data is safeguarded by industry-leading data governance and privacy practices in the most trusted cloud on the market today.  

At Microsoft, we have a long-standing practice of protecting our customers’ information. Our approach to responsible AI is built on a foundation of privacy, and we remain dedicated to upholding core values of privacy, security, and safety in all our generative AI products and solutions.

Source: microsoft.com

Continue reading

From code to production: New ways Azure helps you build transformational AI experiences

We’re witnessing a critical turning point in the market as AI moves from the drawing boards of innovation into the concrete realities of everyday life. The leap from potential to practical application marks a pivotal chapter, and you, as developers, are key to bringing it to bear.

The news at Build is focused on the top demands we’ve heard from all of you as we’ve worked together to turn this promise of AI into reality:

• Empowering every developer to move with greater speed and efficiency, using the tools you already know and love.
• Expanding and simplifying access to the AI, data—application platform services you need to be successful so you can focus on building transformational AI experiences.
• And, helping you focus on what you do best—building incredible applications—with responsibility, safety, security, and reliability features, built right into the platform. 

I’ve been building software products for more than two decades now, and I can honestly say there’s never been a more exciting time to be a developer. What was once a distant promise is now manifesting—and not only through the type of apps that are possible, but how you can build them.

With Microsoft Azure, we’re meeting you where you are today—and paving the way to where you’re going. So let’s jump right into some of what you’ll learn over the next few days. Welcome to Microsoft Build 2024!

Create the future with Azure AI: offering you tools, model choice, and flexibility  

The number of companies turning to Azure AI continues to grow as the list of what’s possible expands. We’re helping more than 50,000 companies around the globe achieve real business impact using it—organizations like Mercedes-Benz, Unity, Vodafone, H&R Block, PwC, SWECO, and so many others.  

To make it even more valuable, we continue to expand the range of models available to you and simplify the process for you to find the right models for the apps you’re building.

Azure AI Studio, a key component of the copilot stack, is now generally available. The pro-code platform empowers responsible generative AI development, including the development of your own custom copilot applications. The seamless development approach includes a friendly user interface (UI) and code-first capabilities, including Azure Developer CLI (AZD) and AI Toolkit for VS Code, enabling developers to choose the most accessible workflow for their projects.

Developers can use Azure AI Studio to explore AI tools, orchestrate multiple interoperating APIs and models; ground models using their data using retrieval augmented generation (RAG) techniques; test and evaluate models for performance and safety; and deploy at scale and with continuous monitoring in production.

Empowering you with a broad selection of small and large language models  

Our model catalog is the heart of Azure AI Studio. With more than 1,600 models available, we continue to innovate and partner broadly to bring you the best selection of frontier and open large language models (LLMs) and small language models (SLMs) so you have flexibility to compare benchmarks and select models based on what your business needs. And, we’re making it easier for you to find the best model for your use case by comparing model benchmarks, like accuracy and relevance.

I’m excited to announce OpenAI’s latest flagship model, GPT-4o, is now generally available in Azure OpenAI Service. This groundbreaking multimodal model integrates text, image, and audio processing in a single model and sets a new standard for generative and conversational AI experiences. Pricing for GPT-4o is $5/1M Tokens for input and $15/1M Tokens for output.

Earlier this month, we enabled GPT-4 Turbo with Vision through Azure OpenAI Service. With these new models developers can build apps with inputs and outputs that span across text, images, and more, for a richer user experience. 

We’re announcing new models through Models-as-a-Service (MaaS) in Azure AI Studio leading Arabic language model Core42 JAIS and TimeGen-1 from Nixtla are now available in preview. Models from AI21, Bria AI, Gretel Labs, NTT DATA, Stability AI as well as Cohere Rerank are coming soon.  

Phi-3: Redefining what’s possible with SLMs

At Build we’re announcing Phi-3-small, Phi-3-medium, and Phi-3-vision, a new multimodal model, in the Phi-3 family of AI small language models (SLMs), developed by Microsoft. Phi-3 models are powerful, cost-effective and optimized for resource constrained environments including on-device, edge, offline inference, and latency bound scenarios where fast response times are critical. 

Sized at 4.2 billion parameters, Phi-3-vision supports general visual reasoning tasks and chart/graph/table reasoning. The model offers the ability to input images and text, and to output text responses. For example, users can ask questions about a chart or ask an open-ended question about specific images. Phi-3-mini and Phi-3-medium are also now generally available as part of Azure AI’s MaaS offering.

In addition to new models, we are adding new capabilities across APIs to enable multimodal experiences. Azure AI Speech has several new features in preview including Speech analytics and Video translation to help developers build high-quality, voice-enabled apps. Azure AI Search now has dramatically increased storage capacity and up to 12X increase in vector index size at no additional cost to run RAG workloads at scale.

Bring your intelligent apps and ideas to life with Visual Studio, GitHub, and the Azure platform

The tools you choose to build with should make it easy to go from idea to code to production. They should adapt to where and how you work, not the other way around. We’re sharing several updates to our developer and app platforms that do just that, making it easier for all developers to build on Azure. 

Access Azure services within your favorite tools for faster app development

By extending Azure services natively into the tools and environments you’re already familiar with, you can more easily build and be confident in the performance, scale, and security of your apps.  

We’re also making it incredibly easy for you to interact with Azure services from where you’re most comfortable: a favorite dev tool like VS Code, or even directly on GitHub, regardless of previous Azure experience or knowledge. Today, we’re announcing the preview of GitHub Copilot for Azure, extending GitHub Copilot to increase its usefulness for all developers. You’ll see other examples of this from Microsoft and some of the most innovative ISVs at Build, so be sure to explore our sessions.  

Also in preview today is the AI Toolkit for Visual Studio Code, an extension that provides development tools and models to help developers acquire and run models, fine-tune them locally, and deploy to Azure AI Studio, all from VS Code.  

Updates that make cloud native development faster and easier

.NET Aspire has arrived! This new cloud-native stack simplifies development by automating configurations and integrating resilient patterns. With .NET Aspire, you can focus more on coding and less on setup while still using your preferred tools. This stack includes a developer dashboard for enhanced observability and diagnostics right from the start for faster and more reliable app development. Explore more about the general availability of .NET Aspire on the DevBlogs post.   

We’re also raising the bar on ease of use in our application platform services, introducing Azure Kubernetes Services (AKS) Automatic, the easiest managed Kubernetes experience to take AI apps to production. In preview now, AKS Automatic builds on our expertise running some of the largest and most advanced Kubernetes applications in the world, from Microsoft Teams to Bing, XBox online services, Microsoft 365 and GitHub Copilot to create best practices that automate everything from cluster set up and management to performance and security safeguards and policies.

As a developer you now have access to a self-service app platform that can move from container image to deployed app in minutes while still giving you the power of accessing the Kubernetes API. With AKS Automatic you can focus on building great code, knowing that your app will be running securely with the scale, performance and reliability it needs to support your business.

Data solutions built for the era of AI

Developers are at the forefront of a pivotal shift in application strategy which necessitates optimizations at every tier of an application—including databases—since AI apps require fast and frequent iterations to keep pace with AI model innovation. 

We’re excited to unveil new data and analytics features this week designed to assist you in the critical aspects of crafting intelligent applications and empowering you to create the transformative apps of today and tomorrow.

Enabling developers to build faster with AI built into Azure databases 

Vector search is core to any AI application so we’re adding native capabilities to Azure Cosmos DB with Azure Cosmos DB for NoSQL. Powered by DiskANN, a powerful algorithm library, this makes Azure Cosmos DB the first cloud database to offer lower latency vector search at cloud scale without the need to manage servers. 

We’re also announcing the availability of Azure Database for PostgreSQL extension for Azure AI to make bringing AI capabilities to data in PostgreSQL data even easier. Now generally available, this enables developers who prefer PostgreSQL to plug data directly into Azure AI for a simplified path to leverage LLMs and build rich PostgreSQL generative AI experiences.   

Embeddings enable AI models to better understand relationships and similarities between data, which is key for intelligent apps. Azure Database for PostgreSQL in-database embedding generation is now in preview so embeddings can be generated right within the database—offering single-digit millisecond latency, predictable costs, and the confidence that data will remain compliant for confidential workloads. 

Making developer life easier through in-database Copilot capabilities

These databases are not only helping you build your own AI experiences. We’re also applying AI directly in the user experience so it’s easier than ever to explore what’s included in a database. Now in preview, Microsoft Copilot capabilities in Azure SQL DB convert queries into SQL language so developers can use natural language to interact with data. And, Copilot capabilities are coming to Azure Database for MySQL to provide summaries of technical documentation in response to user questions—creating an all-around easier and more enjoyable management experience.

Microsoft Copilot capabilities in the database user experience

Microsoft Fabric updates: Build powerful solutions securely and with ease

We have several Fabric updates this week, including the introduction of Real-Time Intelligence. This completely redesigned workload enables you to analyze, explore, and act on your data in real time. Also coming at Build: the Workload Development Kit in preview, making it even easier to design and build apps in Fabric. And our Snowflake partnership expands with support for Iceberg data format and bi-directional read and write between Snowflake and Fabric’s OneLake.

Spend a day in the life of a piece of data and learn exactly how it moves from its database home to do more than ever before with the insights of Microsoft Fabric, real-time assistance by Microsoft Copilot, and the innovative power of Azure AI.  

Build on a foundation of safe and responsible AI

What began with our principles and a firm belief that AI must be used responsibly and safely has become an integral part of the tooling, APIs, and software you use to scale AI responsibly. Within Azure AI, we have 20 Responsible AI tools with more than 90 features. And there’s more to come, starting with updates at Build.

New Azure AI Content Safety capabilities

We’re equipping you with advanced guardrails that help protect AI applications and users from harmful content and security risks and this week, we’re announcing new  feature for Azure AI Content Safety. Custom Categories are coming soon so you can create custom filters for specific content filtering needs. This feature also includes a rapid option, enabling you to deploy new custom filters within an hour to protect against emerging threats and incidents.  

Prompt Shields and Groundedness Detection are both available in preview now in Azure OpenAI Service and Azure AI Studio help fortify AI safety. Prompt shields mitigate both indirect and jailbreak prompt injection attacks on LLMs, while Groundedness Detection enables detection of ungrounded materials or hallucinations in generated responses.  

Features to help secure and govern your apps and data

Microsoft Defender for Cloud now extends its cloud-native application protection to AI applications from code to cloud. And, AI security posture management capabilities enable security teams to discover their AI services and tools, identify vulnerabilities, and proactively remediate risks. Threat protection for AI workloads in Defender for Cloud leverages a native integration with Azure AI Content Safety to enable security teams to monitor their Azure OpenAl applications for direct and in-direct prompt injection attacks, sensitive data leaks and other threats so they can quickly investigate and respond.

With easy-to-use APIs, app developers can easily integrate Microsoft Purview into line of business apps to get industry-leading data security and compliance for custom-built AI apps. You can empower your app customers and respective end users to discover data risks in AI interactions, protect sensitive data with encryption, and govern AI activities. These capabilities are available for Copilot Studio in public preview and soon (coming in July) will be available in public preview for Azure AI Studio, and via the Purview SDK, so developers can benefit from the data security and compliance controls for their AI apps built on Azure AI.

Two final security notes. We’re also announcing a partnership with HiddenLayer to scan open models that we onboard to the catalog, so you can verify that the models are free from malicious code and signs of tampering before you deploy them. We are the first major AI development platform to provide this type of verification to help you feel more confident in your model choice. 

Second, Facial Liveness, a feature of the Azure AI Vision Face API which has been used by Windows Hello for Business for nearly a decade, is now available in preview for browser. Facial Liveness is a key element in multi-factor authentication (MFA) to prevent spoofing attacks, for example, when someone holds a picture up to the camera to thwart facial recognition systems. Developers can now easily add liveness and optional verification to web applications using Face Liveness, with the Azure AI Vision SDK, in preview.

Our belief in the safe and responsible use of AI is unwavering. You can read our recently published Responsible AI Transparency Report for a detailed look at Microsoft’s approach to developing AI responsibly. We’ll continue to deliver more innovation here and our approach will remain firmly rooted in principles and put into action with built-in features.

Move your ideas from a spark to production with Azure

Organizations are rapidly moving beyond AI ideation and into production. We see and hear fresh examples every day of how our customers are unlocking business challenges that have plagued industries for decades, jump-starting the creative process, making it easier to serve their own customers, or even securing a new competitive edge. We’re curating an industry-leading set of developer tools and AI capabilities to help you, as developers, create and deliver the transformational experiences that make this all possible.

Source: microsoft.com

Continue reading

AI-powered dialogues: Global telecommunications with Azure OpenAI Service

In an era where digital innovation is king, the integration of Microsoft Azure OpenAI Service is cutting through the static of the telecommunications sector. Industry leaders like Windstream, AudioCodes, AT&T, and Vodafone are leveraging AI to better engage with their customers and streamline their operations. These companies are pioneering the use of AI to not only enhance the quality of customer interactions but also to optimize their internal processes—demonstrating a unified vision for a future where digital and human interactions blend seamlessly.

Leveraging Azure OpenAI Service to enhance communication

Below we look at four companies who have strategically adopted Azure OpenAI Service to create more dynamic, efficient, and personalized communication methods for customers and employees alike. 

1. Windstream’s AI-powered transformation streamlines operational efficiencies: Windstream sought to revolutionize its operations, enhancing workflow efficiency and customer service.  

Windstream streamlined workflows and improved service quality by analyzing customer calls and interactions with AI-powered analytics, providing insights into customer sentiments and needs. This approach extends to customer communications, where technical data is transformed into understandable outage notifications, bolstering transparency, and customer trust. Internally, Windstream has capitalized on AI for knowledge management, creating a custom-built generative pre-trained transformer (GPT) platform within Microsoft Azure Kubernetes Service (AKS) to index and make accessible a vast repository of documents, which enhances decision-making and operational efficiency across the company. The adoption of AI has facilitated rapid, self-sufficient onboarding processes, and plans are underway to extend AI benefits to field technicians to provide real-time troubleshooting assistance through an AI-enhanced index of technical documents. Windstream’s strategic focus on AI underscores the company’s commitment to innovation, operational excellence, and superior customer service in the telecommunications environment.

2. AT&T automates for efficiency and connectivity with Azure OpenAI Service: AT&T sought to boost productivity, enhance the work environment, and reduce operational costs. 

AT&T is leveraging Azure OpenAI Service to automate business processes and enhance both employee and customer experiences, aligning with its core purpose of fostering connections across various aspects of life including work, health, education, and entertainment. This strategic integration of Azure and AI technologies into their operations allows the company to streamline IT tasks and swiftly respond to basic human resources inquiries. In its quest to become the premier broadband provider in the United States and make the internet universally accessible, AT&T is committed to driving operational efficiency and better service through technology. The company is employing Azure OpenAI Service for various applications, including assisting IT professionals in managing resources, facilitating the migration of legacy code to modern frameworks to spur developer productivity, and enabling employees to effortlessly complete routine human resources tasks. These initiatives allow AT&T staff to concentrate on more complex and value-added activities, enhancing the quality of customer service. Jeremy Legg, AT&T’s Chief Technology Officer, highlights the significance of automating common tasks with Azure OpenAI Service, noting the potential for substantial time and cost savings in this innovative operational shift. 

3. Vodafone revolutionizes customer service with TOBi and Microsoft Azure AI: Vodafone sought to lower development costs, quickly enter new markets, and improve customer satisfaction with more accurate and personable interactions. 

Vodafone, a global telecommunications giant, has embarked on a digital transformation journey, central to which is the development of TOBi, a digital assistant created using Azure services. TOBi, designed to provide swift and engaging customer support, has been customized and expanded to operate in 15 languages across multiple markets. This move not only accelerates Vodafone’s ability to enter new markets but also significantly lowers development costs and improves customer satisfaction by providing more accurate and personable interactions. The assistant’s success is underpinned by Azure Cognitive Services, which enables it to understand and process natural language, making interactions smooth and intuitive. Furthermore, Vodafone’s initiative to leverage the new conversational language understanding feature from Microsoft demonstrates its forward-thinking approach to providing multilingual support, notably in South Africa, where TOBi will soon support Zulu among other languages. This expansion is not just about broadening the linguistic reach but also about fine-tuning TOBi’s conversational abilities to recognize slang and discern between similar requests, thereby personalizing the customer experience.  

4. AudioCodes leverages Microsoft Azure for enhanced communication: AudioCodes sought streamlined workflows, improved service level agreements (SLAs), and increased visibility. 

AudioCodes, a leader in voice communications solutions for over 30 years, migrated its solutions to Azure for faster deployment, reduced costs, and improved SLAs. The result? The company’s ability to serve its extensive customer base, which includes more than half of the Fortune 100 enterprises. The company’s shift towards managed services and the development of applications aimed at enriching customer experiences is epitomized by AudioCodes Live, designed to facilitate the transition to Microsoft Teams Phone. AudioCodes has embraced cloud technologies, leveraging Azure services to streamline telephony workflows and create advanced applications for superior call handling, such as its Microsoft Teams-native contact center solution, Voca. By utilizing Azure AI and AI, Voca offers enterprises robust customer interaction capabilities, including intelligent call routing and customer relationship management (CRM) integration. AudioCodes’ presence on Azure Marketplace has substantially increased its visibility, generating over 11 million usage hours a month from onboarded customers. The company plans to utilize Azure OpenAI Service in the future to bring generative AI capabilities into its solutions.

The AI enhanced future of global telecommunications 

The dawn of a new era in telecommunications is upon us, with industry pioneers like Windstream, AudioCodes, AT&T, and Vodafone leading the charge into a future where AI and Azure services redefine the essence of connectivity. Their collective journey not only highlights a shared commitment to enhancing customer experience and operational efficiency, but also paints a vivid picture of a world where communication transcends traditional boundaries, enabled by the fusion of cloud infrastructure and advanced AI technologies. This visionary approach is laying the groundwork for a paradigm where global communication is more seamless, intuitive, and impactful, demonstrating the unparalleled potential of AI to weave a more interconnected and efficient fabric of global interaction. 

Our commitment to responsible AI

With responsible AI tools in Azure, Microsoft is empowering organizations to build the next generation of AI apps safely and responsibly. Microsoft has announced the general availability of Azure AI Content Safety, a state-of-the art AI system that helps organizations keep AI-generated content safe and create better online experiences for everyone. Customers—from startup to enterprise—are applying the capabilities of Azure AI Content Safety to social media, education, and employee engagement scenarios to help construct AI systems that operationalize fairness, privacy, security, and other responsible AI principles. 

Source: microsoft.com

Continue reading

Azure AI Health Bot helps create copilot experiences with healthcare safeguards

The generative AI era is driving demand for chatbots and copilots for health that assist patients and medical professionals with various administrative and clinical tasks. These chatbots would potentially use large language models (LLMs) to generate conversational AI chat experiences that can provide accurate and reliable information based on large amounts of medical literature and data.

As a result of the growing demand, many healthcare organizations are striving to build their own healthcare copilot experiences that deliver intelligent and engaging chat experiences leveraging LLMs and generative AI.

In the process, healthcare organizations have realized that as part of healthcare’s unique needs, they need a way to combine the benefits of using generative AI for engaging chat experiences with the benefits of protocol-based flows and custom workflows to provide accurate and relevant information. A hybrid approach that combines both would allow them to offer a more personalized and comprehensive service to their customers and end users.

Moreover, healthcare chat experiences need to leverage domain-specialized models and health-specific safeguards to meet the healthcare industry quality bar.

To address these needs, we’re adding new healthcare-specific safeguards for generative AI in private preview within the Azure AI Health Bot services. Preview customers can experience an integration with Microsoft Copilot Studio, allowing healthcare organizations to build their own copilot experiences. Customers can sign up for the private preview here.

• Providing reusable healthcare-specific functionality: providing healthcare-specific, pre-built capabilities, use cases and scenarios—including pre-packaged healthcare intelligence plugins, templates, content, and healthcare-specialized skills and connectors.
• Answering the unique needs of healthcare: enabling customers to build copilots for their patients and doctors, supporting protocol-based workflows side-by-side with generative AI-based answers, and allowing customers to keep alignment with up-to-date industry standards, guidelines, and protocols.
• Applying healthcare-specific safeguards: allowing customers to build copilots responsibly adapted to healthcare needs, apply health-adapted compliance controls, and implement health-specific safeguards and quality measures that are specialized for healthcare.

Generative AI capabilities 

In April 2023, we announced the preview of Azure AI Health Bot with Azure OpenAI Service, enabling fallback answers based on generative AI.

Today, we are expanding those capabilities beyond fallback answers, enabling our healthcare customers to further enrich their copilot experiences with the following capabilities in private preview:  

• Power generative answers that are grounded on customer’s own sources. The sources are incorporated during the copilot experience, alongside authored descriptive scenarios, protocol-based pre-built flows, and skills. Customers are able to bring in their Azure OpenAI Service endpoint and index to enable generative answers grounded on their desired sources.
• Generative answers that are grounded on the customer’s websites. These sources are real-time queried and can include medical guidelines, health articles, patient treatments, frequently asked questions, appointment scheduling information, and many more. This approach ensures that patients receive not only medical guidance but also support for the many aspects of their healthcare journey.
• New healthcare intelligence capabilities to incorporate generative answers grounded on credible healthcare sources. Sources include the National Institutes of Health (NIH), the Food and Drug Administration (FDA), and others.
• Seamlessly use pre-built protocol-based healthcare intelligence capabilities such as symptom checkers and triage, and a rich gallery of pre-built protocol templates side-by-side with generative AI based answers.
• Credible generative AI fallback ensures reliable and accurate responses in healthcare-related scenarios. In cases where answers are not available, this feature leverages credible content to enhance responses, providing users with reliable guidance backed by clinical Retrieval-Augmented Generation (RAG) support. Helping to mitigate potential errors and ensures the delivery of trusted information in healthcare settings.

Built-in safeguards

Azure AI Health Bot with generative AI technology provides built-in healthcare safeguards now in private preview, for building copilot experiences that fits healthcare’s unique requirements and needs. Those include:

◉ Clinical safeguards include healthcare-adapted filters and quality checks to allow verification of clinical evidence associated with answers, identifying hallucinations and omissions in generative answers, credible sources enforcement, and more.

◉ Healthcare chat safeguards include customizable AI-related disclaimers that are incorporated into the chat experience presented to users, enabling the collection of end-user feedback, and analyzing the engagement through built-in dedicated reporting, as well as healthcare-adapted abuse monitoring, among other things.

◉ Healthcare-adapted compliance controls include built-in Data Subject Rights (DSRs), pre-built consent management, out-of-the-box audit trails, and more.

Source: microsoft.com

Continue reading

Accelerate your productivity with the Whisper model in Azure AI now generally available

Human speech remains one of the most complex things for computers to process. With thousands of spoken languages in the world, enterprises often struggle to choose the right technologies to understand and analyze audio conversations while keeping right data security and privacy guardrails in place. Thanks to generative AI, it has become easier for enterprises to analyze every customer interaction and derive actionable insights from these interactions.

Azure AI offers an industry-leading portfolio of AI services to help customers make sense of their voice data. Our speech-to-text service in particular offers a variety of differentiated features through Azure OpenAI Service and Azure AI Speech. These features have been instrumental in helping customers develop multilingual speech transcription and translation, both for long audio files and for near-real-time and real-time assistance for customer service representatives.

Read More: MB-500: Microsoft Dynamics 365 - Finance and Operations Apps Developer

Today, we are excited to announce that OpenAI Whisper on Azure is generally available. Whisper is a speech to text model from OpenAI that developers can use to transcribe audio files. Starting today, developers can begin using the generally available Whisper API in both Azure OpenAI Service as well as Azure AI Speech services on production workloads, knowing that it is backed by Azure’s enterprise-readiness promise. With all our speech-to-text models generally available, customers have greater choice and flexibility to enable AI powered transcription and other speech scenarios.

Since the public preview of the Whisper API in Azure, thousands of customers across industries across healthcare, education, finance, manufacturing, media, agriculture, and more are using it to translate and transcribe audio into text across many of the 57 supported languages. They use Whisper to process call center conversations, add captions for accessibility purposes to audio and video content, and mine audio and video data for actionable insights. 

We continue to bring OpenAI models to Azure to enrich our portfolio and address the next generation of use-cases and workflows customers are looking to build with speech technologies and LLMs. For instance, imagine building an end-to-end contact center workflow—with a self-service copilot carrying out human-like conversations with end users through voice or text; an automated call routing solution; real-time agent assistance copilots; and automated post-call analytics. This end-to-end workflow, powered by generative AI, has the potential to bring a new era in productivity to call centers around the world.

Whisper in Azure OpenAI Service 

Azure OpenAI Service enables developers to run OpenAI’s Whisper model in Azure, mirroring the OpenAI Whisper model functionalities including fast processing time, multi-lingual support, and transcription and translation capabilities. OpenAI Whisper in Azure OpenAI Service is ideal for processing smaller size files for time-sensitive workloads and use-cases. 

Lightbulb.ai, an AI innovator, is looking to transform call center workflows, has been using Whisper in Azure OpenAI Service.

“By merging our call center expertise with tools like Whisper and a combination of LLMs, our product is proven to be 500X more scalable, 90X faster, and 20X more cost-effective than manual call reviews and enables third-party administrators, brokerages, and insurance companies to not only eliminate compliance risk; but also to significantly improve service and boost revenue. We are grateful for our partnership with Azure, which has been instrumental in our success, and we’re enthusiastic about continuing to leverage Whisper to create unprecedented outcomes for our customers.”

Tyler Amundsen, CEO and Co-Founder, Lightbulb.AI

Try out the Whisper REST (representational state transfer) API in the Azure OpenAI Studio. The API supports translation services from a growing list of languages to English, producing English-only output. 

OpenAI Whisper model in Azure AI Speech

Users of Azure AI Speech can leverage OpenAI’s Whisper model in conjunction with the Azure AI Speech batch transcription API. This enables customers to easily transcribe large volumes of audio content at scale for non-time-sensitive batch workloads.

Developers using Whisper in Azure AI Speech also benefit from the following additional capabilities:

• Processing of large file sizes up to 1GB in size with the ability to process large amounts of files with up to 1000 files in a single request that processes multiple audio files simultaneously.
• Speaker diarization which allows developers to distinguish between different speakers, accurately transcribe their words, and create a more organized and structured transcription of audio files.
• And lastly, developers can use Custom Speech in Speech Studio or via API to finetune the Whisper model using audio plus human labeled transcripts.

Customers are using Whisper in Azure AI Speech for post-call analysis, deriving insights from audio and video recordings, and many more such applications.

Source: microsoft.com

Continue reading

Azure OpenAI Service powers the Microsoft Copilot ecosystem

Many AI systems are designed for collaboration: Copilot is one of them. Copilot—powered by Microsoft Azure OpenAI Service—allows you to simplify how you design, operate, optimize, and troubleshoot apps and infrastructure from cloud to edge. It utilizes language models, the Azure control plane, and insights about your Azure and Arc-enabled assets. All of this is carried out within the framework of Azure’s steadfast commitment to safeguarding data security and privacy.

A brief history of AI collaboration with copilots

In aviation terms a copilot is responsible for assisting the pilot in command, sharing control of the airplane, and handling various navigational and operational tasks. Having a copilot ensures that there is a second trained professional who can take over controls if the main pilot is unable to perform their duties, thereby enhancing safety.

Microsoft originally introduced the concept of a copilot two years ago as an AI pair programmer in GitHub to assist developers in generating code, catching errors, and suggesting improvements. Today, Azure OpenAI Service powers more than just GitHub Copilot. Microsoft 365 Copilot performs as a digital companion for your whole life creating a single Copilot user experience across Bing, Edge, Microsoft 365, and Windows.

AI at the service of others

Microsoft Copilot represents a profound shift in how AI-powered software can support the user experience, the architecture, the services that it uses, and how we think about safety and security.

“We now have machines that are so fluent in human language. Every place that you interact with a machine ought to be much more fluent in human natural language and I think we’ll start to see that change coming in a lot of different places as well and it will really redefine the interfaces that we’re used to.”—Eric Boyd, head of AI at Microsoft.

Copilots powered by Azure OpenAI Service can be trained on a specific set of data to adapt the model to a specific domain. We’re seeing developments across a variety of sectors. For example:

Language translation: Language translation models can help bridge communication gaps between people who speak different languages. This can be particularly useful in situations such as emergency response, disaster relief, and international diplomacy.

Educational support: Educational chatbots that can help students with homework, provide personalized tutoring, and answer questions related to different subjects.

Crime investigation: Financial crimes such as money laundering and fraud are linked to human trafficking, child exploitation, terrorism, theft, and wildlife trafficking. SymphonyAI’s new Sensa Copilot acts as a sophisticated AI assistant to a financial crime investigator by automatically collecting, collating, and summarizing financial and third-party information.

Medical reporting: Generative AI has the potential to increase the power and accessibility of self-service reporting, making it easier for healthcare organizations and their providers to identify operational improvements, including ways to reduce costs and to find answers to questions both locally and within a broader context.

Climate change: Azure OpenAI Service can be used to generate educational materials or assist in research on topics related to climate change, including natural disasters, global warming, and environmental conservation.

Inclusive and diverse avatars: DeepBrain AI includes a library of photo-realistic and virtual avatars that businesses can use for training videos, news broadcasts, marketing videos, and more. An integral part of the digital world, avatars foster a sense of inclusivity and diversity by allowing people to choose representations that reflect their individuality, regardless of physical appearance or other limitations.

Industrial advances: ABB is partnering with Microsoft to integrate Azure OpenAI Service into its ABB Ability™ Genix Industrial Analytics and AI suite with the goal of boosting real-time insights and asset longevity by 20% and reducing unplanned downtimes by 60%. Additionally, it will aid in monitoring and optimizing industrial emissions and energy usage, contributing to sustainability goals.

Prioritizing human agency

The Copilot System powered by Azure OpenAI Service builds on our existing commitments to data security and privacy in the enterprise. Copilot automatically inherits your organization’s security, compliance, and privacy policies for Microsoft 365. Data is managed in line with our current commitments. Copilot prioritizes human agency and puts the user in control. This includes noting limitations, providing links to sources, and prompting users to review, fact-check, and fine-tune content based on their own knowledge and judgment.

AI systems can analyze and learn from copious amounts of data and help employees make decisions based on that data. They can be programmed for specific tasks such as image recognition and natural language processing.

While technology has the potential to generate both favorable and adverse consequences, technological developments such as Copilot are proving far more likely to help society steer a straight and humane course toward a future that benefits us all.

Our commitment to responsible AI

With Responsible AI tools in Azure, Microsoft is empowering organizations to build the next generation of AI apps safely and responsibly. Microsoft has announced the general availability of Azure AI Content Safety, a state-of-the art AI system that helps organizations keep AI-generated content safe and create better online experiences for everyone. Customers—from startup to enterprise—are applying the capabilities of Azure AI Content Safety to social media, education, and employee engagement scenarios to help construct AI systems that operationalize fairness, privacy, security, and other responsible AI principles.

Source: microsoft.com

Continue reading