At Ignite 2023, Microsoft unveiled a comprehensive vision for its end-to-end AI stack, showcasing innovations that span from cloud infrastructure to AI-driven applications and security measures.
For Microsoft and its ecosystem, this year’s Ignite conference proved to be exceptional and different. Traditionally, Ignite has been a conference that typically focused on infrastructure and operations, while Microsoft’s flagship event, Build, usually catered to developer audiences. However, announcements about generative AI directed at developers and ML engineers took center stage at Ignite 2023. It was not limited to developers or IT professionals, but it became a watershed moment for the entire Microsoft ecosystem.
Microsoft wants to be a major force in the AI ecosystem, and Microsoft CEO and Chairman Satya Nadella made this clear in his keynote address. From developing its own AI accelerator chips to launching a marketplace for copilots, Microsoft has a long-term strategy in place.
Here is a detailed analysis of how Microsoft is capitalizing on AI to retain its leadership and dominance in the industry:
Azure is the new AI operating system, and Copilots are the new apps
Microsoft has an extremely successful track record of building platforms. The earliest version of the platform was built on Windows, where developers leveraged OLE and COM to build applications through Visual Basic. Announced in the early 2000s, Microsoft.NET and Visual Studio led to the creation of a new platform that rekindled interest in developers creating web services. Last decade, Microsoft successfully launched another platform in the form of Azure.
When Microsoft creates a platform, it leads to a new ecosystem of independent software vendors and solution providers, helping enterprises leverage it. This was evident from the success of Microsoft Windows, Office, Visual Studio and most recently Azure.
With AI, Microsoft wants to repeat the magic of creating a brand new platform that results in a thriving ecosystem of developers, ISVs, system integrators, enterprises and consumers.
This season, Azure becomes the operating system, providing the runtime and platform services, while the apps are the AI assistants that Microsoft calls copilots. So, Azure is the new Windows and copilots are the new applications. The foundation models, such as GPT-4, form the kernel of this new OS. Similar to Visual Studio, Microsoft has invested in a set of developer tools in the form of AI Studio and Copilot Studio. This stack closely resembles Windows, .NET and Visual Studio, which ruled the developer landscape for decades.
Microsoft’s approach clearly demonstrates a sense of urgency. This is obvious given the current market dynamics and the lessons learned from the failed attempts to build an ecosystem around the mobile platform. Satya is incredibly committed to ensuring that Microsoft becomes the company that pioneers artificial intelligence by bringing the capabilities of generative AI closer to its customers. He does not want the company to miss the next big thing in technology, like they did with search and mobile.
In just a few months, the company has shipped multiple copilots for its products, ranging from the Bing search engine to Microsoft 365 to the Windows operating system. It also added various capabilities to the Edge browser, enhancing the user experience. The speed with which Microsoft has embraced generative AI in recent months is astounding, making it one of the leading AI platform companies.
Microsoft invests in developing its own CPU, GPU and DPU
For decades, CPUs set the rules for software architecture and shaped its development. Now, AI software is shaping the development of chips, giving rise to purpose-built processors.
Microsoft formally announced that it would begin manufacturing its own silicon and processors, including CPUs, AI accelerators and data processing units.
Let us start with the CPU. Azure Cobalt, Microsoft’s own CPU, is based on Arm architecture for optimal performance and watt efficiency, and it powers common Azure cloud workloads. The first generation of the series, Cobalt 100, is a 64-bit 128-core chip that improves performance by up to 40% over current generations of Azure Arm chips and powers services such as Microsoft Teams and Azure SQL. After Neoverse N1, the first Arm-based CPU purchased from Ampere Computing, Cobalt 100 becomes the second Arm-based processor available on Azure.
Then there is Azure Maia, the first in a series of custom AI accelerators designed to run cloud-based training and inference for AI workloads like OpenAI models, Bing, GitHub Copilot and ChatGPT. With 105 billion transistors, the Maia 100 is the first generation in the series and one of the largest chips on 5nm process technology. It features numerous innovations in the areas of silicon, software, networking, racks and cooling. The new AI accelerator becomes an alternative to the GPU by optimizing Azure AI’s end-to-end systems to run state-of-the-art foundation models like GPT.
Finally, Azure Boost, Microsoft’s own DPU, became generally available. Microsoft acquired Fungible, a DPU company, earlier this year in order to improve the efficiency of Azure data centers. Software functions such as virtualization, network management, storage management and security are offloaded to dedicated hardware with Azure Boost, allowing the CPU to devote more cycles to workloads rather than systems management. Because the heavy lifting is moved to a purpose-built processor, this offloading significantly improves the performance of cloud infrastructure.
Apart from bringing its own silicon to the mix, Microsoft has partnered with AMD, Intel and NVIDIA to bring the latest CPUs and GPUs to Azure. It will have the latest NVIDIA H200 Tensor Core GPU by next year to run larger foundation models with reduced latency. AMD’s new MI300 accelerator will also become available on Azure early next year.
Less reliance on OpenAI with homegrown and open source foundation models
While Azure continues to be the preferred platform to run inference on OpenAI-based models for enterprises, Microsoft is investing in training its own foundation models that complement existing models available in Azure OpenAI and Azure ML.
Phi-1-5 and Phi-2 are small language models that are lightweight and need fewer resources than traditional large language models. Phi-1-5 has 1.3 billion parameters, while Phi-2 has 2.7 billion parameters, making them much smaller compared to Llama 2, which starts with 7 billion parameters and goes up to 70 billion parameters. These SLMs are ideal for embedding within Windows to provide a local copilot experience without making the roundtrip to the cloud. Microsoft is releasing an extension for Visual Studio Code that allows developers to fine-tune these models in the cloud and deploy them locally for offline inference.
Microsoft Research has developed Florence, a foundation model that brings multimodal capabilities to Azure Cognitive Services. This model allows users to analyze and understand images, video and language to offer customizable options for building computer vision applications. This model is already available in Azure.
Azure ML now supports additional open source foundation models, including Llama, Code Llama, Mistral 7B, Stable Diffusion, Whisper V3, BLIP, CLIP, Flacon and NVIDIA Nemotron.
Azure ML, Microsoft’s ML PaaS offers model-as-a-service to consume foundation models as an API without the need to provision GPU infrastructure. This significantly simplifies the integration of AI with modern applications.
The combination of Azure OpenAI and Azure model catalog delivers the most comprehensive and widest range of foundation models to customers, which becomes the key differentiating factor of Azure.
Microsoft Graph and Fabric at the Core of The Data Platform
AI requires a large amount of data for pre-training, fine-tuning and retrieval. Microsoft Fabric and Microsoft Graph are two key products that contribute significantly to Microsoft’s generative AI efforts.
Microsoft Fabric, which was announced at Microsoft Build 2023, is a significant addition to Microsoft’s data product line. Satya emphasized its significance by comparing it to the release of SQL Server, implying a fundamental shift in Microsoft’s data management and analytics strategy.
At Ignite 2023, Microsoft announced the general availability of Fabric. It includes a component named OneLake, which is a transformative data lakehouse platform. OneLake is integrated into Azure Machine Learning and Azure AI Studio, representing a major enhancement in Azure Machine Learning’s data management capabilities. This platform is designed to handle large and varied datasets in a unified and efficient manner, optimizing data storage and retrieval for AI applications. Its integration with Azure AI platforms is particularly crucial for scenarios that require high-volume data processing and complex computational tasks, common in advanced AI and machine learning projects. What’s interesting about OneLake is the concept of shortcuts that bring data from external sources, including Amazon S3 and Databricks, into the fold of Fabric.
Microsoft Graph, a powerful tool in Microsoft’s arsenal, plays a pivotal role in the realm of AI copilots. It has become pivotal for developing AI copilots, offering a unified API to access diverse data across Microsoft 365 services. Microsoft Graph enables copilots to provide personalized assistance by aggregating data from emails, calendar events and team interactions. This integrated approach ensures a contextual understanding of users’ professional environments, which is essential for making intelligent suggestions. Microsoft Graph supports real-time data access, which is crucial for timely copilot responses. Its compliance with Microsoft 365’s security standards ensures the safe handling of sensitive data.
Microsoft Fabric and Microsoft Graph become the foundation for building copilots based on real-time data available through APIs.
Overall, Microsoft’s strategy at Ignite 2023 demonstrates a clear focus on leading the AI revolution, leveraging its platform heritage and innovating in hardware and software to maintain industry dominance.
Read the full article here