AI models have become integral to enhancing computing efficiency, productivity, and user experiences. Developing small language models (SLMs) is a key focus, enabling more efficient processing on personal computing devices.
The problem addressed by researchers is the high computational demand for AI models, which often require substantial power and resources, limiting their deployment on smaller devices like personal computers. Reducing resource consumption while maintaining high performance is crucial for integrating AI seamlessly into everyday computing.
Existing methods involve using larger AI models that consume significant computational power, which can impact the overall performance of personal computers. These models rely heavily on the central processing unit (CPU) and graphics processing unit (GPU), which can slow down other tasks and reduce efficiency.
Microsoft researchers introduced Phi Silica, a small language model specifically designed for the Neural Processing Units (NPUs) in their new Copilot+ PCs. Phi Silica is part of the Phi family of models and is intended to deliver high-performance AI capabilities while consuming minimal power. This design allows the CPU and GPU to remain available for other tasks, enhancing the overall computing experience.
Phi Silica stands out with its 3.3 billion parameters, making it the smallest model in the Phi family. Despite its compact size, Phi Silica achieves impressive performance metrics. It boasts a first-token latency of 650 tokens per second and consumes only 1.5 Watts of power. This efficiency ensures that the PC’s CPU and GPU are not burdened, allowing for smoother operation of other applications. Phi Silica’s token generation also reuses the NPU’s KV cache and runs on the CPU, producing approximately 27 tokens per second.
Developers can access the Phi Silica API through the Windows App SDK and other AI-powered functionalities such as Optical Character Recognition (OCR), Studio Effects, Live Captions, and Recall User Activity APIs. This integration enables developers to create innovative experiences that leverage AI within the Windows ecosystem. Microsoft plans to release additional APIs, including Vector Embedding, RAG API, and Text Summarization, further expanding the capabilities available to developers.
Phi Silica joins the ranks of other models in the Phi-3 series, including Phi-3-mini with 3.8 billion parameters, Phi-3-small with 7 billion parameters, Phi-3-medium with 14 billion parameters, and the recently announced Phi-3-vision with 4.2 billion parameters. However, Phi Silica is unique as the first state-of-the-art SLM to be shipped with Windows, marking a significant milestone in bringing advanced AI capabilities directly to end-users.
The introduction of Phi Silica follows Microsoft’s announcement of the Copilot+ PC, which promises to deliver Windows PCs equipped with dedicated AI processors. The first Copilot+ PCs will launch in mid-June, featuring Qualcomm’s Arm-based Snapdragon X Elite and Plus chips. Microsoft will offer these AI-powered laptops in collaboration with major PC manufacturers throughout the summer. Intel is also developing its Copilot+ PC processor, codenamed Lunar Lake, slated for release in the third quarter of 2024.
Key features of Phi Silica:
- Model Size and Efficiency: Phi Silica is the smallest model in the Phi family, with 3.3 billion parameters. It delivers high performance with a first token latency of 650 tokens per second while consuming only 1.5 Watts of power, ensuring minimal resource usage on the PC’s CPU and GPU.
- Token Generation: This function utilizes the NPU’s KV cache and runs on the CPU, producing approximately 27 tokens per second, enhancing the overall computing experience.
- Developer Integration: Developers can access the Phi Silica API through the Windows App SDK. It includes functionalities like OCR, Studio Effects, Live Captions, and Recall User Activity APIs, enabling innovative AI applications within the Windows ecosystem.
- Advanced AI Capabilities: Phi Silica is the first state-of-the-art small language model shipped with Windows, marking a significant milestone in AI accessibility for end-users and developers.
- Collaborative Efforts: Launched alongside Microsoft’s Copilot+ PCs, which feature Qualcomm’s Snapdragon X Elite and Plus chips, and Intel’s upcoming Lunar Lake processors. These AI-powered laptops will be available starting in mid-June 2024.
- Performance and Power Usage: Designed to run efficiently on Copilot+ PCs’ NPUs, ensuring rapid local inferencing while maintaining low power consumption, significantly elevating productivity and accessibility within the Windows platform
In conclusion, Microsoft’s development of Phi Silica addresses the critical challenge of resource consumption in AI models. By providing a high-performance, efficient model that operates within the constraints of personal computing devices, Phi Silica enhances the user experience and paves the way for more innovative applications. This model allows the integration of AI into everyday computing, offering powerful tools without compromising system performance.
Asif Razzaq is the CEO of Marktechpost Media Inc.. As a visionary entrepreneur and engineer, Asif is committed to harnessing the potential of Artificial Intelligence for social good. His most recent endeavor is the launch of an Artificial Intelligence Media Platform, Marktechpost, which stands out for its in-depth coverage of machine learning and deep learning news that is both technically sound and easily understandable by a wide audience. The platform boasts of over 2 million monthly views, illustrating its popularity among audiences.
Wanda Parisien is a computing expert who navigates the vast landscape of hardware and software. With a focus on computer technology, software development, and industry trends, Wanda delivers informative content, tutorials, and analyses to keep readers updated on the latest in the world of computing.
