وصول Microsoft Foundry Local إلى Android: إطلاق العنان للذكاء الاصطناعي على الجهاز مع إمكانات الكلام والمواقع المحلية

Microsoft is pushing the boundaries of AI accessibility with the introduction of Foundry Local on Android. This innovative platform empowers developers to build mobile applications that execute AI models directly on devices, eliminating the need for cloud connectivity and enabling enhanced privacy, speed, and flexibility. This announcement extends the reach of AI, making it readily available to a wider range of users and use cases. Furthermore, updates to Foundry Local expand it to edge, hybrid, and on-prem scenarios using Azure Arc-enabled Kubernetes.

On-Device AI: A Paradigm Shift

Foundry Local on Android marks a significant step towards democratizing AI. By enabling on-device AI processing, Microsoft addresses several critical challenges:

• Enhanced Privacy: Processing sensitive data, such as healthcare or financial information, locally ensures that it never leaves the device, adding an extra layer of security.
• Reduced Costs: Minimizing reliance on cloud calls translates to lower operational expenses for applications that heavily utilize AI.
• Offline Functionality: Applications can continue to function even in environments with limited or no internet connectivity.
• Improved Performance: Eliminating round trips over the internet significantly improves the speed and responsiveness of AI-powered features.

PhonePe, a digital payments platform serving over 618 million users, is already leveraging Foundry Local to power an upcoming AI-driven experience, demonstrating the platform’s scalability and real-world applicability.

Speech-to-Text API: Whisper-Powered Innovation

The new Speech API in Foundry Local unlocks powerful voice-driven scenarios. This feature utilizes the Whisper model to provide on-device speech-to-text capabilities with low latency. Key highlights include:

• Offline Transcription: Transcribe audio and understand speech without requiring an internet connection.
• Privacy-Focused Design: By default, audio never leaves the device, ensuring user privacy.
• Seamless User Experience: Enables intuitive voice prompting in offline environments, such as filling out forms or capturing notes.

The provided code snippet demonstrates how to easily acquire a Whisper model from the Foundry Catalog, load it into memory, and transcribe audio, showcasing the simplicity and ease of integration.

The Enhanced Foundry Local SDK

The updated Foundry Local SDK simplifies AI development with a focus on ease of use and performance:

• Self-Contained Packaging: Eliminates the need for separate executables for model serving.
• Smaller Footprint: Reduced package and runtime size for improved efficiency.
• Simplified API: Streamlined APIs for easy model downloading and loading.
• OpenAI Compatibility: APIs for chat completions and audio transcription that follow OpenAI request/response formats.
• Optional OpenAI Compliant Web Server: Facilitates integration with other tools and frameworks like LangChain and the OpenAI SDK.
• Windows ML Integration: Smart device detection for optimized performance.

The SDK automatically selects the most performant model for the device hardware and ensures that all necessary hardware runtime and drivers are installed. Code examples show how to acquire and use models for chat completions, highlighting the ease with which developers can implement AI functionality.

Foundry Local and Azure Arc-Enabled Kubernetes: AI at the Edge

Extending beyond personal computing devices, Foundry Local now supports edge, hybrid, sovereign, and disconnected environments through integration with Azure Arc-enabled Kubernetes.

• Seamless Deployment: Deploy validated AI solutions to edge environments such as manufacturing rigs or industrial machinery.
• Managed Microsoft Stack: Leverages Azure Local to provide a fully managed Microsoft stack for edge, hybrid, sovereign, and disconnected deployments.

This capability enables customers to run AI in containers on-premises, addressing scenarios with limited connectivity or specialized hardware requirements.

Looking Ahead: A Robust Roadmap

Microsoft envisions a future where Foundry Local becomes the leading platform for on-device AI. Their roadmap includes:

• General Availability: Bringing Foundry Local to General Availability for broader accessibility.
• Android Support: Strengthening Android support with ongoing enhancements.
• Windows AI Foundry: Continuing to advance Windows AI Foundry capabilities.
• New Capabilities: Investing in richer capabilities like tool calling, support for Linux, multi-modality, and expanded support for on-prem servers.

The announcement emphasizes Microsoft’s commitment to co-creation with partners and customers to deliver a powerful and trustworthy on-device AI experience.

Conclusion

The release of Foundry Local on Android, along with the expansion to edge environments and the enhanced SDK, represents a significant leap forward in making AI more accessible, private, and performant. By empowering developers to build on-device AI applications and providing a robust platform for edge deployments, Microsoft is paving the way for a future where AI is seamlessly integrated into our daily lives, even in the most challenging environments.

Source: Microsoft