MediaTek SoCs to use Meta’s Llama 2 to handle gen AI processing at Edge
Date: 29/08/2023
MediaTek developing technologies to enable generative AI capability in edge devices such as smart phones, IOT, automotives and smart homes. MediaTek is working on making Meta’s open-source Large Language Model (LLM) Llama 2 to run on its latest APUs and NeuroPilot AI Platform. The idea is to enable edge devices to perform
Generative AI processing at edge without depending on the cloud computers, so that they can perform on-device AI processing at any time during network unavailability. The Other advantages include seamless performance, greater privacy, better security and reliability, lower latency and lower operation cost.
Although many of the present SOC chips designed for the edge computers have a lot of AI capabilities, they are still not fully leveraged for effective AI applications running using on-device computer power due to lack of support to interface with AI software models available in the market which are basically designed for cloud environment. MediaTek said its 5G smartphone SoC shipped today is equipped with APUs designed to perform a wide variety of Generative AI features, such as AI Noise Reduction, AI Super Resolution, AI MEMC and more.
MediaTek said its next-generation flagship chipset, to be introduced later this year to feature a software stack optimized to run Llama 2, as well as an upgraded APU with Transformer backbone acceleration, reduced footprint access and use of DRAM bandwidth, further enhancing LLM and AIGC performance. These advancements facilitate an expediated pace for building use cases for on-device Generative AI.
“The increasing popularity of Generative AI is a significant trend in digital transformation, and our vision is to provide the exciting community of Llama 2 developers and users with the tools needed to fully innovate in the AI space,” said JC Hsu, Corporate Senior Vice President and General Manager of Wireless Communications Business Unit at MediaTek. “Through our partnership with Meta, we can deliver hardware and software with far more capability in the edge than ever before.”
MediaTek expects Llama 2-based AI applications to become available for smartphones powered by the next-generation flagship SoC, scheduled to hit the market by the end of the year.
How Code Llama works:
Code Llama is a code-specialized version of Llama 2 that was created by further training Llama 2 on its code-specific datasets, sampling more data from that same dataset for longer. Essentially, Code Llama features enhanced coding capabilities, built on top of Llama 2. It can generate code, and natural language about code, from both code and natural language prompts (e.g., “Write me a function that outputs the fibonacci sequence.”) It can also be used for code completion and debugging. It supports many of the most popular languages being used today, including Python, C++, Java, PHP, Typescript (Javascript), C#, and Bash.
For more details visit:
https://ai.meta.com/blog/code-llama-large-language-model-coding/
News Source: MediaTek
