As generative AI’s influence on the consumer tech industry widens, we’ve seen a growing number of brands, manufacturers, and companies make impressive strides in innovation, at least as far as AI is concerned. Not one to be outdone, MediaTek announced that it will be showcasing its upcoming LLM demo at Mobile World Congress (MWC) 2024, scheduled to take place later this month in Barcelona. Part of MediaTek’s official announcement reads:
“Last August, we announced that we are working closely to leverage Llama 2, Meta’s open-source Large
Language Model (LLM), as part of our ongoing investment in creating technology and an ecosystem that
enables the future of AI… At Mobile World Congress 2024, MediaTek will demonstrate an optimized Llama 2 Generative AI application on-device using MediaTek’s APU edge hardware acceleration on the Dimensity 9300 and 8300 for the first time.”
MediaTek adds that it will be using its latest APUs and NeuroPilot AI platform, which will work in tandem with Llama 2 in enabling generative AI apps to run directly on-device, instead of a cloud-based method. Based on how current generative AI for mobile devices works at the moment, this does seem to be a more promising solution – for example, Google’s AI-based Magic Editor feature on its Pixel phones often requires users to initially backup their photo to the cloud before allowing them to make edits on their files.
MediaTek states that providing users with an on-device method for Generative AI is a more ideal scenario, as it opens the floodgates for seamless performance, better privacy, and security, lower latency, as well as the ability to work in cases where there’s no online connectivity. In addition, on-device integration for Llama 2 will require powerful chipsets to handle the workload involved, in which case MediaTek’s dimensity 9300 and 8300 mobile chipsets are ideal hardware solutions, at least according to the company.
As for the chips, MediaTek states that both Dimensity SoCs can support Llama 2 7B applications. In particular, the Dimensity 9300 is tailor-made for generative AI features. For instance, the 9300 features a 7th-gen APU that’s responsible for hardware-accelerated generative AI onto mobile devices – as such, MediaTek claims that the 9300 comes with 8x faster transformer-based generative Al, 2x faster integer and floating-point compute improvement, 45% more power efficiency, and even support for on-device LoRA Fusion.
Attendees to MWC 2024 who are interested in checking out MediaTek’s upcoming AI endeavors are invited by the company at Booth 3D10 in Hall 3, where the demo is scheduled to take place.
