In May 2023, Meta launched MTIA v1, the first-generation self-developed AI training and inference chip using TSMC's 7nm process, designed for deep learning recommendation models that are improving the various experiences of its applications and technologies.
Recently, Meta announced the launch of its latest custom chip, MTIA v2, designed for AI workloads, marking a performance improvement for Meta's AI tools to help power suggested ad models on Facebook and Instagram.
The MTIA is part of Meta's growing investment in AI infrastructure. As AI workloads become more important to products and services, this efficiency will be at the heart of delivering the best possible experience to users around the world. MTIA v1 is an important step in improving the compute efficiency of Meta's infrastructure, better enabling software developers to build AI models to facilitate a better user experience.
Meta says the company's next-generation large-scale infrastructure is being built on top of AI, including support for new generative AI products, recommender systems, and advanced AI research. As the computing needs to support AI models increase with the complexity of the models, they expect this investment to grow in the coming years.
It is reported that MTIA v2 uses TSMC's 5nm process, and the cache per PE has increased from 128KB to 384KB in the previous generation, and the frequency has been increased from 800MHz to 1.35GHz, and the dense computing power under INT8 precision has reached nearly 3.5 times that of the previous generation, and the sparse computing power has reached nearly 7 times that of the previous generation, reaching 708TFLOPS. However, MTIA v2 has a 13% larger area, while power consumption has increased by 3.6 times that of the previous generation, reaching 90W (the previous generation only had 25W).
Meta says MTIA has been deployed in data centers and is now servicing models in production.
The results so far show that the MTIA chip can handle both low- and high-complexity ranking and recommendation models, which are key components of Meta's products. MTIA can achieve higher efficiency compared to commercially available GPUs.
The MTIA will be an important component of Meta's long-term roadmap to build and scale the most robust and efficient infrastructure for unique AI workloads.
"We're designing custom chips to work with existing infrastructure and new, more advanced hardware that may be leveraged in the future, including next-generation GPUs," Meta said. Achieving our ambition for custom chips means investing not only in compute chips, but also in memory bandwidth, network and capacity, and other next-generation hardware systems. ”
The company noted that there are currently several projects underway to expand the scope of MTIA, including support for GenAI workloads.