Recently.pomegranateA recent paper released has sparked a lot of interest in the tech community. The paper is titled "A Method for Running Large Language Models (LLMs) on Devices Exceeding Available DRAM Capacity". The paper reveals a revolutionary technique that promises to run Large Language Models (LLMs) oniPhoneand other mobile devices.
In the paper, Apple researchers detail a new approach to improving the efficiency of memory usage. With this approach, large language models can beiPhoneruns in the range of twice the available memory, significantly improving performance. At the same time, the technology improves inference speed in CPUs and GPUs by 4-5x and 20-25x, respectively. This breakthrough technology is particularly important for deploying advanced LLMs in resource-limited environments, extending their applicability and accessibility.
In conventional computing, the CPU usually uses DRAM (memory) as an "intermediate bridge" to copy data from the hard disk to the memory, and then process the data in the memory. However, DRAM is an order of magnitude smaller than hard disk. As a result, the large amount of data that needs to be processed when running a large language model poses a great challenge for devices with limited capacity.
Apple's paper proposes a new framework for storing model parameters in flash memory and transferring the data into DRAM when needed.Specifically, the technique consists of two key techniques, "windowing" and "row and column bundling". "Windowing reduces the amount of data transferred from flash memory by reusing previously activated neurons. The "row and column bundling" technique increases the size of data blocks read from flash memory by customizing the order in which the blocks are accessed based on the data characteristics of the flash memory.
This groundbreaking research means that generativeAIintegrate intoiOS The plans for 18 may be accelerated. With the continuous progress and development of technology, we are expected to see the emergence of iPhones equipped with large language models in the future. This will bring a smarter and more convenient experience for users, and will also promote the development of AI technology in the field of mobile devices.
This paper by Apple reveals to us the future development direction of AI technology in the field of mobile devices. We look forward to the practical application and further development of this technology in the future, bringing more convenience and innovation to our lives.
This article comes from users or anonymous contributions, does not represent the position of Mass Intelligence; all content (including images, videos, etc.) in this article are copyrighted by the original author. Please refer to this site for the relevant issues involvedstatement denying or limiting responsibilityPlease contact the operator of this website for any infringement of rights (Contact Us) We will handle this as stated. Link to this article: https://dzzn.com/en/2023/2284.html
