Recently.AIA team of researchers at the company Graphcore has announced a new technique called SparQ Attention on the Arxiv page. This technology aims to reduce the memory bandwidth requirements of large language models, thereby improving the efficiency and performance of the models.
The research team says that the SparQ Attention technique reduces the memory bandwidth requirements within the attention block by selectively fetching the cache history. This means that when dealing with large language models, the technique can effectively reduce the model's dependence on memory bandwidth, thereby increasing the speed and efficiency of the model's operation.
What's more, this technology can be directly applied to off-the-shelf large language models during the inference process without modifying the pre-training settings or performing additional fine-tuning. This provides a more efficient and flexible solution for existing large language models, further advancing the AI field.
The research team also validated the effectiveness of the SparQ Attention technique by evaluating the performance of the Llama 2 and Pythia models in a variety of downstream tasks. The results show that the technique can incorporatelarge modelThe memory and bandwidth requirements are reduced by a factor of eight with no reduction in accuracy. This definitely proves the advantages and potential of the technology.Graphcore's release of SparQ Attention technology brings a new breakthrough to the AI field. By reducing the memory and bandwidth requirements for large language models, the technology is expected to drive the further development and application of AI technology.
This article comes from users or anonymous contributions, does not represent the position of Mass Intelligence; all content (including images, videos, etc.) in this article are copyrighted by the original author. Please refer to this site for the relevant issues involvedstatement denying or limiting responsibilityPlease contact the operator of this website for any infringement of rights (Contact Us) We will handle this as stated. Link to this article: https://dzzn.com/en/2023/2037.html
