Tsinghua University and Huawei release new technology to increase text input limits for large models

Recently.Tsing Hua or Qinghua University, Beijingtogether withHuawei (brand)A team of researchers has jointly released a new technology that can improve the limits of big model text input through semantic compression techniques. The introduction of this technology provides a broader space for the application of big models in the field of text processing.

The technique draws inspiration from source coding in information theory and employs pre-trained models to reduce semantic redundancy in long inputs, which are then passed to the LLM to perform downstream tasks. This process not only improves the length of the text input, but also does not require significant computational cost or fine-tuning.

Experimental results show that the approach effectively extends the context window of the Big Language Model and is applicable to a range of tasks including question answering, summarization, a small amount of learning and information retrieval. The introduction of this technique will help to improve the performance and efficiency of big models in the text processing domain.

Huawei (brand)This collaboration with Tsinghua University demonstrates the importance of theAIThe possibility of cross-border cooperation and innovation in the field of technology. Each of the two companies has strong technical strength and R&D capabilities, and through joint research, they can learn from each other and jointly promote the development of technology.

With the continuous development of AI technology, the application of large models in the field of text processing has become more and more widespread. However, the text input limitation of large models has been a bottleneck that restricts their performance improvement. This new technology from Tsinghua and Huawei provides a broader space for the text input of large models, which is expected to promote the further development of AI technology in the field of text processing.

This new technology from Tsinghua and Huawei provides a new solution for text input with large models, which will help improve the performance and efficiency of AI technology in text processing. We expect this technology to lead to more innovations and applications in the future.