In order to solve the problem of training cost, Beijing Zhiyuan Artificial Intelligence Research Institute (BJAIIRI) and Institute of Computing Technology of Chinese Academy of Sciences (ICTCAS) and other organizations attempted to train LLMs with hundreds of billions of parameters for the first time through a growth strategy, during which the parameters are augmented from a smaller model to a larger one.
With $100,000 + 26 days, a low-cost giga-parameter LLM is born
Previous: 老黄参投，MIT华人女科学家融资2亿美元