Tencent releases an "extremely small" open-source model, which in reality only takes up 600MB of storage space.

On February 10th, Tencent released an open-source "minimal" model HY-1.8B-2Bit aimed at consumer hardware scenarios. This model is based on a small-sized model with 1.8 billion parameters, utilizing 2Bit quantization technology to effectively reduce the parameter size to around 0.3 billion. The actual storage footprint is only about 600MB, smaller than some common mobile phone applications, marking a new breakthrough in edge deployment. The model is based on the industry-leading 2Bit edge quantization solution developed by the Tencent Mix team. Through 2Bit quantization-aware training on the HY-1.8B-Instruct model, the model size is reduced to 1/6 of the original accuracy model, while also increasing inference speed by 2-3 times on real edge devices, significantly enhancing user experience. In terms of capabilities, the model retains the original chain of thoughts and can provide appropriate depth of reasoning for tasks of different complexity. This is the industry's first practical implementation of 2Bit industrial-level quantization for edge models. With the prevalence of large language models, the challenge for the industry is how to apply models on devices such as smartphones, headphones, or smart home appliances, especially considering the increasing demand for offline deployment and privacy in many applications. This calls for more small yet powerful models that can run efficiently on the edge. Currently, the model is available on the open-source modeling community Huggingface and Github.