The transformation modules of image compression algorithms based on deep learning involves complex architectures and large quantities of computation. To speed up the encoding and decoding process, a method was proposed to reduce the number of parameters and multiply-accumulation operations (MACs) of the original network with knowledge distillation while maintaining the image quality as much as possible. The original and the lightweight networks were trained simultaneously, and the lightweight network performance was improved by receiving feature information from the original network. When designing the lightweight network, group convolution was introduced to retain more feature information and reduce the number of parameters and MACs of the network as much as possible, while the number of channels was reduced. Experiments on the test datasets Kodak and DIV2K showed that, compared with the original network, the lightweight network after knowledge distillation still maintained good image quality while the amount of parameters and MACs was approximately one-sixteenth that of the original network.