Unilm is a large-scale self-supervised pre-training model across tasks, languages and modalities. It is pre-trained by self-supervised learning, which enables the model to be transferred to different tasks and languages, with wide application value. The design goal of Unilm is to provide a unified pre-training model that can handle various natural language processing tasks, such as machine translation, text summarization, question answering, etc.