CoDesc: A Large Code–Description Parallel Dataset

[Download Paper][Source Code]

Translation between natural language and source code can help software development, by enabling developers to comprehend, ideate, search, and also write computer programs in natural language. Despite growing interest from the industry and the research community, this task is often difficult for the lack of large standard datasets suitable for training modern deep neural models, standard noise removal methods, and standard benchmarks. This leaves researchers to collect new small-scale datasets, resulting in inconsistencies across published works. In this study, we present CoDesc - a large parallel dataset containing 4.2 million Java methods and natural language descriptions. With extensive empirical analysis, we identify and remove prevailing noise patterns from the dataset. We demonstrate the proficiency of CoDesc in two complementary tasks for code--description pairs: code summarization and code search. We show that the dataset helps improve code search by up to 22% and achieve the new state-of-the-art in code summarization. We also show its effectiveness in pretrain-finetune setup, opening the possibility of building pretrained language models for source code. We release the dataset, a data processing tool, and a benchmark to encourage future research.