CodeT5 is a series of open code large language models released by Salesforce Research for code understanding and generation, including two versions, CodeT5 and CodeT5+. The related papers are "CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation" and "CodeT5+: Open Code Large Language Models for Code Understanding and Generation".
Purpose: The CodeT5 and CodeT5+ models can serve as AI programming assistants to improve the productivity of software developers. Within Salesforce, the team built an AI programming assistant demo in the form of a VS Code plugin based on CodeT5, providing three capabilities: text-to-code generation (generating code from natural language descriptions), code autocompletion (completing an entire function given the target function name), and code summarization (generating a function summary in natural language).
Main capabilities: The models are based on the T5 architecture and use identifier-aware training, making them suitable for tasks such as code summarization, defect detection, and clone identification. The project released two large CodeT5 checkpoints: Salesforce/codet5-large and Salesforce/codet5-large-ntp-py, introduced by the CodeRL paper. In addition, it also released fine-tuned checkpoints for all downstream tasks in the paper, as well as a CodeT5-base fine-tuned checkpoint Salesforce/codet5-base-multi-sum for multilingual code summarization.
Usage: The project code and models are open source and can be obtained through the GitHub repository. Model checkpoints are released on HuggingFace. Specific installation commands, dependencies, and running steps are not yet provided.
Applicable scenarios: Suitable for research and development related to code understanding and generation, such as code summarization, defect detection, clone identification, text-to-code generation, code autocompletion, and more.
Limitations: The code is released under the BSD-3 license, but users are required not to use this software to promote or profit from the following: violence, hate and division, environmental destruction, human rights violations, or harm to people's physical and mental health. Users are encouraged to contact codeT5@salesforce.com to inform them of the application scenarios and to use appropriate documentation when developing high-risk applications. If you have questions, suggestions, requests, or bug reports, please create a GitHub issue.
