Oscar is an open-source contributor agent architecture project aimed at improving open-source software development by creating automated assistance (i.e., "agents") for open-source maintenance. The project believes there are many opportunities to reduce the tedious work involved in maintaining large and small open-source projects.
Large language models (LLMs) are capable of semantic analysis of natural language (such as issue reports or maintainer instructions) and converting between natural language instructions and program code, which creates new opportunities for agents to interact more smoothly with people. LLMs may be only a small part (but a key part) of the overall picture; most of the agent's operations will be executing standard, deterministic code.
Unlike many development-oriented LLM applications, Oscar does not attempt to enhance or replace the code-writing process at all. After all, writing code is the fun part of software development. Instead, the philosophy is to focus on the uninteresting parts, such as handling incoming issues, matching issues to existing documentation, and so on.
Oscar is highly experimental. We do not yet know where it will go or what we will learn. Even so, our first prototype, the @gabyhelp bot, has already had many successful interactions in the Go issue tracker. Currently, Oscar is developed with the support of the Go project. At some point in the future, it may (or may not) become independent as a separate project.
The specific goals of the Oscar project include: reducing the effort maintainers spend resolving issues (note that resolving does not always mean fixing); reducing the effort maintainers spend resolving change lists (CLs) or pull requests (PRs) (note that resolving does not always mean submitting/merging); reducing the effort maintainers spend resolving forum questions; and enabling more people to become efficient maintainers. Its non-goal is to automate coding away. Instead, we focus on automating away maintainers' tedious work.
Maintainers' tedious work is not unique to the Go project, so we aim to build an architecture that any software project can reuse and extend to build customized agents suited to its own project's needs. Hence Oscar: the open-source contributor agent architecture. What that specifically means is still being explored.
So far, we have identified three capabilities that will be important components of Oscar: indexing and presenting relevant project context during contributor interactions; using natural language to control deterministic tools; and analyzing issue reports, CLs/PRs, and group discussions to help improve them in real time during or shortly after submission, and to appropriately label and route them.
It is reasonable for LLMs to contribute here, because open-source maintenance is essentially interacting with people using natural language, and natural language is what LLMs are best at.. Therefore, it is not surprising that all of these have LLM-related components. On the other hand, all of these are also supported by a large amount of deterministic code. Our approach is to leverage the areas where LLMs excel—semantic analysis of natural language and translation from natural language to programs—and rely on deterministic code to do the rest.
The following sections examine these three important capabilities in turn. Please note that we are still experimenting and expect to identify more important capabilities over time.
Indexing and presenting relevant project context: Software projects are complex behemoths. Only at the very beginning is it possible for a maintainer to keep all important details and context in their head, and even then, if possible, these exist only in one person's head, which does not help when a new contributor arrives with a bug report, feature request, or question. To address this problem, maintainers write design documents, API references, FAQs, man pages, blog posts, and so on. Now, instead of having to provide context directly, maintainers can provide links to written context that already exists. Acting as a project search engine is still not the best use of a maintainer's time. Once a project grows to medium size, no single maintainer can keep track of all potentially relevant context, making it even harder to act as a project search engine.
On the other hand, LLMs have proven to be an excellent platform for building project search engines. LLMs can analyze documents and generate embeddings, which are high-dimensional (e.g., 768-dimensional) floating-point unit vectors with the property that semantically similar documents are mapped to vectors pointing in similar directions. Combined with a vector database to retrieve vectors similar to an input vector, LLM embeddings provide a very effective way to index all the context of an open source project, including documentation, issue reports, CLs/PRs, and forum discussions. When a new issue report arrives, the agent can use the LLM-based project context index to identify highly relevant context, such as similar previous issues or related project documentation.
Our prototype agent implements this capability and replies to new issues in the Go repository with up to ten highly relevant links, adding context to the report. (If the agent cannot find anything that seems relevant enough, it stays silent and does not reply at all.) In the first few weeks of running the agent, we identified the following benefits of such an agent: the agent presents relevant context to contributors. It is common for new issue reports to duplicate existing issue reports: a new bug may be reported multiple times within a short period, or a non-bug may be reported every few months. When the agent replies with links to duplicate reports, contributors can close their new report and then follow the earlier issue. When the agent replies with links to reports that look like duplicates but are not actually duplicates, contributors can provide additional context todistinguish their reports. For example, in golang/go#68196, after the agent replied with a near-duplicate, the original reporter commented: "Good bot :). Based on the discussion in this issue, I understand that what was suggested here may not be possible. If so, I still suggest keeping the issue open for a while to see how many Go users care about this problem." Another example, in golang/go#67986, after the agent replied with an exact duplicate, the original reporter commented: "Ugh, I spent quite a bit of time searching existing issues. Not sure how I missed [that]."
The agent even presents relevant context to project maintainers. Once a project reaches medium size, no one can remember all the context, even highly focused project maintainers. When the agent replies with a relevant report link, this eliminates the time the maintainer spends looking it up. If the maintainer has completely forgotten the relevant report, or has never seen it (it may have been handled by someone else), the reply is even more helpful, because it can point the maintainer in the right direction and save the effort of repeating the analysis from the earlier issue. For example, in golang/go#68183, a project maintainer filed a bug about the Go compiler mishandling certain malformed identifiers. The agent replied with a similar report link for the same bug filed almost four years earlier but triaged as low priority. The added context made it possible to close the earlier bug and provided an argument for raising the priority of the new bug. Another example, in golang/go#67938, a project maintainer filed a bug about the Go coverage tool causing the compiler to report incorrect subline position information. The agent replied with an earlier related issue from ten years ago (incorrect line numbers) as well as a newer issue about coverage not reporting subline position information at all. The first bug was important context, and the "fix" for the second bug was the root cause of the bug in the new report: the subline position information added at the time was added incorrectly. These links pinpointed the exact code of the bug. Once identified, it was also easy to determine the fix.
The agent interacts with the bug reporter immediately. In all previous examples, the agent replied only one or two minutes after the report was submitted, which means the reporter was still available and sufficiently engaged to give a meaningful response: adding details to clarify the suggestion, closing the report as a duplicate, raising the bug's priority based on past reports, or identifying a fix. In contrast, if hours or days (or longer) pass after the initial report, the original reporter may no longer be available, interested, or able to provide context or additional details. Immediately after an error report is the best time to interact with the reporter and refine the report. Maintainers cannot be expected to be involved in this work all the time, but an agent can.
Finally, note that presenting project context is extensible, so a project can incorporate its context regardless of form. Our prototype's context sources are customized for the Go project, reading issues from GitHub, documentation from go.dev, code reviews from Gerrit, and discussions from Google Groups (such as golang-nuts), but the architecture makes it easy to add additional sources.
Using natural language to control deterministic tools: The second important agent capability is using natural language to control deterministic tools. As open source projects grow, the number of useful tools increases, and tracking all the tools and remembering how to use each one can be difficult. For example, our prototype includes a general facility for editing GitHub issue comments to add or [original text truncated]
