A recent blog post by Dosu's CEO, Devin Stein, sheds light on how they significantly increased accuracy without resorting to complex and error-prone prompt engineering. Instead, they leveraged user feedback and transformed it into few-shot examples for continual learning. Here's the gist of why this matters:

⇢ Dosu automates repetitive engineering tasks like labeling tickets and PRs, reducing interruptions for engineers.

⇢ By collecting user feedback and converting it into in-context learning examples, Dosu adapts continuously and maintains high accuracy.

⇢ Prompt engineering and fine-tuning come with downsides like complexity and data drift, which their approach avoids.

⇢ Their continual learning method is simple: they collect corrections, store them as examples, and use them during task execution to improve accuracy.

The results? Dosu’s label accuracy jumped by over 30%. Definitely worth a read: