Fine-tuning an LLM on your code for better code completions

Show HN: Fine-tuning an LLM on your code for better code completions

Fine-tuning an LLM on your code for better code completions operates in show hn launch.

Profile

Hey folks,Just wanted to share an interesting experiment I ran to see what kind of performance gains can be achieved by fine-tuning a model to code from a single repo.Results: The fine-tuned model achieves a 47% improvement in the code completion task (tab autocomplete). Accuracy goes from 25% to 36% (exact match against ground truth) after a short training run of only 500 iterations on a single RTX 4090 GPU.This is an interesting result because it shows that there are significant gains to be had by fine-tuning to your own code.Full details of the experiment: https://prvn.sh/build-your-own-github-copilot/Highlights of the experiment: - Model: qwen2.5-coder 14b, 4-bit quantized - Training dat

The company sits in the show hn launch segment, where it competes on its core product or service.

Directory data places it at the Show HN launch stage.

Website: prvn.sh/build-your-own-github-copilot