I've been trying my hands at implementing an emacs package for inline completions with this. I have it mostly working and performance is good enough but I haven't been blown away by the quality of its suggestions unfortunately. Which I guess is expected from a 1.5B model.
I'd love to see them making a larger model in the 10-20b range maybe? I know most people wouldn't be able to run that on their machines, but some could.
Running on ollama locally on NVIDIA Spark GB10. Tried it also with vLLM. Pretty fast.
If I can make it clean and decent I will. I might look at again after work and see if I can tune it up. It was a bit flake and I wasn't blown away by the interaction.
I'd love to see them making a larger model in the 10-20b range maybe? I know most people wouldn't be able to run that on their machines, but some could.
Running on ollama locally on NVIDIA Spark GB10. Tried it also with vLLM. Pretty fast.