StarCoder is a 15B LLM for code with 8k context and trained only on permissive data in 80+ programming languages.

Release thread: https://twitter.com/BigCodeProject/status/165417494197606811...

Model: https://huggingface.co/bigcode/starcoder

Paper: https://drive.google.com/file/d/1cN-b9GnWtHzQRoE7M7gAEyivY0k...

Also available in HuggingChat: https://huggingface.co/chat

License text: https://huggingface.co/spaces/bigcode/bigcode-model-license-.... Attachment A at the very end has the usage restrictions.

Paper: https://drive.google.com/file/d/1cN-b9GnWtHzQRoE7M7gAEyivY0k...

Evaluation results are impressive!

https://huggingface.co/spaces/bigcode/bigcode-playground

Can someone explain to me the "Model Formats" section at the end of that page. It makes sense to me as a description of how to format the training data, but it also says "Use these templates to explore the model’s capacities"

How would you make use of this prefix format in a prompt?

  <reponame>REPONAME<filename>FILENAME<gh_stars>STARS
  code<|endoftext|>

Is it possible to load the model for a particular language. Thus minimising the required resources.

The org page has a link to https://huggingface.co/bigcode/starencoder "Encoder model trained on TheStack." which is currently a 404

Just curious what that is vs the main https://huggingface.co/bigcode/starcoder model

I'm excited to see all the model's implementations that are yet to come. Refact.ai seems to be working on code transformation tools and chat inside Jetbrains and VSCode powered by StarCoder https://refact.ai/blog/2023/self-hosted-15b-code-model/

The training cluster is super impressive, any ideas on how to fine tune StarBase for a smaller language than python on something more attainable?

i was not able to find the 80 programing languages it supports using my phone. you would think this would be front and center somewhere.

I might have missed it, what are the minimum hardware requirements needed to run this?

So many things are 404 on this announcement. The demo looks really amazing though, I’m pretty excited to try this out locally.