"Furthermore, AMD OLMo models were also able to run inference on AMD Ryzenâ„¢ AI PCs that are equipped with Neural Processing Units (NPUs). Developers can easily run Generative AI models locally by utilizing the AMD Ryzenâ„¢ AI Software."
Hope these AI PCs will run also something better than 1B model.
What is it useful for ? Spellcheck ?
Baby steps, but how useful is a 1B model these days?
It seems actual domain specific usefulness (say specific programming language, translation, etc) starts at 3B models.
Training a 1B model on 1T tokens is cheaper than people might think. A H100 GPU can be rented for 2.5$ per hour and can train around 63k tokens per second for a 1B model. So you would need around 4,400 hours of GPU training costing only $11k And costs will keep going down.