Is there a better page to link to? I cannot even see "Kosmos" on this page!
Edit: Ah, looks like this is the link to the paper: https://arxiv.org/abs/2302.14045
It was discussed yesterday: https://news.ycombinator.com/item?id=34965326
It can even solve IQ tests...I mean, how much further are we moving the goal post?
Is there a model that can solve differential equations symbolically and numerically? Most of modern engineering just boils down to diff.eqs whether ordinary or partial. It's our current best method to reason about stuff and control them.
I like this feature they are working on
https://arxiv.org/abs/2212.10554
as I'd say the most obvious limitation of today's transformers is the limited attention window. If you want ChatGPT to do a good job of summarizing a topic based on the literature the obvious thing is to feed a bunch of articles into it and ask it to summarize (how can you cite a paper you didn't read?) and that requires looking at maybe 400,000 - 4,000,000 tokens.
Similarly there is a place for a word embedding, a sentence embedding, a paragraph embedding, a chapter embedding, a book embedding, etc. but these have to be scalable and obviously the book embedding is bigger but I ought to be able to turn a query into a sentence embedding and somehow match it against larger document embeddings.
I don’t trust any report of model performance from papers, unless there is a publicly accessible demo. It is way too easy to test things the model has trained on and for the model to then completely fall flat when used by people in the real world.
Another one that looks even more compelling:
Multimodal Chain-of-Thought Reasoning in Language Models, https://arxiv.org/abs/2302.00923
By building in chain of thought and multimodal learning, this 1B parameter model beats GPT-3.5's 170B parameter model.
It's worth noting that this is a comparatively small model (1.6B params from memory).
It'll be interesting what capabilities emerge as they grow that model capacity.
Hmm... LLMs / MLLMs might be truly a unified input / output interface of a would-be AGI, I think.
At Microsoft:
Hey why don't we call our new LLM Cosmos? That's taken by the Azure Cosmos DB guys Damn it... how about Kosmos-1 ?
Did anyone else initially read that as `Kosmos~1`, and wonder what the full name of the project was?
Anyone know if this will be an openly available model?
The examples in the paper are pretty impressive. There is an example of a windows 11 dialog image. The computer can figure out which button to press given the desired outcome of the user. If one where to take this model and scale it, I can see an advanced bot in <5 years navigating the web and doing work based on a text input of a human purely by visual means. Interesting times.