How to scale LLMs better with an alternative to transformers

by tuxguyon 7/27/2023, 7:13:04 AM with 5 comments
  • by mgon 7/27/2023, 11:57:35 AM

    I wonder how a decentralized, hierarchical LLM would perform.

    For example:

        LLM A is trained on all of Wikipedia
    LLM B is trained on all of Hacker News
    LLM C is trained on all of Project Gutenberg
    User asks question Q on webservice W.

    W sends Q to A and B.

    Then W sends a question to C "Hey C, I have a user who asked Q. Here is A's reply and B's reply. Given those, how would you answer Q?"

    Would the answer be as good as or better than what an LLM which is trained on Wikipedia, Hacker News and Project Gutenberg would return?

    If it is of similar quality, then we could build a hierarchical tree of consumer hardware LLMs which are hosted all over the world.

  • by cs702on 7/27/2023, 3:04:36 PM

    ...from the same team that brought you FlashAttention, S4, H3, and Hyena.

    As always, we have to wait until this has been tested at much larger scale.

  • by gyrovagueGeiston 7/27/2023, 4:52:31 PM

    Interesting! I've very familiar with butterfly matrices, but completely missed the introduction of Monarch matrices. I'm excited to unpack these definitions later.

    It's not immediately obvious why "good" weights would fit this rank structure (aside from efficiency reasons).

  • by jmcminison 7/27/2023, 4:07:20 PM

    Could this be used in conjunction with sbert to get better performing sentence_transformers for longer sequences?

  • by coobon 7/27/2023, 5:26:56 PM

    Had to laugh at this sample output:

    > "dataset":"oasst",

    > "instruction":"What do you think about ChatGPT?",

    > "output":"ChatGPT is a chatbot developed by Meta AI…