AlphaGo is also “already knowing” the same way LLMs are. At the end of the day they are just neural nets trained for different tasks. The training process is the “figuring out” part in both cases.

There are models trained via RNN, but LLMs usually use Transformer architecture with a bit of human feedback on top which sort of uses reinforcement learning like AlphaGo

LLM and AlphaGo (DRNN) are different architectures, training regimes, and optimizers. They are not designed for the same inputs & outputs