https://github.com/shellinabox/shellinabox might be close to what you want.

JupyterLab has this and using special keys like esc in vim even works