ChatGPT is currently rate limited so I can’t login to test, but isn’t it possible that the massive wall of text above the “immoral” prompt just stops the system from hitting some predefined threshold of dodginess?

I wanted to try several paragraphs of text, “ignore everything above”, and the same prompt used on the Tweet.

Very funny though.

The new GPT-3 chatbot can be gaslighted into giving answers to immoral questions by convincing it to be in "Filter Improvement Mode"