> there's a natural "limit" on the happiness I can enjoy at any given moment
This is roughly my experience, although, you can choose between peaks & valleys vs emotional stability. Highs are generally followed by lows and vice versa. The classic rockstar chooses to swing wildly between drugs/sex/crowds/etc. and addiction/depression/death while the buddhist munk reaches nirvana by maintaining an even keel.
Keep in mind there are other life considerations besides 'happiness' of course:
1. retrospective life satisfaction
2. responsibility
3. morality
4. procreation
5. etc.
I love the question and the sibling reply by tgflynn. Yes, this happens, it's normal, especially in your 30s. Great isn't it!? Stopping "living out other people's dreams" (the ones you were told you should have) is a major milestone of maturity. And once past it on the path, it keeps getting better. Add a sprinkle of daily gratitude, an inner revolution of lowered expectations, some celebration of the ordinary miracles around us. Find something to move towards, but not so unambitious that you'll ever get there. And one or two people to love.
You can choose between playing evolutionary biology games and hacking your reward function. Hacking your reward function is easy, so many paths towards achieving that (drugs,hobbies,self-hypnosis,spirituality, philosophy, meditation…) winning at the evolutionary biology games isn’t hence why most people drift between trying to win and consoling themselves with a quick hack when they lose. You don’t have to be Floyd Mayweather on one side or a monk on the other, anything in between you’re happy with is fine, as long as you know both paths are open to you.
Many forms of spirituality, notably Buddhism and Advaita Vedanta, teach that happiness is not something that is produced by things we do or have but is inherent in our true nature. In fact our activities only interfere with our ability to recognize that nature and experience the bliss it entails.
As someone who once had many great plans and hopes for my life, all of which have come to naught, I can attest that I have found this to be true. The person I was 10 or 20 years ago would have rather died than live the life I ended up with, but today my most common emotional state is one of bliss.