You are probably looking for a declarative programming language, like Prolog or Curry:
https://curry.pages.ps.informatik.uni-kiel.de/curry-lang.org...
Matlab and GNU Octave can do symbolic equation solving:
https://docs.octave.org/v8.2.0/Solvers.html
https://www.youtube.com/watch?v=ZSsuuyISr6Y&t=1s
Matlab is a nice, but expensive commercial product.
GNU Octave is free software (libre and gratis):
Not in other comments, so commercial Mathematica can do it. Theorem proving languages such as coq or agda can as well, especially if you're doing really heavy math.
Probably pretty soon we could have the machines (LLMs) convert our prompts into code or solve it directly, depending on exactly what needs to be solved.
Metafont (and derivatives like Metapost) do that.
Broadly speaking, it sounds like you're looking for a Computer Algebra System; in addition to some of the recommendations other posts have made, Wikipedia provides a list: https://en.wikipedia.org/wiki/List_of_computer_algebra_syste...