Yes, though you can also compare MS Office to something like TeX, and find that either option is inadequate, depending on where your focus lies.

I haven't used Octave, but I find that Python + PyLab is a great replacement for Matlab in many cases. It's certainly a subset of Matlab in terms of features, but the things it does do, it does really well.