Rent-seeking behaviour isn't always about manipulating public policy. If public policy allows you to commit rent-seeking behaviour, you can still choose not to carry it out. (Of course, you could make a lot more money by rent-seeking from those who have no other options…)
But yes, your main point is valid: it is entirely possible to (e.g.) be a landlord without being abusive. Heck, you can even give your tenants a better deal than they could've got otherwise, if the circumstances are right. The issue is that this behaviour (in many places) is actively disincentivised, so unethical people can push out the ethical ones.
Rent seeking REQUIRES manipulation of public policy to benefit you.
Being a landlord at the 5 building scale gives you no leverage in public policy. It makes you a capitalist.