>Usually it's got some modified permissions which allow it to run as another user (commonly root). They should be designed to be secure against an attack but nothing is bug-free. So you know what you do? You remove those permissions. Suddenly it's just a normal executable with the same capabilities as any other executable on the system (of which there are many even on a minimal system).
Or you don't even leave it on the system, which means that it can't exploit the hole. If you're aware of the hole, you don't put it on the system in the first place, and if you don't know about it, it's impossible for someone to use an exploit that doesn't exist.
That's rather simpler than leaving it on there, and hoping that permissions work. Nothing's bug free, so why take the chance?
First, there's no chance about it because you can't do anything with it once it's a normal executable. There's no "hoping" about it - remove special permissions and audit the whole FS and it can't be used for an attack (any more than any other binary on a minimal system could be). That being said, you do it to improve the efficiency of the system for the users and reduce time to troubleshoot issues.
Or you don't even leave it on the system, which means that it can't exploit the hole. If you're aware of the hole, you don't put it on the system in the first place, and if you don't know about it, it's impossible for someone to use an exploit that doesn't exist.
That's rather simpler than leaving it on there, and hoping that permissions work. Nothing's bug free, so why take the chance?