Changing the port number can boost security if it makes it easier to scan the log for attempts.

I think it is a good idea.

It is the expectation that this is enough that is the bad idea.