I recommend Letting the R5 of a guild the authority to cancel a rally
You must be new to the game and want to cancel all the annoying "fake rallies". It's part of the game. It's a way of hiding your troops. Just like food blinking red it's very annoying at first but eventually you will get used to it.