Yes, you can remove someone from an invitation or cancel the invitation to that person so long as the person has not clicked on the invitation or registered for the event.