You can also argue that even if time travel is impossible, as conventional thinking still tends to assert, then the lack of intervention in our present, to say nothing of our past, is no guarantee that at some future time (perhaps literally tomorrow), some rogue members of an alien civilization will view Earth as fair game from close range. And since such members might be their equivalent of motorcycle rebels, raisin' hell for the sake of it, well, there goes the argument that a sufficiently advanced race wouldn't take advantage of an immature one such as ours.
The argument that a race sufficiently advanced to travel interstellar distances wouldn't have a chance of being defeated is better, but there's no saying whether the above argument might still apply. For example, on Earth today, the existence of high-tech militaries is not absolute assurance that mercenaries wouldn't want to try their hand at doing small-scale operations that don't require anything more advanced than Rugers or AK
Hence, if some interstellar equivalent of the bad guys in those Mad Max
movies comes along, we -- meaning humanity -- might still have a chance of holding them off, at least until the interstellar cavalry comes along. If it ever does.
[Edited 2006-05-08 12:34:35]
What's fair is fair.