I think it's just that the Avenger's are Earth's Mightiest, and they have the heroes from Earth who have consistently been portrayed as some of the mightiest. The only ones I can think of who should be included in "Earth's Mightiest" and aren't from North America (or are X-Men) are T'Challa and Brian Braddock.
I think it's mostly just a factor of the fact that since Marvel is an American company, most of their mighty heroes are American. If the international scene was fleshed out more beyond Wakanda and MI:13 and I knew more international heroes, then maybe I would see it like other people do, but as of right now I'm fine.
If anyone could put forth some more diverse heroes (excluding X-Men) that they think should be on the team I would appreciate it.