Following the opening of the Dark Portal, the nations of Azeroth and Khaz Modan were conquered by the Horde. The refugees from Azeroth, led by Lord Anduin Lothar, fled across the sea to Lordaeron. There, Lothar convinced the leaders of the human nations, as well as the dwarves of Ironforge, gnomes of Gnomeregan, and high elves of Quel'Thalas, to join forces in the Alliance of Lordaeron.
Lordaeron specifically was a pretty overpowered kingdom, setting the standards for a faction that could actually hold many races without much problems. Many high elves were in Lordaeron, mostly in the shape of noblesmen and counselors due to their vast skills, so there are cases of high elves in Lordaeron. Reclusive, yes, but they were there in Lordaeron!