The area of the world known as Middle-earth is a land that stretches from Belegaer in the west to the areas of Rhûn in the east and Haradwaith to the South.
Prior to the Second Age, Middle-earth included the lands of Beleriand in the west to the birthplaces of Elves and Men, Cuiviénen and Hildórien, in the east. The cataclysms at the end of the First Age and the Downfall of Númenor during the Second Age ruined these lands and they became lost.
During the Second Age, the surviving elves of Beleriand lived mostly in the areas of Lindon, Eregion, and Imladris. The Silvan elves had already been living in Lórien and Mirkwood along the Hithaeglir. The elves' numbers would dwindle throughout the Second and Third Ages, and by the start of the Fourth Age the realms of the Silvan elves would be the largest left, though in time even those faded.
Excerpt of the Map of Middle-earth (early Third Age),
from "The Atlas of Middle-earth" by Karen Wynn Fonstad