Because people are mobile and infrastructure is not, substantial shifts in population have significant effects on infrastructure at both ends. A town or city that is growing unexpectedly (e.g., many towns in the Dakotas in the current gas & oil boom) must invest heavily in new infrastructure, placing great strain on its finances and other resources. One facing a sudden decline in population (e.g., Detroit) can lose the tax base needed to pay for the maintenance on what it has already built, creating a vicious circle that reinforces the decline.
Although the examples given are growth in rural areas and decline in a big city, the predominant movement of population in the US for the last half century has been in the opposite direction. We have become one of the most urbanized nations on earth, leaving large rural areas so underpopulated that they are straining to maintain the minimum infrastructure needed to support the handfuls of families who are left. Cities, meanwhile, are often straining to accommodate additional millions, often with 100 to 200-year old sewers and water systems, and street grids and traffic management systems that are completely inadequate.