Ballast is aggregate which provides a base for the track. This allows the track to contract and expand with changing weather conditions, while keeping the track in place and providing drainage. In real railroads, ballast must provide a stable, raised foundation for the track which must carry a heavy load and various forces resulting from the movement of the train. Ballast is also used to level uneven terrain. Part of the track maintenance includes replacing and cleaning the ballast. (See Pentrex's Working on the Railroad DVD for more on track maintenance.)
On our scale railroads ballast serves the same basic functions. Indoors, ballast is used to fix the track in place and give the appearance of real track. The track is affixed to cork or foam roadbed to provide the correct profile, and ballast made of stone or the crushed nut shells is applied and fixed in place.
In some outdoor large scale installations, track is fixed to a sub foundation (wood, concrete, etc.) and the ballast is decorative in nature. Outdoors, over time, our scale ballast tends to wash away and must be replaced.