Today's update adds the option to play internet radio stations in-game, as an easy way to listen to music while puzzling.
To play music, either select the "Music" option from the menu or hit the musical notes icon while playing a puzzle. The Music page will appear. From there you can choose from over 450 internet radio stations. You can filter and sort the stations by name or genre to narrow the list, and adjust the music's volume.
If you're looking for a calm, instrumental background to puzzle by, may I suggest my new favorite, "Peaceful Currents Radio." :)