A quick and dirty explanation (for more, try Google):
Ogg, or Ogg Vorbis files are compressed audio files, which can be sounds and/or music. They are something like MP3 files, except they are a newer format, they can be better quality, and the format is free and without patent restrictions, unlike MP3s.
Personally, what I use to play them is the VLC Media Player, which is a free, open-source media player that plays just about anything and is also cross-platform. Other media players can also play OGGs, but VLC is what I use. (Look it up, if you find pages with an orange cone, you're in the right website, it is also a small download and it is clean.) It also uses self-contained codecs, and therefore will not mess up any installed codecs on your computer (at least it shouldn't).
EDIT/UPDATE: There is also a small plug-in you can download to play Oggs in Windows Media Player, but last time I checked it is very buggy and causes issues with Windows.