On my windows machine I also have this issue.
I think I may know where at least part of the problem is:
Back when we included the Xvid codec library in the dependencies to compile the game, I remember that version of Xvid had some issues - I've been following development of Xvid a little for at least a couple of years now - and shortly after we included what was then the current Xvid code in the packages used for Windows the Xvid team released a new bugfix version that fixed various issues including ones that would cause crashing and other unstable behavior.
I don't think we ever updated the code we used with the newer version after it became available, so that on Windows we were still building with an unstable and buggy library.
Mattn probably doesn't have the issue on Linux because many Linux distros grab updates to development libraries and other resources used to compile software - I know at least Debian and all the *buntu versions do this.
By comparison, Windows only gets updates natively from Microsoft, which only provides updates for the OS itself, other MS products (Office, Powerpoint, Visual Studio, etc.) and sometimes drivers that are approved by Microsoft (which are typically out-of-date).
Third-party stuff such as Xvid, OGG codecs, etc., and especially any open-source stuff and other libraries, even MinGW, do NOT get automatically updated with Windows Update. If Microsoft has nothing to do with it, it must be updated by hand, meaning checking the website of a library development team and downloading a new version when one comes out.
Linux can update nearly all of its components via an automatic updater, Window doesn't, and this is a sucky drawback to Windows.
I have a feeling that the Windows libraries used to build the Windows version are still using dated versions of libraries, while the Linux users are able to easily compile and build with the latest versions thanks to their periodic updates. This may include the old, buggy, and unstable Xvid codec library (which has been updated to new versions several times since when we first included it in the game).