The map would 'twitch' back and forth, and most, if not all, of the misc_models were misplaced on the map, along an x slide.
I destroyed brushes one at a time trying to find the corrupted brush(es), it ended up needing a complete map wipe to get rid of it (and yes, I'm sure it wasn't just the last brush.