Well, excellent sound quality from an MP3 player means nothing if you have crummy headphones. I would usually say that this feature should have the lowest priority.
On purely audio quality, the best fully loaded PMP (i.e. video + audio) has to be the Cowon D2+. Having used one of these myself, there is nothing like it. Even
CNET love the thing! It does support both FLAC and OGG, but there is no support for APE.
As for an audio-only MP3 player, it has to be the SanDisk Sansa Clip+. Don't let price fool you because this device is hardcore. Again it features support for FLAC and OGG but not APE.
If you're looking a player with great sound quality with support for FLAC and APE, you might have to look at the Sony line. Reading forums, I've heard great things about the Sony A-840 series (although not tried it myself), but you will struggle to find it unless you live or plan to visit Asia.
What players did you have in mind?