I dont have much experience with the new small digital recorders (i have a lottt of experience with other types of digital recording devices) However, I'll share with you the little knowledge I did acquire from using a couple of them to record friends of mine playing concerts.
There's one thing you want to avoid getting (if you're planning on recording for very long periods of time, over a 10 mins or so at a time) Make sure your device isn't the kind that writes the file to disk AFTER its done recording. Many of them do this. For instance, if you record a file for 45 mins to an hour once you press the "stop" button it will then write the file to disk (or rather to your memory card)
You want to get the kind that writes directly to your card as it records. This is important if you're doing long recording times (more than 10 mins or so at a time is a problem for a lot of those units. they will lockup trying to write the huge file often) So make sure if you want to record very long files that you get one that writes directly to the card as it records.