Ok well I'm not particularly knowledgeable about any of these consoles so just giving a heads up, but...
The GP2X, Wiz and Cannoo (all made by Gamepark Holdings) all seem to fit your bill. Sure there are other portable emulators out there, notably the "Dingoo" and just a plain ole PSP...