yeah, you need to get a regulated one. i've seen unregulated 3v ones go up to 5v under zero load. it's 166% higher than it should be! those are usually the kind you get from flea markets or shady shops in chinatown.
You can replace the startup sound with a silent one. just make a 2.7 second wav recording at 22khz. Name it "gp2xsound.wav" and put it in the root of the sd card. Put it in your GP2X and hold Start+Select when it starts. It'll replace the existing sound.
This is a great program! Wayy better than the crappy built in one. Do you think you can add an auto-fix function to fix formatting issues in Project Gutenberg books?
I also have a 4gb Transcend 150x SD card. It works with the $3 reader just fine.
You should try disabling USB Enhanced Host Controller in your Device Manager and reformatting the card using USB 1.1. Afterward, just reenable the Enhanced Host Controller.