Most of the problems with overclocking comes from the RAM. Some people say they can't overclock beyond 133MHz. My old GP32 couldn't overclock beyond 133MHz, which is one of the reasons why I sold it. I don't know if these unfortunate people ever opened up their GP32s, but I think some units were shipped with a different or faulty RAM chip that becomes unstable beyond 133MHz. The RAM is only stable up to around 166MHz. Note that GP32s sold now should have no problem, and I have not heard of any problems. Especially FLU units, since they are newer then the non-FLU.
Overclocking itself is actually the same as it would be on a computer, yet we don't have your standard BIOS, so we need to tell the CPU directly when we want to overclock.
The default speed is 40 or 66MHz, and Gamepark assures proper functionality up to 133MHz.
The lower the speed, the longer the batteries will last. The CPU itself isn't the largest battery hog, but the slower it goes, the slower all of the devices will be polled and updated. I did an experiment a while back, and the lowest speed setting (20MHz) with the LCD turned off can last up to about 35 hours! 133MHz can last around 10 hours on rechargable batteries. The rule here is decrease the battery life by about 30 minutes for every 16MHz speed increase.