I think the problem is that Pandora has a USB2 port and most mice and eyboards use USB1 spec for compatibility...
So... you need a hub to plugin your usb mouse.
Also, plugging in a USB1.1 device (such as mouse or keyboard) crashes the USB host, and it will no longer recognize things plugged into it, even if they are USB2.0 compatible. The only way to recover is to reboot the Pandora, unfortunately. I'm pretty sure this is a hardware flaw with the OMAP chip itself, since the BeagleBoard does the same thing.