Input Lag


Rockthesmurf

Advanced Member
Joined
Jul 18, 2003
Messages
1,115
Age
40
Location
Manchester, UK
Website
Visit site
Hi there,


Milkshake's recent contribution of added Paid Application support to the repository he created (http://boards.openpa...ed-in-the-repo/) has left me in the position where I have a commercial game I would like to release on the Pandora, and it is all just about ready to go, except there is an input lag problem. This problem has existed since day one for me with my Pandora development. Milkshake has kindly tested build on his device and also experiences the same problem. For many weeks (well months) now my Pandora has been with Evil Dragon awaiting repair, so I am unable to really do anything more myself to look into the input lag issue, although in honesty I do not know what to test next except trying to make an extremely cut down reproduction case.


First of all, I should probably mention the game in question is KAMI RETRO, further details about the game can be found here:


iPhone - http://itunes.apple....d424188363?mt=8


iPad - http://itunes.apple.com/us/app/kami-retro-hd/id433330514?mt=8


Android - https://market.andro...ro.global&hl=en


WebOS Handsets - https://developer.pa...games.kamiretro


WebOS Tablets - https://developer.pa...mes.kamiretrohd


Right now I do not know for sure what price it will be sold for on Pandora, although £1.99 is a possibility. Anyway, I am getting ahead of myself, until this input issue is fixed I can not proceed. I have already spoken to a couple of developers about this issue, and I will post my question below, on the off chance anyone can offer any suggestions:


My problem seems to be that input events don't always come through for the touch screen, or at least sometimes are delayed by a substantial amount of time.


My touch screen event processing code (which runs once a frame) is quite straight forward, something along the lines of:



Code:
while ( XPending( m_XDisplay ) )

{

	XNextEvent( m_XDisplay, &xev );

	switch ( xev.type )

	{

    	case ButtonPress:

    	{

        	// Pass event onto generic input handler which works without any problems on many other platforms

    	}

    	break;


    	case ButtonRelease:

    	{

        	// Pass event onto generic input handler which works without any problems on many other platforms

    	}

    	break;


    	case MotionNotify:

    	{

        	// Pass event onto generic input handler which works without any problems on many other platforms

    	}

    	break;	}

}


The main problems I experience are:

  • Sometimes events either do not seem to come through at all, or come through very late
  • Sometimes I get a motion event without a corresponding down event (in which case I just pretend a down event has come in, as we shouldn't be able to get motion without a down event first)
  • Input is generally a bit laggy (but then again, it is on a number of devices, if it is at least consistent it isn't *as* bad)


My application is using OpenGL (ES 1) for rendering, AL for audio, and x11 for initial screen setup/event handling. Before I sent my Pandora off for repair I did try switching over to SDL for input event handling, however this didn't work for me (I believe I will need to change my screen setup code to use X11 too for this to work).


It's a long shot, but could anyone offer any suggestions as to what is causing this bug? I'd love to find a way to fix it, without having my Pandora, and get the game on the AppStore so everyone can check it out (quite possibly I could provide a Lite version too, for free, for people to play if they do not want to pay for the full game).


Due to the nature of the game (flicking characters around on the screen) it is rather crucial that input events come in (and ideally with as little lag as possible). I should also note my company, Paw Print Games, owns a few other licenses, and I have tried running these games on Pandora (using the same underlining engine) and the problem occurs in the other projects too. I'm very much hoping to be able to put not only KAMI RETRO but future projects on the Pandora, I don't necessarily expect for them to even pay for the development costs initially, but I'd like to be able to do something to help the Pandora.


Best regards,


Steve
 
Last edited by a moderator:
You could try to use the keys in joystickmode. Maybe that would help?
 
You could try to use the keys in joystickmode. Maybe that would help?

The nature of the game is you sometimes have to control multiple characters at once, a sort of frantic 'flick this guy from this ledge to this bounce pad, quickly flick another guy over a spike trap and pull a lever, flick a third guy onto a moving platform' so it is not something that is easily controllable with the keyboard/joystick. It really is designed to be played with the user interacting with the screen.


Some of the other projects we have here at Paw Print Games could be made to work with the joystick, and this is certainly something we will explore it we bring these titles to the Pandora market, however initially we are just looking to get KAMI RETRO released, as we have already put in the initial developer time for this project, and want to see what returns (if any) it makes, to work out the feasibility of releasing more titles on the Pandora (I want the Pandora to succeed, but I can't do that at the expense of my company failing).


Steve
 
Hmm, the touchscreen registers how hard you press it.


Maybe it's set so that it will only register if you press it really hard?


MyPaint uses the sensitivity feature, maybe that can be of help?
 
Hmm, the touchscreen registers how hard you press it.


Maybe it's set so that it will only register if you press it really hard?


MyPaint uses the sensitivity feature, maybe that can be of help?

Potentially, I wonder if that could help with the fact sometimes events do not appear to come through at all? I presume MyPaint is open source, so I could take a nose through there sources for it.


Steve
 
Hmm, the touchscreen registers how hard you press it.
Really? I never knew that! Coool


Back on topic, have you tried timing your screen rendering? If the rendering time ends up being quite long, this will cause your events to be delayed (unless the rendering is threaded out of course)
 
Back on topic, have you tried timing your screen rendering? If the rendering time ends up being quite long, this will cause your events to be delayed (unless the rendering is threaded out of course)

The game runs at 60 FPS solid, the input lag issue doesn't seem to have any bearing on frame rate from what I remember. Also the input problems are not there on other platforms (iPhone, iPad, Android, WebOS, Bada, BlackBerry QNX), so I do not believe there are any issues with the game code, or the general handling of input events.


Steve
 
are all those other deviced capacitive touch screens though? if so EvilDragon could be onto something with the pressure sensitivity of the resistive touchscreens in our pandoras.
 
are all those other deviced capacitive touch screens though? if so EvilDragon could be onto something with the pressure sensitivity of the resistive touchscreens in our pandoras.

Well, I noticed it's there in BrainParty as well. It has the same lag when you click too fast.


When clicking too fast, you don't press very hard, so maybe it's set to not register at that point.
 
The game runs at 60 FPS solid, the input lag issue doesn't seem to have any bearing on frame rate from what I remember. Also the input problems are not there on other platforms (iPhone, iPad, Android, WebOS, Bada, BlackBerry QNX), so I do not believe there are any issues with the game code, or the general handling of input events.
I used to not have these input lag at all.


My tipical minesweeping setup is : audacious + panmines (surprisingly ;D)


I had these lag in the input while I used to not have them... My only change is the HF6 flash (I was running an initial flash + pnd updates)
 
I am pretty sure when I did some tests the input lag issue/dropped events was noticeable when pushing very hard and very lightly, that said I'd still be interested to see some code that deals with pressure sensitivity (I'd like to just switch it off/switch the threshold right down, and make another build).


I can't find a link to the Pandora source for MyPaint, does anyone know where this might be? Or is it simply built from the mainline source over here: git://gitorious.org/mypaint/mypaint.git

The game runs at 60 FPS solid, the input lag issue doesn't seem to have any bearing on frame rate from what I remember. Also the input problems are not there on other platforms (iPhone, iPad, Android, WebOS, Bada, BlackBerry QNX), so I do not believe there are any issues with the game code, or the general handling of input events.
I used to not have these input lag at all.


My tipical minesweeping setup is : audacious + panmines (surprisingly ;D)


I had these lag in the input while I used to not have them... My only change is the HF6 flash (I was running an initial flash + pnd updates)

I'm certainly interested in what you are saying here; so an earlier flash didn't exhibit lag, but the most recent does? That is interesting, it would certainly back up my initial hunch that the problem is firmware/driver/hardware related, no anything inside my software (as you can see from the code I posted, there really isn't much too what I do for input handling, it's hard to see where I could be going wrong).


Steve
 
X does buffer events, get busy etc, perhaps thats it; in older days on X it was a big problem (like 15 years ago), but I wouldn't think it'd be a problem on Pandora, but hey, I've not used touchscreen through X -- or coudl be a driver oddity.


Did you use the touch devices via tslib, which is quite probably more immediate? ie: You'll notice other apps and emus don't have a lag problem


Pulling the TS events that way works fine for ScummVM and other apps, and is trivial to implement.


jeff
 
X does buffer events, get busy etc, perhaps thats it; in older days on X it was a big problem (like 15 years ago), but I wouldn't think it'd be a problem on Pandora, but hey, I've not used touchscreen through X -- or coudl be a driver oddity.


Did you use the touch devices via tslib, which is quite probably more immediate? ie: You'll notice other apps and emus don't have a lag problem


Pulling the TS events that way works fine for ScummVM and other apps, and is trivial to implement.


jeff

I get the event from x11, if you look at the code block in my initial post on this thread you can see the code I am using. Using tslib does indeed sound like a good suggestion. I have never used tslib before, but your promises of trivial implementation are enticing! :eek: )


I will look into this.


Steve
 
tslib is accessing the touchscreen directly, whereas X11 events also access tslib in the end, it's just the longer way.


ScummVM really works great, and it uses tslib, so it really sounds like the solution to me :)
 
You coudl also read the /dev/event# devices to get the raw events, I imagine the TS heads right through there. Using events is not too hard, though can be tricky, and you'd have to handle calibration and so on yourself. But assuming an already calibrated display (a fairly valid assumption?), it may just work nicely (and your game may not care about extreme accuraqcy.. ie: not like operating a desktop with pixel accurate requirement?)


I didn't build any TS stuff into libpnd (pandora lib that is built into firmware), but there is some /dev/event handling for the other keys/events, so you might be able to see how its done pretty easily; ie:


http://git.openpandora.org/cgi-bin/gitweb.cgi?p=pandora-libraries.git;a=blob;f=lib/pnd_io_evdev.c;h=e88870a22304d89752d9cb5efd695ba56faf30bc;hb=9605ab5fb29ce107a849948eb7ada8d3296093ed


See also..


http://git.openpandora.org/cgi-bin/gitweb.cgi?p=pandora-libraries.git;a=blob;f=include/pnd_device.h;h=e0945739e80a9104d5c591e126fc182dbbe32cbd;hb=9605ab5fb29ce107a849948eb7ada8d3296093ed


--> specifically, the device name for the TS is 'touchscreen' (use the #define though, to be future safe), so it shoudl be findable in evdev pretty easy


jeff
 
I find that when the CPU is maxed out, the touchscreen ignores presses, at least on the desktop.
 
Sebt3 actually suggested that the x-server isn't get time to breathe some time ago to me, however I did put in a millisecond sleep in my render loop, which didn't appear to make any different (providing my sleep implementation was doing something sane on Pandora, which it should be, as Pandora uses pthread's in my engine). As my game runs at 60 FPS, we are talking about around a 16-17ms update loop, so sleeping for 1ms of that means sleeping for around 60 ms per second - which seems like more than enough to me, but then again what do I know, maybe the x-server needs 10% of the CPU time to do its thing (I'd hope not!).


I can certainly try another build with a sleep in, as that's a straight forwards change to make.


Steve
 
Back
Top