GP2X Emulation - Recompilation


k0in

Still Fresh
Joined
Oct 31, 2005
Messages
16
As far as I know, QEMU uses dynamic recompilation of code. It's not an interpreter, like most emulators do, it is recompiler. This is the way they improve speed of emulation.

Here is section of documentation.
Like bochs [3], QEMU emulates an x86 CPU. But QEMU is much faster than bochs as it uses dynamic compilation. Bochs is closely tied to x86 PC emulation while QEMU can emulate several processors.

The question is - could it be possible to use something similar to emulate comparable devices on GP2X? (When I say comparable, I mean their total CPU+GPU power and features are just a few times weaker than those of GP2X, not 10-20 times, like Z80)
And is there any source of information about it on the net?

When I thought about it, the only problem I saw was (dynamic) jumps/calls to calculated adress. Everythin beside it could be recompiled and recalculated statically. And while QEMU does on-fly recompilation, it would be better to "recompile" code before run, completely, probably even on PC, not GP2X.

Any thoughts from experienced devs?
 
Though I am not sure regarding legal questions - you may have the ROM legally, if you bought a copy or if it's freeware, but i am not sure if it's allowed to recompile it for different system.
 
The problem with attempting to convert an entire program from one processor type to another is that you don't know what is code and what is data until the program is actually executed. Also, some programs have copy code to another place before execution, modify code before being run, or even take a block of data and decompress it. None of this can be supported by recompiling the executable code. The only way of supporting these features would be by using dynamic on-the-fly recompilation - this way, if a block of code is changed, the recompiled block can be thrown away and recompiled next time it is required.

The problem with dynamic recompilation (otherwise known as 'Dynarec') is that it needs to be rebuilt for each and every processor type - so you have one for x86, one for arm, etc, where an interpreter written in c is fully portable. They are also much more difficult to create, and use up a lot of memory for storage of the original code, recompiled code, and internal data structures.
 
The problem with dynamic recompilation (otherwise known as 'Dynarec') is that it needs to be rebuilt for each and every processor type - so you have one for x86, one for arm, etc, where an interpreter written in c is fully portable. They are also much more difficult to create, and use up a lot of memory for storage of the original code, recompiled code, and internal data structures.

There is something to the idea of using a profile driven approach to dynamicaly recompiling the most executed parts of a ROM. It seems to me that the trickiest part would be dealing with memory accesses. Usually a system's memory is emulated with an address translation table similar to virtual memory.

When you recompile ROM code to be native code, you also have to include the code for translating the emulated memory addresses to the actual hardware addresses where the emulator is storing it. There are so many ways to calculate memory addresses, it would be difficult to recompile the code into native code that correctly calculates the right hardware address. It can be done, but it requires clever emulation of the emulated system's memory and deep knowledge of all the addressing modes possible on the emulated system.
 
Last edited by a moderator:
The problem with attempting to convert an entire program from one processor type to another is that you don't know what is code and what is data until the program is actually executed.


out of technical curiousity: how's that? i'd guess that if you can tell codeVSdata at run time you can also tell at compile time... at least in x86 binaries. or am i wrong?
 
Last edited by a moderator:
Take for example the following code:

1:JMP EAX
2:DCB 'Hello World'
3:INC EDI <<< EAX actually jumps here if we know the value
4:MOV ECX,AEF24857
5:pUSH EBP
6:CALL DWORD PTR DS:[ESI+48018]

Runtime & Dynarec: you will know the value of EAX, DS and ESI, and since you are executing the code as the compile, you are following the path of execution, and will get everything correctly.

Full compilation: You can't possibly know the value of these registers, as they could be created using some maths equations earlier on. How can you tell that the EAX is not going to point inside the "Hello World" string? If you try and disassemble it, you'll realise that it translates to perfectly valid assembler:

'H' DEC EAX
'el' INS BYTE PTR ES:[EDI],DX
'l' INS BYTE PTR ES:[EDI],DX
'o' OUTS DX,DWORD PTR ES:[EDI]
' wo' AND BYTE PTR DS:[EDI+6F],DL
'rl' JB SHORT SeiPSPTo.00446FDA
'd' ADD BYTE PTR FS:[EAX],AL

Now, if the compiler takes this as code and recompiles it, not only will it have superfluous code that will never be executed, it will also cause the compiler to misalign and read the instruction directly after the string incorrectly, as it will take it as "6447" rather than "47" which it is supposed to be. In this case, it's not that bad, as it just causes a superfluous prefix to be added to the INC EDI instruction, but what if the last byte of the string was F3 and the actual instruction afterwards was A5? Suddenly you've prefixed a "MOVS DWORD PTR ES:[EDI],DWORD PTR DS:[ESI]" instruction with a REP prefix (which stands for repeat this instruction), which can cause major problems, and will certainly cause the recompiled program to do something weird and possibly even crash.
 
The problem with attempting to convert an entire program from one processor type to another is that you don't know what is code and what is data until the program is actually executed. Also, some programs have copy code to another place before execution, modify code before being run, or even take a block of data and decompress it. None of this can be supported by recompiling the executable code. The only way of supporting these features would be by using dynamic on-the-fly recompilation - this way, if a block of code is changed, the recompiled block can be thrown away and recompiled next time it is required.

The problem with dynamic recompilation (otherwise known as 'Dynarec') is that it needs to be rebuilt for each and every processor type - so you have one for x86, one for arm, etc, where an interpreter written in c is fully portable. They are also much more difficult to create, and use up a lot of memory for storage of the original code, recompiled code, and internal data structures.

I see the point with data and code. It could be tracked easily even without any emulation, if no dynamic jumps/call are used. Is it often case, the "JUMP EAX" or "push 123; ret" things? I thought, modern (and even not modern) compilers don't use such technics.
 
Last edited by a moderator:
It is indeed easy with constant jumps - you can simply follow every path of flow control. However compilers make heavy use of basing jumps from a processor register or memory location, as they use them regularly for jump tables (eg. for switch statements, calling OS api functions, and the like).

eg.

Some compiler prefer this method of calling an API:

CALL DWORD PTR DS:[<&KERNEL32.GetVersionExA>]

Whilst others prefer this method:

MOV EDI,DWORD PTR DS:[<&KERNEL32.GetModuleHandleA>]
PUSH ESI
CALL EDI

and others just put the base address in a register, and reference it using another register:

CALL DWORD PTR DS:[EAX+ECX*4]
 
It is indeed easy with constant jumps - you can simply follow every path of flow control. However compilers make heavy use of basing jumps from a processor register or memory location, as they use them regularly for jump tables (eg. for switch statements, calling OS api functions, and the like).

eg.

Some compiler prefer this method of calling an API:

CALL DWORD PTR DS:[<&KERNEL32.GetVersionExA>]

Whilst others prefer this method:

MOV EDI,DWORD PTR DS:[<&KERNEL32.GetModuleHandleA>]
PUSH ESI
CALL EDI

and others just put the base address in a register, and reference it using another register:

CALL DWORD PTR DS:[EAX+ECX*4]

Then you need some pre-emalution, yep, to make this code run and to get the needed adresses in execution/emulation time. (hoping they will be the same next time, and it's only the speed question, not really dynamic calculation)

But then I think, stuff like virtual functions in Object-ORiented Languages, Like C++, they are really calculated in RunTime, and that's bad.
 
Last edited by a moderator:
Back
Top