Qemu Benchmarking - We Have A Pentium


marovada

Member
Joined
Oct 13, 2009
Messages
109
Benchmarking with Norton sysinfo in dos:

stock 500mhz Pandora = Pentium 128mhz

600mhz Pandora = Pentium 158mhz

900mhz Pandora = Pentium 231mhz

"Usual caveats" means I know the benchmarks may not be a true reflection of speed esp in games.

Blurry pics available on demand.
 
If the stock 500Mhz is even close to a Pentium @ 128Mhz, why was Doom running like it was barely hitting 33Mhz I wonder.
 
marovada said:
"Usual caveats" means I know the benchmarks may not be a true reflection of speed esp in games.

Benchmarking a virtual machine is a waste of time. QEmu is a virtual machine.

Seriously, I can show virtual machines that claim to be 1000MHz 486's and all sorts of nonsense but they can't run simple programs anywhere near that. Benchmarking, especially old DOS-style benchmarking, is a closed loop of repetitive instructions - any clever VM is going to optimise and then cache such loops very easily and quickly and you'll get ridiculous benchmarking figures. Then you load up something *real* like a game and it'll perform like a stunned sloth.

I'd be impressed if my desktop could truly and reliably emulate a decent Pentium once all the hardware is used and you're deep enough to run things like DOS games. Truth is that, without the in-kernel x86-x86 translation module that used to come with QEmu, you can't even get near that on quad-cores etc. once you have everything running simultaneously. It's barely tolerable to run and install a Linux distro like that without virtualisation off-loading tricks like QEmu's module or VT instructions on the processor (and thus x86 compatibility). The emulation of another architecture for a start easily knocks QEmu on Pandora back down to the realms of 386's if not less. If you can even consistently get within 50% of the performance of a decent 486 with sound, complete emulation, etc. enabled, then you'd really have a device worth having. Maybe in benchmarks that do idle-loops and guess at time / instruction counts, but when they have to *do* something like write to IO ports or do FPU arithmetic, that's when it'll show it's true speed.

If you can play Doom smoothly, I'll be a little impressed. If you can get anywhere NEAR Quake emulation (of course Quake runs, I mean running Quake in a QEMU virtual machine) then I'd be incredibly impressed. I used to play Quake at 320x200 on a 133MHz Pentium with software rendering (in it's day, that was fabulous!) and my friend had a 90MHz one that could do pretty much the same with the help of a 3DFX card. Get even CLOSE to that under complete emulation and you really, really do have a device worth buying. I suspect that it wouldn't even load, or you'd get bored waiting for the first frame to draw.
 
Last edited by a moderator:
I tried to head off these arguments with "usual caveats apply" but to no avail. Oh well, at least the benchmarks provoked some discussion :rolleyes:
 
typs lik dis said:
I wasn't aware that the usual caveats included being meaningless.

Ouch. I think it's time to stop posting here :unsure:
 
Last edited by a moderator:
marovada, I wouldn't take it too personally.

Maybe a further possible explanation would help. These sorts of "benchmarks" are usually simple tests meant to determine the clock speed of a fixed CPU architecture. They're actually a lot like "bogoMIPS" on Linux. Because of this, any instruction with a predictable number of cycles is fair game. This includes some really slow instructions like divides. This is especially true as processors get more complex and harder to count cycles for; the big slow instructions ran back to back will be more stable. Code sequences like this might perform really well in qemu and other binary translators, because very roughly speaking part of CPU emulator overhead is proportional to the number of cycles vs the number of instructions. That's because things like translation overhead, hit to instruction cache, register allocation, cycle counting, etc etc are more proportional to number of instructions, whereas the actual execution of the instruction might be similar on both architectures. Another big part of this is that these benchmarks will avoid memory accesses, because the timing of these are a lot less stable/predictable. Memory accesses can be one of the most expensive parts of a CPU emulator. A similar thing can be said for branches, on both counts. So you'll get highly predictable, many iteration, moderate length (not so large as to blow cache, not so small as to incur branch overhead) loops of slow instructions. Something that plays really nicely with an emulator and isn't the least bit representative of real code.

'course, any or all of that could be completely false for this particular test, so just take that as a hypothetical..
 
Exophase said:
marovada, I wouldn't take it too personally.

Maybe a further possible explanation would help. These sorts of "benchmarks" are usually simple tests meant to determine the clock speed of a fixed CPU architecture. They're actually a lot like "bogoMIPS" on Linux. Because of this, any instruction with a predictable number of cycles is fair game. This includes some really slow instructions like divides. This is especially true as processors get more complex and harder to count cycles for; the big slow instructions ran back to back will be more stable. Code sequences like this might perform really well in qemu and other binary translators, because very roughly speaking part of CPU emulator overhead is proportional to the number of cycles vs the number of instructions. That's because things like translation overhead, hit to instruction cache, register allocation, cycle counting, etc etc are more proportional to number of instructions, whereas the actual execution of the instruction might be similar on both architectures. Another big part of this is that these benchmarks will avoid memory accesses, because the timing of these are a lot less stable/predictable. Memory accesses can be one of the most expensive parts of a CPU emulator. A similar thing can be said for branches, on both counts. So you'll get highly predictable, many iteration, moderate length (not so large as to blow cache, not so small as to incur branch overhead) loops of slow instructions. Something that plays really nicely with an emulator and isn't the least bit representative of real code.

'course, any or all of that could be completely false for this particular test, so just take that as a hypothetical..

Thanks Exophase. I didn't really believe that the benchmarks were accurate. I got the idea of this sort of benchmarking from the guy making a dos emulator for the Nintendo DS: DSX86 (you would need to see his older posts)

This guy is writing a dos emulator for both arm and mips almost from scratch! The arm code is for the ds processor whereas the mips code is for a flash card with its own inbuilt processor. He learnt mips assembly just to build ds2X86 again from scrach!! If this can be done for a ds then think of the possibilities for the Pandora. Either:

- qemu could be built more efficiently for arm and without all of the current bugs;
- qemu user mode could be fixed to work with wine; or
- a dos emulator could be built maybe starting with Pate's code.

Anyway, these are all pipe dreams because I have never programmed in assembly (other than copying poke/peek sequences in 80's computers :D ). c wouldn't be efficient enough for this kind of task.

Edit: I think yourself, Pickle and Ari64 would love Pate's blog posts. He goes into great detail on the assembly code and his methodology - presumably so that someone else could in the future pick up on where he finishes.
 
Last edited by a moderator:
Yeah actually I have been keeping current with Pate's blog posts, thanks B) DSx86 is interesting, especially when you consider that it's an interpreter-based DOS emulator that seems much faster at what it does than DOSBox is with a dynamic recompiler to ARM. It shows a lot what you can accomplish with a really thoroughly optimized bare metal ARM ASM design. However, it doesn't look like he's going to be doing protected mode emulation on the ARM version, so there won't be a lot of potential for seeing this ported to platforms like Pandora in any way that's useful. And a lot of what he has done is strictly with old DOS games in mind, providing an old DOS HLE environment, strictly old PC hardware, and taking some shortcuts for things that those games don't need.

Pate's interpreter is very good (a lot of hand-crafted ARM interpreters really shine), but it still shouldn't be able to compete with a decently optimized dynarec, with all other factors remaining the same. DOSBox's dynarec must need work, or the program has other problems.

qemu's ARM dynarec, on the other hand, has made great strides thanks to Laurent and has a lot of promise. But I agree with you, to really unlock its potential, qemu user-mode + wine is the best path forward. This should be much more efficient than the system-mode emulation you're running.
 
I'm looking forward to Laurent's dynarec. BTW I've tried bochs and it is super slow compared with qemu.
 
Back
Top