GP2X Feasibility Of Mpeg4 Audio Decoding


Epicenter

Very Active Member
Joined
Oct 9, 2005
Messages
2,068
Age
36
Location
USA
Website
www.epicgaming.us or http
Pardon my newbishness to the MPEG4 format, but I was exploring today the options of audio formats to decode in realtime running a game. MP3 is much less 'open' and bears licensing issues, and carries a heavy CPU toll- even with a fixed-point decoder lib. Same for OGG Vorbis, although it is at least entirely open. I'm seeing a 40 FPS hit with audio decoding using either of those two. While a remaining 84 FPS is still nice, I absolutely need 90 FPS+ for my purpose, and will be rendering more than this, so this sort of performance is just unacceptable.

I'm considering some new ideas, like 22khz or 44khz ADPCM, but these are fairly large files, even though they seem to carry no major CPU cycle toll to use. This brings into play more substantial load time, RAM consumption and of course SD card space constraints.

An odd idea hit me-- in my list of possible conversions in the application I was using to test my source material in different formats, I found some MPEG-4 audio solutions. This brings me to wonder, does the MPEG4 format contain some sort of support for audio tracks, or does it merely expect to contain video frames and be laid side-by-side with AC3 or AAC et cetera in a container?

The reason I ask is, if MPEG4 contains support for audio already, perhaps there is the possibility of utilizing the MMSP2's built-in MPEG4 decoding hardware to offload the hefty task of audio decoding from the ARM920T. Unlikely I suppose since it is probably specially geared to work only on the low-level common math tasks for MPEG4 codecs in the aid of decoding video frames. However, it never hurts to ask-- has this been considered and/or explored? Is documentation and understanding of the MPEG4 functionality of the MMSP2 still too poor?

Thanks for any information.
 

greenmikey

dancin' to the moldies
Joined
Aug 28, 2003
Messages
747
Age
35
Location
Usa, Ohio
Website
Visit site
I don't doubt with full documentation we could do it..but magic eyes are a bit too secretive for us. We probably could use tricks to offload lots of little things on to it...some day someone might do something cool with it.
 

BradN

Member
Joined
Jun 22, 2003
Messages
577
I'm not sure there is such a thing as mpeg4 audio, but I wouldn't be surprised if magiceyes slipped some mp2/3/aac decoding assist hardware in there, especially considering the target market. This could give a big boost to video and audio playback time if it exists (and is utilized) as well.

We know (well, at least I think I remember) they have video encoding hardware, why would they stop there and not do audio encoding? If they do audio encoding, why not decoding?

Also, I can't fathom why magiceyes wouldn't want to give out tech docs to gp2x developers - it makes a great demo platform for their chip, and the better it performs, the better their chip looks.
 

Epicenter

Very Active Member
Joined
Oct 9, 2005
Messages
2,068
Age
36
Location
USA
Website
www.epicgaming.us or http
Because they can charge companies like GPH who are buying their chips because they are on a tight budget, to buy custom-made shitty firmware from them that uses the hardware's true abilities, while keeping how it actually works shielded to secure a stream of constant revenue, from what I hear.
 

Squidge

Certified Guru
Joined
Nov 16, 2003
Messages
8,495
Location
UK
Website
Visit site
Epicenter posted on Sep 27 2006 at 05:40 AM said:
Shikaku posted on Sep 26 2006 at 08:39 PM said:
I don't think the video processor processes audio...
Right now it doesn't. The question is, could it be made to.

Yes :)
 
Last edited by a moderator:

TheFr00n

Still Fresh
Joined
Mar 12, 2006
Messages
86
Hmm. I see where you're going with this, and it is an interesting idea. However, I gotta point out that the *.mp4 format is simply a file container, similar to other file containers such as *.avi and *ogm. It is a box that contains a video and an audio stream.

Therefore, even if an *.mp4 file contains new, proprietary audio streams, they wouldn't be decoded by the MMSP2. Ultimately, the term MP4 refers (AFAIK) to video only.

That said, if you could find a way to use the MMSP2 to decode the audio, might well result in the performance boost you want.

But I have a question: As DivX and XviD are implementations of the MP4 video standard, wouldn't asking the MMSP2 to decode the audio as well actually result in a performance hit, since it's already processing video?
 

BradN

Member
Joined
Jun 22, 2003
Messages
577
TheFr00n posted on Sep 27 2006 at 03:07 AM said:
But I have a question: As DivX and XviD are implementations of the MP4 video standard, wouldn't asking the MMSP2 to decode the audio as well actually result in a performance hit, since it's already processing video?

If there is audio decoding circuitry, it's almost certainly separate than the video stuff. That's the nice thing about chips with special purpose hardware - it can usually be used simultaneously with other things, at least if it's well designed. In the case where it'd be doing both (video and audio decoding) at the same time, it might not make a huge difference performance wise, but it'll take some load off the processor and might allow running at a lower clock rate -> better battery life.

Where the performance side comes into play is in games where you want background music... it's possible the decoding hardware is more or less efficient with memory access (so it might be better or worse performance wise than decoding audio on the 940, as far as impacting the 920's access to memory). If you had another job for the 940 anyway (rendering, etc), then it would almost certainly be a win.
 
Last edited by a moderator:

PokeParadox

Founder of Pirate Games - Penjin Coder
Staff member
Joined
Dec 8, 2005
Messages
6,551
Age
37
Location
UK
Website
www.projectinfinity.org.uk
doc5avage posted on Sep 27 2006 at 10:14 AM said:
Isn't Apple's AAC files an Mpeg4 variant?

AAC is the proposed standard audio codec for MP4 streams.
Xvid and DivX are mpeg4 complient video implementations.

MP4, the container can contain mp3 or AAC audio (I think also AC3) + mpeg4 complient video(divX,xvid,3ivx etc...) + subtitles + chapter info.
 
Last edited by a moderator:

greenmikey

dancin' to the moldies
Joined
Aug 28, 2003
Messages
747
Age
35
Location
Usa, Ohio
Website
Visit site
Squidge posted on Sep 27 2006 at 02:55 AM said:
Epicenter posted on Sep 27 2006 at 05:40 AM said:
Shikaku posted on Sep 26 2006 at 08:39 PM said:
I don't think the video processor processes audio...
Right now it doesn't. The question is, could it be made to.

Yes :)
survey says squidge *might* have a card up his sleeve or is workin on something...*cough*snes*cough*
 
Last edited by a moderator:

doc5avage

Member
Joined
Sep 19, 2006
Messages
173
Age
38
Location
Australia
Website
Visit site
PokeParadox posted on Sep 28 2006 at 01:37 AM said:
doc5avage posted on Sep 27 2006 at 10:14 AM said:
Isn't Apple's AAC files an Mpeg4 variant?

AAC is the proposed standard audio codec for MP4 streams.
Xvid and DivX are mpeg4 complient video implementations.

MP4, the container can contain mp3 or AAC audio (I think also AC3) + mpeg4 complient video(divX,xvid,3ivx etc...) + subtitles + chapter info.

my bad
 
Last edited by a moderator:

Epicenter

Very Active Member
Joined
Oct 9, 2005
Messages
2,068
Age
36
Location
USA
Website
www.epicgaming.us or http
BradN posted on Sep 27 2006 at 09:27 AM said:
Where the performance side comes into play is in games where you want background music... it's possible the decoding hardware is more or less efficient with memory access (so it might be better or worse performance wise than decoding audio on the 940, as far as impacting the 920's access to memory). If you had another job for the 940 anyway (rendering, etc), then it would almost certainly be a win.
Well, I tried OGG decoding on the 940T with Dzz's modified libtremor for it. But it dropped my game's framerate to HALF and did the same to someone else's. So I think they are competing for memory too much...
 
Last edited by a moderator:

BradN

Member
Joined
Jun 22, 2003
Messages
577
Personally, I bet hardware audio decoding would be a lot easier on memory access than through software - all the lookup tables are probably built into the chip, etc...
 
Top