Audio Driver (with bounty!)


I have a really hard time to grasp how the whole audio topology works, and where all the pieces come from :/
Same for me... This is the reason why this is not a stroll but climbing Mt. Fuji :) And why it takes that long.
I just have started my PandaES to check how the working setup was built. And it turned out that v6.5-rc2 changed the device tree subdirectory structure (to the better) but that is not handled properly by my build scripts.
Why do I work on PandaES? Because it has one of the handsfree speaker drivers wired to a jumper. And it is easier to have a remote controllable setup than the Pyra. And, I have (had?) a working 3.15 AESS setup - but did not yet find the SD card.
 
Same for me... This is the reason why this is not a stroll but climbing Mt. Fuji :) And why it takes that long.
I see what you did there :D
I just have started my PandaES to check how the working setup was built. And it turned out that v6.5-rc2 changed the device tree subdirectory structure (to the better) but that is not handled properly by my build scripts.

Why do I work on PandaES? Because it has one of the handsfree speaker drivers wired to a jumper. And it is easier to have a remote controllable setup than the Pyra. And, I have (had?) a working 3.15 AESS setup - but did not yet find the SD card.
I got sound from the legacy interface with a Letux kernel and device tree I received from @hns. I still get "Invalid argument" when trying the AESS sound path, so not much progress. I'll try and replicate the kernel build in my own build environment.

On a related topic, I found this talk today:

They present a tool for producing firmware files for the Linux audio SoC subsystem. The tool is called "alsatplg" (ALSA topology) and is part of the official ALSA project. It comes bundled with alsa-utils on any modern Linux distro. The tool takes plaintext configuration files (and blobs) and produces a firmware file to be loaded by the Linux kernel. Support for the firmware format was merged in Linux-4.2, and considered stable in Linux-4.8. The current firmware ABI version is 5, but ABI version 4 is also compatible.

What's especially interesting in the talk is that they mention that the tool was initially developed by Liam Girdwood when he was working for TI and the OMAP processors. He later joined Intel and continued working on it. Liam Girdwood is also the name you find in most commits for the AESS firmware repositories. The official AESS firmware, released by TI for Linux-3.8, has firmware ABI version 1.

@hns has spent a considerable amount of time to update the AESS firmware repositories to produce an AESS firmware that can be loaded by mainline Linux. The only way to produce it is by compiling the tools and binary on a compatible board itself, and this compilation breaks from time to time when kernel headers change. Apart from that, the code for generating the firmware is quite bad and hard to understand IMHO. This is not really a good situation.

So, an idea took form: since the firmware @hns have converted can successfully be loaded by Linux, it should be compatible with firmware ABI version 4 or 5. What if we can use the alsatplg tool to decode it back into plaintext configuration files (and blobs)? Then we would have the firmware information in a much more readable format and a well-maintained tool for compiling it! Unfortunately, it wasn't that easy :(

Code:
$ alsatplg -d omap_aess-adfw.bin -o omap_aess-adfw.bin.decoded
ALSA lib decoder.c:98:(snd_tplg_decode) first block must be manifest (value 3)
failed to decode context omap_aess-adfw.bin: Invalid argument

The manifest block is actually described in the video above and shouldn't be that hard to inject into the binary, but I haven't tried yet.
Anyone feeling adventurous could have a try! :D

Since we already have a stable firmware binary, I will not focus that much on it for now. I'll focus on getting AESS support working (unless @hns beats me to it)
 
Great idea and good find for the alsatplg utility...really hope we find a way to finally enable and use the AESS.
Unfortunately on Pyra Alsautils (and maybe Alsa too) is too old and don't have the -d option to decompile a firmware.

On a side note...today i tested again the firmware you compiled on the bookworm image ( i know that kernel was not able to use...but still i tried)...an i noted a strange thing....when loaded the USB ports were disabled...and i noted this because the same thing happened on standard pyra os.
Maybe this are unrelated...but who knows.
 
What's especially interesting in the talk is that they mention that the tool was initially developed by Liam Girdwood when he was working for TI and the OMAP processors. He later joined Intel and continued working on it. Liam Girdwood is also the name you find in most commits for the AESS firmware repositories. The official AESS firmware, released by TI for Linux-3.8, has firmware ABI version 1.
Maybe you can get some help or pointings to the right direction from Liam Girdwood himself. Who knows, it costs nothing to ask... Maybe he's even pleased by your efforts...
 
Great idea and good find for the alsatplg utility...really hope we find a way to finally enable and use the AESS.
Unfortunately on Pyra Alsautils (and maybe Alsa too) is too old and don't have the -d option to decompile a firmware.
You don't have to decode the firmware on the Pyra. Any OS which can run alsatplg may do it. I tried it on Arch and got the error posted above.
On a side note...today i tested again the firmware you compiled on the bookworm image ( i know that kernel was not able to use...but still i tried)...an i noted a strange thing....when loaded the USB ports were disabled...and i noted this because the same thing happened on standard pyra os.
Maybe this are unrelated...but who knows.
Strange indeed :confused:
 
Great idea and good find for the alsatplg utility...really hope we find a way to finally enable and use the AESS.
Unfortunately on Pyra Alsautils (and maybe Alsa too) is too old and don't have the -d option to decompile a firmware.
You don't have to decode the firmware on the Pyra. Any OS which can run alsatplg may do it. I tried it on Arch and got the error posted above.
Decoding is only needed once in life of mankind either... As soon as it has been converted into a .conf file only the compile mode is needed to build the firmware from scratch. And there is nothing wrong using the binary format of the firmware.
Anyways this topology file conversion is not the road block. It is "cosmetics" to make future life easier.
The real roadblock is that the AES DSP isn't running properly yet. Hence we can use only the legacy mode (with the existing firmware files).
 
Finally got some work done on this today, and I might have found the reason the AESS firmware is not running as it should!

You see, early during firmware loading we do this:
C:
static int aess_load_fw(struct snd_soc_component *component,
                        struct snd_soc_tplg_hdr *hdr)
{
        struct omap_aess *aess = snd_soc_component_get_drvdata(component);
        const void *fw_data = snd_soc_tplg_get_data(hdr);

        /* get firmware and coefficients header info */
        memcpy(&aess->fw_hdr, fw_data, sizeof(struct fw_header));
...
        /* store AESS firmware for later context restore */
        aess->fw_data = fw_data;

        return 0;
}
Notice how aess->fw_data is pointing to the start of a struct fw_header.

Later, we do this:
C:
/**
 * omap_aess_load_fw_param - Load the AESS FW inside AESS memories
 * @aess: Pointer on aess handle
 * @data: Pointer on the AESS firmware (after the header)
 *
 * Load the different AESS memories PMEM/DMEM/SMEM/DMEM
 */
static void omap_aess_load_fw_param(struct omap_aess *aess)
{
...
        pmem_ptr = (u32 *) aess->fw_data;
...
        omap_aess_write(aess, OMAP_AESS_BANK_PMEM, 0, pmem_ptr, pmem_size);
...
}
We're writing the fw header straight into PMEM memory bank! :mad: There's even an outdated comment about a missing data parameter to this function where they specifically mention "after the header".

The fix is easy:
Diff:
diff --git a/sound/soc/ti/aess/omap-aess-fw.c b/sound/soc/ti/aess/omap-aess-fw.c
index 8daff9fb1488..48cd6d7e3d33 100644
--- a/sound/soc/ti/aess/omap-aess-fw.c
+++ b/sound/soc/ti/aess/omap-aess-fw.c
@@ -658,7 +658,7 @@ static int aess_load_fw(struct snd_soc_component *component,
        dev_info(aess->dev, "AESS Firmware version %x\n", aess->fw_hdr.version);
 
        /* store AESS firmware for later context restore */
-       aess->fw_data = fw_data;
+       aess->fw_data = fw_data + sizeof(struct fw_header);
 
        return 0;
 }

With this patch, I can dump pmem from /sys/kernel/debug/omap-aess/pmem however much I like and I don't get any segfaults any more!
I'll send the patch to the Letux and Pyra kernel mailing list, once I get my mail client setup again. :oops:

Still can't play any sound though

EDIT:
I can confirm that PMEM dumped from Linux-3.8 and my latest kernel are identical now with this patch. Reading PMEM from /sys/kernel/debug/omap-aess/pmem reads it directly from the AESS memory banks.
 
Last edited:
I've sent a patch to the Letux mailing list and notified @hns about the findings. He sounded very excited in his reply :D He was gonna do some testing on his PandaES asap. The PandaES is a similar architecture to the Pyra.

I've also pushed the code I used for debugging this issue: https://github.com/Risca/dump_pmem
It's a kernel module with all the boiler plate needed to load the AESS firmware, parse it, write the DSP bits to the DSP, then dump the PMEM memory area.

It was when I was putting together all the pieces in one, single place when I could grasp the whole picture and realize what was going on.
 
Yes, it works for me as well. Congratulations to Risca to have found the reason for this issue.

The fix I will add is a little different. It will be in omap_aess_load_fw_param() and not aess_load_fw().
The reason is that the code in omap_aess_load_fw_param() is exactly the same as in the 3.15 kernel but aess_load_fw() had been reworked and obviously a bug was introduced (by me).
So it will be fixed where it was really wrong: https://git.goldelico.com/?p=letux-...ff;h=52383c444c44db67b991dbe9c35aac8774fba6f7

With the DSP obviously running now, we can use the ABE tools for further tests: https://github.com/omap-audio/audio-test/tree/master/tools/abe
Especially abe_check now reports that buffers are exchanged between the Linux system on omap4/5 core and the AE DSP.

Great progress and kudos to Risca!
 
Back
Top