Am Thu, 25 Sep 2014 20:16:10 +0300 hat Grazvydas Ignotas notasas@gmail.com geschrieben:
Hi,
The question is different: not why we don't want to have them. But does the TWL6040/AESS need them at all? Are they closed source or not? I don't know yet.
I'm quite sure AESS does, it's closed source and undocumented, so a binary blob is needed to drive that thing.
Well, that's something we need to check.
Maybe a mixture is possible? If normal audio (for gaming via normal speakers and headphones) works without AESS and binary blob, but 3G/4G and BT Headset needs a binary blob, then I would be fine with that. Those chips need binary blobs anyways.
But I agree that normal operation shouldn't need a binary blob, so normal audio playing, gaming, etc.
1x for 3G/4G modem
I thought 3G/4G was just for mobile Internet. From what I read on the forums, not many people are expecting to make calls with pyra, many said to be buying Pyra variant without 3G. I'm sure I never will make calls with my Pyra, there is a phone for that.
Well, I also won't do phonecalls with it, but I know quite a few people at the GamesCom asked for it. So if easily possible, the sure, but it's not the main priority.
1x for digital BT headset
We have that connected on pandora, but it was never used, there is no driver, we don't even know if hardware works. By default bluez is sending audio via UART data link, so nobody ever complained about missing drivers for the mcBSP link.
The reason not many people use it on the Pandora is simply because it doesn't work properly. I tried it once but didn't get it to work, so that's not a real reason. I agree that gamers won't do it (latency and bad audio quality), but if someone wants to make a phone call, they will probably use a Bluetooth headset for it.
So while not main priority, it would be nice to use.
1x for digital FM radio
We don't even know what wifi chip will be used or if it will have FM link. In either case it's not a huge loss, and we can use newer Jorjin chip without FM (if it's finally available).
Well, that REALLY is low priority, agreed.
which one should we leave out?
I think any/all except speakers/headphone can be left out, especially BT.
If BT works, it would be nice. Streaming music to car radios, using a headset for phone calls would be nice as well.
But as mentioned: Main audio should be simple without binary blobs.
The TWL is a quite simple DAC. Well, two stereo channels of them. It just has a different audio source interface and amplifiers integrated. And an ADC for microphones and Vibramotor driver. So it looks more complex but it isn't, because it is just 6 chips in one case...
Maybe, but the driver regards it as one single chip and that makes it complex. Perhaps multiple drivers for different parts would make the system simpler (divide and conquer), but I'd guess those 6 subchips are strongly related and are not as separate as you portray them here.
Well, what I certainly wouldn't like is tons of audio controls in the mixer. So we need to check upfront whether we can configure the driver to only enable / show what we need.
And even better if we can enable / disable those on the fly, i.e. only show 3G audio control if 3G is enabled, same for BT, etc.
Everything else would be too confusing.
The TWL6040 is a DAC - and does not introduce additional latencies. I wonder where this assumption is coming from?
But it basically doesn't work without AESS, and that thing is driven by a binary blob and complex multi-layer driver. If there is some problem with the blob, we're screwed (maybe, just maybe Peter would be able to help..).
Yep, I don't want binary blobs for the main features as well. That's something we really need to check upfront!
This is why I still have on my to-do list to provide a PandaBoard ES based full demo (including one of the speakers). So that you can listen and do measurements. But such a demo must be carefully set up. Especially if people expect a demo to be better in the first shot than the OpenPandora which had some development time for the audio system... And I have no Pandora to compare and improve the quality until it is better.
I guess ED should send you one then.
I can do that (I thought you had one, but maybe I just planned to send you one).
Here is my suggestion: Right now, we're mostly making assumptions. Maybe the TWL is working fine, maybe it's even worse than we think.
So we should do tests.
As you know, I've got professional audio and video equipment to make proper measurements.
Here's what I'd like to do:
For audio quality:
1. I will create an audio file which includes various tests: Sweep and noise (for testing linearity), some high-end orchestral recordings and some chiptunes (for testing noise and faithful audio reproduction).
2. I will play that file using my video editing suite, the EVM and a Pandora and measure using a professional frequency analyzer. I will record that with a highspeed camera (both audio and video), so we can compare the frequencies of each system and also analyze the recorded audio output directly on a PC.
3. I will do the same with different volume output (Line Out and max headphones output) to see that linearity and quality doesn't change with the amp.
That should be enough.
For latency:
notaz: Could you create a simple tool that will do a small white flash on the screen together with a short beep (100% synchronized) which I can run both on the Pandora and EVM?
I can then record both with a highspeed camera and test the latency.
For those tests to work we need a proper audio driver (without using a binary blob) for the TWL.
Would that be a suggestion?
Hi,
Am 26.09.2014 um 13:12 schrieb Michael Mrozek EvilDragon@openpandora.org:
Am Thu, 25 Sep 2014 20:16:10 +0300 hat Grazvydas Ignotas notasas@gmail.com geschrieben:
Hi,
The question is different: not why we don't want to have them. But does the TWL6040/AESS need them at all? Are they closed source or not? I don't know yet.
I'm quite sure AESS does, it's closed source and undocumented, so a binary blob is needed to drive that thing.
Well, that's something we need to check.
Maybe a mixture is possible?
I think so. My tests on PandaBoard and EVM did simply run aplay without installing AESS.
If normal audio (for gaming via normal speakers and headphones) works without AESS and binary blob, but 3G/4G and BT Headset needs a binary blob, then I would be fine with that. Those chips need binary blobs anyways.
But I agree that normal operation shouldn't need a binary blob, so normal audio playing, gaming, etc.
The result of my tests and Peter’s comments:
the TWL6040 already works on the EVM and PandaBoard w/o any blob. With Linux from kernel.org w/o any modification.
The only issue I know is that there is a „plop" sound when speakers are enabled (driver / calibration issue).
1x for 3G/4G modem
I thought 3G/4G was just for mobile Internet. From what I read on the forums, not many people are expecting to make calls with pyra, many said to be buying Pyra variant without 3G. I'm sure I never will make calls with my Pyra, there is a phone for that.
Well, I also won't do phonecalls with it, but I know quite a few people at the GamesCom asked for it. So if easily possible, the sure, but it's not the main priority.
1x for digital BT headset
We have that connected on pandora, but it was never used, there is no driver, we don't even know if hardware works. By default bluez is sending audio via UART data link, so nobody ever complained about missing drivers for the mcBSP link.
The reason not many people use it on the Pandora is simply because it doesn't work properly. I tried it once but didn't get it to work, so that's not a real reason. I agree that gamers won't do it (latency and bad audio quality), but if someone wants to make a phone call, they will probably use a Bluetooth headset for it.
So while not main priority, it would be nice to use.
1x for digital FM radio
We don't even know what wifi chip will be used or if it will have FM link. In either case it's not a huge loss, and we can use newer Jorjin chip without FM (if it's finally available).
Well, that REALLY is low priority, agreed.
which one should we leave out?
I think any/all except speakers/headphone can be left out, especially BT.
If BT works, it would be nice. Streaming music to car radios, using a headset for phone calls would be nice as well.
But as mentioned: Main audio should be simple without binary blobs.
The TWL is a quite simple DAC. Well, two stereo channels of them. It just has a different audio source interface and amplifiers integrated. And an ADC for microphones and Vibramotor driver. So it looks more complex but it isn't, because it is just 6 chips in one case...
Maybe, but the driver regards it as one single chip and that makes it complex. Perhaps multiple drivers for different parts would make the system simpler (divide and conquer), but I'd guess those 6 subchips are strongly related and are not as separate as you portray them here.
Well, what I certainly wouldn't like is tons of audio controls in the mixer. So we need to check upfront whether we can configure the driver to only enable / show what we need.
And even better if we can enable / disable those on the fly, i.e. only show 3G audio control if 3G is enabled, same for BT, etc.
Everything else would be too confusing.
That is neither a hardware nor a kernel issue. It is a matter of what and how well thought a GUI displays the mixers.
If it simply shows all controls in a bigtable that is bad GUI design. It should group elements and allow to hide them. Or put them in tabs or whatever.
There are systems that use amixer but don’t even present the mixer controls. The solution is to use „state files“. So the user can choose between 3 or 4 different states (which may include hundreds of fine-tuned mixer settings).
Usually a sophisticated phone has these states: * phone call through earpiece * handsfree phone call * phone call through headset (wired) * phone call through bluetooth * play music * play FM radio
Game console operation is like the play music state.
The TWL6040 is a DAC - and does not introduce additional latencies. I wonder where this assumption is coming from?
But it basically doesn't work without AESS, and that thing is driven by a binary blob and complex multi-layer driver. If there is some problem with the blob, we're screwed (maybe, just maybe Peter would be able to help..).
Yep, I don't want binary blobs for the main features as well. That's something we really need to check upfront!
I hope Peter can comment on that, since he is most experienced with this topic.
This is why I still have on my to-do list to provide a PandaBoard ES based full demo (including one of the speakers). So that you can listen and do measurements. But such a demo must be carefully set up. Especially if people expect a demo to be better in the first shot than the OpenPandora which had some development time for the audio system... And I have no Pandora to compare and improve the quality until it is better.
I guess ED should send you one then.
I can do that (I thought you had one, but maybe I just planned to send you one).
I only have one with broken LCD and disassembled :)
Here is my suggestion: Right now, we're mostly making assumptions. Maybe the TWL is working fine, maybe it's even worse than we think.
So we should do tests.
Exactly. Everything else is poking about in a fog.
As you know, I've got professional audio and video equipment to make proper measurements.
Here's what I'd like to do:
For audio quality:
- I will create an audio file which includes various tests:
Sweep and noise (for testing linearity), some high-end orchestral recordings and some chiptunes (for testing noise and faithful audio reproduction).
Ok.
- I will play that file using my video editing suite, the EVM and
a Pandora and measure using a professional frequency analyzer. I will record that with a highspeed camera (both audio and video), so we can compare the frequencies of each system and also analyze the recorded audio output directly on a PC.
What is the input specification of the analyzer? Is it measuring the signals on a real (or virtual) 8 ohm or 16 ohm speaker connected to it? I ask because the TWL class D amplifier might not show the right signals if operated without a real speaker coil.
Or do you plan to drive a defined speaker with signals and do you measure with a microphone (resembling the tympanum and cochlea)?
So I would suggest:
you define a headset / speakers + measurement microphone in some box.
And the headset / speakers are driven by a) Pandora b) PandaBoard ES (for Headset and Speaker) c) EVM (headset only) d) your video editing suite e) maybe some other devices (e.g. a MP3 player or some Notebook sound card)
The Pyra speakers should ideally be placed in the Pyra Case (with proper box volume).
And it will be interesting to see if there are differences between b) and c)
I ask that because it is very easy to measure something differently from what was planned (or is the real operating condition) if the measuring setup isn’t well designed.
- I will do the same with different volume output (Line Out and max
headphones output) to see that linearity and quality doesn't change with the amp.
That should be enough.
For latency:
notaz: Could you create a simple tool that will do a small white flash on the screen together with a short beep (100% synchronized) which I can run both on the Pandora and EVM?
I can then record both with a highspeed camera and test the latency.
Well, this is nice, but the human brain is very good at hiding or ignoring smaller latencies. So you might optimize something nobody can recognise.
What I mean is this:
http://www.hifi-forum.de/viewthread-42-38.html
They say +/-80 ms is unnoticeable. Well others disagree with that statement and we are not the first ones to think about it:
"http://www.hifi-forum.de/viewthread-42-38.html
Gemäss den Angaben in den Richtlinien der ITU (ITU-R BT.1359-1 App. 1 [1998]) sind Asynchronitäten vom Durchschnittsfernsehzuschauer ab +45/-125ms erkennbar und bis +90/-180ms tolerierbar (plus bedeutet, dass der Ton dem Bild vorausläuft
Yes, I know that the game players are not „Durchschnittsfernsehzuschauer“ :) They are trained to have much better audio-visual sensors than the average…
For those tests to work we need a proper audio driver (without using a binary blob) for the TWL.
Would that be a suggestion?
Yes, looks good to me.
I can prepare the Panda-ES for speakers.
BR, Nikolaus
Hi,
- I will play that file using my video editing suite, the EVM and
a Pandora and measure using a professional frequency analyzer. I will record that with a highspeed camera (both audio and video), so we can compare the frequencies of each system and also analyze the recorded audio output directly on a PC.
What is the input specification of the analyzer? Is it measuring the signals on a real (or virtual) 8 ohm or 16 ohm speaker connected to it? I ask because the TWL class D amplifier might not show the right signals if operated without a real speaker coil.
I can set that up. I'll start with standard Line Out signals, as they should have the exact same levels with the same audio file. (I hope the EVM also has a standard fixed line out as the Pandora has?)
Those should be fine to test the DAC, etc.
For the amplifier, I'll test the headphones port afterwards.
Or do you plan to drive a defined speaker with signals and do you measure with a microphone (resembling the tympanum and cochlea)?
I can additionally test it with a professional audio studio monitor and a Sennheiser ME66 (should be good enough for that).
I can then record both with a highspeed camera and test the latency.
Well, this is nice, but the human brain is very good at hiding or ignoring smaller latencies. So you might optimize something nobody can recognise.
When audio is delayed for more than 20ms, it's clearly noticable (20ms is about a frame in PAL) for trained people. I can easily see that (and if audio comes 10ms too early, it's even more noticable).
One thing that's important: You cannot compare TV to gaming. People in Germany are used to german dubbed movies, so when watching a movie, synchronity isn't that important. For sound effects: It depends. With explosions, you don't notice it, as most people haven't experienced in real life when exactly you can hear the sound of an explosion. Knocking on a door however is different - a 20ms delay, as mentioned, can easily be noticed.
With retro video gaming, this is different. Sound effects (jumping, shooting, etc.) are immediately connected with the user pressing a button. It feels weird if it's delayed - it can even make the game unplayable. I tried adding an artificial delay to Sonic and was way worse when playing it - as I'm used to the immediate jumping sound when pressing the button, so my control wasn't as accurate with a delay.
There will always be a bit of delay, I just want to check that it isn't too bad.
Most modern SoCs are meant for playing music, and gaming isn't as accurate as it was before (because all HDMI devices have a horrible video delay as well), so the delay is of low priority to the manufacturers.
What I mean is this:
http://www.hifi-forum.de/viewthread-42-38.html
They say +/-80 ms is unnoticeable. Well others disagree with that statement and we are not the first ones to think about it:
I know I can notice +10/-20ms, as synchronizing audio to video is something I do almost daily for over 10 years now.
"http://www.hifi-forum.de/viewthread-42-38.html
Gemäss den Angaben in den Richtlinien der ITU (ITU-R BT.1359-1 App. 1 [1998]) sind Asynchronitäten vom Durchschnittsfernsehzuschauer ab +45/-125ms erkennbar und bis +90/-180ms tolerierbar (plus bedeutet, dass der Ton dem Bild vorausläuft
Yes, I know that the game players are not „Durchschnittsfernsehzuschauer“ :) They are trained to have much better audio-visual sensors than the average…
For those tests to work we need a proper audio driver (without using a binary blob) for the TWL.
Would that be a suggestion?
Yes, looks good to me.
I can prepare the Panda-ES for speakers.
BR, Nikolaus