Hi, since I have not yet much progress on the MIPI (except that the DT additions compile and there is a 'lcd' node appearing somewhere in /sys), I have now tried to look at the HDMI port to get a better understanding.
But it does not work well.
I get this from the console when unplugging and replugging the cable:
[ 439.404266] hpd 0 [ 439.435455] omapdss: in hpd work 0, state=0 [ 439.439849] omapdss: hdmi_hotplug_detect_worker state = 0 [ 444.749298] hpd 1 [ 445.247863] omapdss: in hpd work 1, state=0 [ 445.253570] omapdss error: hdmi_hotplug_detect_worker state = 1 [ 445.318145] omapdss: in hpd work 7, state=0 [ 445.323699] omapdss HDMI error: M = 618 Mf = 196608 [ 445.329010] omapdss HDMI error: range = 1 sd = 6 [ 445.334045] HDCP: hdcp_3des_cb: hdcp_keys not loaded = 0 [ 445.334045] [ 445.353698] Failed to set PHY power mode to 2 [ 445.358428] omapdss error: Failed to enable PHY TX power [ 445.366119] omapdss HDMI error: failed to power on device [ 445.371826] omapdss error: failed to power on [ 445.376464] omapdss error: hdmi_hotplug_detect_worker state = 7
so it appears that my config is not correctly powering the hdmi port.
Any ideas/hint what I could test?
BR, Nikolaus
Am 27.01.2014 um 17:00 schrieb Dr. H. Nikolaus Schaller:
Hi, since I have not yet much progress on the MIPI (except that the DT additions compile and there is a 'lcd' node appearing somewhere in /sys), I have now tried to look at the HDMI port to get a better understanding.
But it does not work well.
I get this from the console when unplugging and replugging the cable:
[ 439.404266] hpd 0 [ 439.435455] omapdss: in hpd work 0, state=0 [ 439.439849] omapdss: hdmi_hotplug_detect_worker state = 0 [ 444.749298] hpd 1 [ 445.247863] omapdss: in hpd work 1, state=0 [ 445.253570] omapdss error: hdmi_hotplug_detect_worker state = 1 [ 445.318145] omapdss: in hpd work 7, state=0 [ 445.323699] omapdss HDMI error: M = 618 Mf = 196608 [ 445.329010] omapdss HDMI error: range = 1 sd = 6 [ 445.334045] HDCP: hdcp_3des_cb: hdcp_keys not loaded = 0 [ 445.334045] [ 445.353698] Failed to set PHY power mode to 2 [ 445.358428] omapdss error: Failed to enable PHY TX power [ 445.366119] omapdss HDMI error: failed to power on device [ 445.371826] omapdss error: failed to power on [ 445.376464] omapdss error: hdmi_hotplug_detect_worker state = 7
so it appears that my config is not correctly powering the hdmi port.
Any ideas/hint what I could test?
More observations:
I tried i2cdetect -r -y 5
and w/o cable it is fast to find nothing. But with cable it is slow - and also finds nothing.
I had another computer connected to the DVI port of the monitor and after unplugging, I really got the LXDE X-Server! Although the errors were present.
But only for ~10 seconds. Then:
[ 643.021820] [drm:omap_crtc_error_irq] *ERROR* tv: errors: 00008000
So at least I know that it *can* work :)
On next boot I did see this error message in the beginning but the HDMI display was stable.
Now let's grab a mouse and keyboard :)
BR, Nikolaus
Am 27.01.2014 um 17:12 schrieb Dr. H. Nikolaus Schaller:
Am 27.01.2014 um 17:00 schrieb Dr. H. Nikolaus Schaller:
Hi, since I have not yet much progress on the MIPI (except that the DT additions compile and there is a 'lcd' node appearing somewhere in /sys), I have now tried to look at the HDMI port to get a better understanding.
But it does not work well.
I get this from the console when unplugging and replugging the cable:
[ 439.404266] hpd 0 [ 439.435455] omapdss: in hpd work 0, state=0 [ 439.439849] omapdss: hdmi_hotplug_detect_worker state = 0 [ 444.749298] hpd 1 [ 445.247863] omapdss: in hpd work 1, state=0 [ 445.253570] omapdss error: hdmi_hotplug_detect_worker state = 1 [ 445.318145] omapdss: in hpd work 7, state=0 [ 445.323699] omapdss HDMI error: M = 618 Mf = 196608 [ 445.329010] omapdss HDMI error: range = 1 sd = 6 [ 445.334045] HDCP: hdcp_3des_cb: hdcp_keys not loaded = 0 [ 445.334045] [ 445.353698] Failed to set PHY power mode to 2 [ 445.358428] omapdss error: Failed to enable PHY TX power [ 445.366119] omapdss HDMI error: failed to power on device [ 445.371826] omapdss error: failed to power on [ 445.376464] omapdss error: hdmi_hotplug_detect_worker state = 7
so it appears that my config is not correctly powering the hdmi port.
Any ideas/hint what I could test?
More observations:
I tried i2cdetect -r -y 5
and w/o cable it is fast to find nothing. But with cable it is slow - and also finds nothing.
I had another computer connected to the DVI port of the monitor and after unplugging, I really got the LXDE X-Server! Although the errors were present.
But only for ~10 seconds. Then:
[ 643.021820] [drm:omap_crtc_error_irq] *ERROR* tv: errors: 00008000
So at least I know that it *can* work :)
On next boot I did see this error message in the beginning but the HDMI display was stable.
Now let's grab a mouse and keyboard :)
Ok, I could easily reproduce the problem reported by ED.
The sequence was just to open some window and move it fast around the screen.
After a while, the X-server crashes with this log:
X.Org X Server 1.12.4 Release Date: 2012-08-27 X Protocol Version 11, Revision 0 Build Operating System: Linux 3.2.0-4-mx5 armv7l Debian Current Operating System: Linux gta04 3.8.13 #322 SMP PREEMPT Mon Jan 27 14:59:29 CET 2014 armv7l Kernel command line: elevator=noop console=ttyO2,115200n8 root=/dev/mmcblk0p2 rw rootwait earlyprintk fixrtc Build Date: 21 October 2013 06:40:59PM xorg-server 2:1.12.4-6+deb7u1 (Moritz Muehlenhoff jmm@debian.org) Current version of pixman: 0.26.0 Before reporting problems, check http://wiki.x.org to make sure that you have the latest version. Markers: (--) probed, (**) from config file, (==) default setting, (++) from command line, (!!) notice, (II) informational, (WW) warning, (EE) error, (NI) not implemented, (??) unknown. (==) Log file: "/var/log/Xorg.0.log", Time: Sat Jan 1 00:46:50 2000 (==) Using config file: "/etc/X11/xorg.conf" (==) Using system config directory "/usr/share/X11/xorg.conf.d" [ 569.206298] omapdss HDMI error: M = 618 Mf = 196608 [ 569.206329] omapdss HDMI error: range = 1 sd = 6 [ 569.206329] HDCP: hdcp_3des_cb: hdcp_keys not loaded = 0 [ 569.206329] [ 569.206451] Enter hdmi_core_init [ 569.206512] Enable video_path [ 569.206542] [drm:omap_crtc_error_irq] *ERROR* tv: errors: 00008000
Backtrace:
Segmentation fault at address (nil)
Fatal server error: Caught signal 11 (Segmentation fault). Server aborting
Please consult the The X.Org Foundation support at http://wiki.x.org for help. Please also check the log file at "/var/log/Xorg.0.log" for additional information.
Server terminated with error (1). Closing log file. xinit: connection to X server lost
_XSERVTransSocketOpenCOTSServer: Unable to open socket for inet6 _XSERVTransOpen: transport open failed for inet6/gta04:0 _XSERVTransMakeAllCOTSServerListeners: failed to open listener for inet6
X.Org X Server 1.12.4 Release Date: 2012-08-27 X Protocol Version 11, Revision 0 Build Operating System: Linux 3.2.0-4-mx5 armv7l Debian Current Operating System: Linux gta04 3.8.13 #322 SMP PREEMPT Mon Jan 27 14:59:29 CET 2014 armv7l Kernel command line: elevator=noop console=ttyO2,115200n8 root=/dev/mmcblk0p2 rw rootwait earlyprintk fixrtc Build Date: 21 October 2013 06:40:59PM xorg-server 2:1.12.4-6+deb7u1 (Moritz Muehlenhoff jmm@debian.org) Current version of pixman: 0.26.0 Before reporting problems, check http://wiki.x.org to make sure that you have the latest version. Markers: (--) probed, (**) from config file, (==) default setting, (++) from command line, (!!) notice, (II) informational, (WW) warning, (EE) error, (NI) not implemented, (??) unknown. (==) Log file: "/var/log/Xorg.0.log", Time: Sat Jan 1 00:47:16 2000 (==) Using config file: "/etc/X11/xorg.conf" (==) Using system config directory "/usr/share/X11/xorg.conf.d" [ 594.981750] omapdss HDMI error: M = 618 Mf = 196608 [ 594.981750] omapdss HDMI error: range = 1 sd = 6 [ 594.981750] HDCP: hdcp_3des_cb: hdcp_keys not loaded = 0 [ 594.981750] [ 594.981903] Enter hdmi_core_init [ 594.981933] Enable video_path [ 594.981964] [drm:omap_crtc_error_irq] *ERROR* tv: errors: 00008000
But I am *not* sure if the segfault is triggered by the
[ 569.206542] [drm:omap_crtc_error_irq] *ERROR* tv: errors: 00008000
It more appears to come after the new X-Server is started.
BR, Nikolaus
Am Mon, 27 Jan 2014 17:21:29 +0100 hat "Dr. H. Nikolaus Schaller" hns@goldelico.com geschrieben:
Hi,
I get this from the console when unplugging and replugging the cable:
[ 439.404266] hpd 0 [ 439.435455] omapdss: in hpd work 0, state=0 [ 439.439849] omapdss: hdmi_hotplug_detect_worker state = 0 [ 444.749298] hpd 1 [ 445.247863] omapdss: in hpd work 1, state=0 [ 445.253570] omapdss error: hdmi_hotplug_detect_worker state = 1 [ 445.318145] omapdss: in hpd work 7, state=0 [ 445.323699] omapdss HDMI error: M = 618 Mf = 196608 [ 445.329010] omapdss HDMI error: range = 1 sd = 6 [ 445.334045] HDCP: hdcp_3des_cb: hdcp_keys not loaded = 0 [ 445.334045] [ 445.353698] Failed to set PHY power mode to 2 [ 445.358428] omapdss error: Failed to enable PHY TX power [ 445.366119] omapdss HDMI error: failed to power on device [ 445.371826] omapdss error: failed to power on [ 445.376464] omapdss error: hdmi_hotplug_detect_worker state = 7
Those errors seem normal, I get them as well, but HDMI works fine.
[ 594.981750] omapdss HDMI error: M = 618 Mf = 196608 [ 594.981750] omapdss HDMI error: range = 1 sd = 6 [ 594.981750] HDCP: hdcp_3des_cb: hdcp_keys not loaded = 0 [ 594.981750] [ 594.981903] Enter hdmi_core_init [ 594.981933] Enable video_path [ 594.981964] [drm:omap_crtc_error_irq] *ERROR* tv: errors: 00008000 But I am *not* sure if the segfault is triggered by the [ 569.206542] [drm:omap_crtc_error_irq] *ERROR* tv: errors: 00008000 It more appears to come after the new X-Server is started.
I don't these errors are the cause for the segfault. These entries appear, as you mentioned, when you start the X Server. I've been running the PSX emulators for 1,5 hours straight on X (as long as you don't do much else, it doesn't crash) That entry was already in the log at that time.
And then I moved some windows around and it segfaulted. Without any additional log entry.
Seems like we need to enable some verbose output of X.
On 01/27/2014 06:36 PM, Dr. H. Nikolaus Schaller wrote:
Am 27.01.2014 um 17:29 schrieb Michael Mrozek:
Seems like we need to enable some verbose output of X.
I think we should edit /etc/X11/xinit/xserverrc and add
-logverbose 20
to exec /usr/bin/X
I've looked at this and it's stack corruption of some sort in Xorg itself, somewhere in input handling code to be more precise, not in fbdev or some plugin. A function picks up address 0 from stack as return address, and jumps to it. This doesn't even give meaningful backtrace, so is really difficult to debug. It might be something that only shows up on ARM, so is not fixed upstream. Seems to be triggered by input events, not drawing related.
Maybe we can pick up another Xorg version from Debian stable or something, I'm not sure this can be fixed until the weekend.
Grazvydas
Am Tue, 28 Jan 2014 04:07:09 +0200 hat Grazvydas Ignotas grazvydas.ignotas@openpandora.org geschrieben:
Hi,
thanks for testing that.
Hmm, are you sure it's really triggered by input events? When I run Iceweasel and load the website "www.kleiderkreisel.de", X crashes as soon as the page is loaded about 70 - 80% (which can take up to 5 - 8 seconds, depending on internet speed).
And it always crashes while loading the website, even if I don't touch any input device...
On the other hand, I can move the mouse around like hell or hack the keyboard - no crash at all.
Sounds a bit weird.
Well, we started with the stable version of Wheezy, and it happened there as well. Older versions are only armel, not armhf, AFAIR.
So we can't do much here.
What I could try though is grabbing the X packages from Ubuntu ARM.
Or does TI have a rootfs available? I know they got Yocto, but do they have it recompiled or do you need to setup your own buildserver...?
I've looked at this and it's stack corruption of some sort in Xorg itself, somewhere in input handling code to be more precise, not in fbdev or some plugin. A function picks up address 0 from stack as return address, and jumps to it. This doesn't even give meaningful backtrace, so is really difficult to debug. It might be something that only shows up on ARM, so is not fixed upstream. Seems to be triggered by input events, not drawing related.
Maybe we can pick up another Xorg version from Debian stable or something, I'm not sure this can be fixed until the weekend.
Grazvydas _______________________________________________ Kernel mailing list Kernel@pyra-handheld.com http://pyra-handheld.com/cgi-bin/mailman/listinfo/kernel
Am 28.01.2014 um 04:32 schrieb Michael Mrozek:
Am Tue, 28 Jan 2014 04:07:09 +0200 hat Grazvydas Ignotas grazvydas.ignotas@openpandora.org geschrieben:
Hi,
thanks for testing that.
Hmm, are you sure it's really triggered by input events? When I run Iceweasel and load the website "www.kleiderkreisel.de", X crashes as soon as the page is loaded about 70 - 80% (which can take up to 5 - 8 seconds, depending on internet speed).
And it always crashes while loading the website, even if I don't touch any input device...
On the other hand, I can move the mouse around like hell or hack the keyboard - no crash at all.
Yes, that was also my impression that moving the window and not moving the mouse/cursor was the trigger.
Sounds a bit weird.
Well, we started with the stable version of Wheezy, and it happened there as well. Older versions are only armel, not armhf, AFAIR.
So we can't do much here.
What I could try though is grabbing the X packages from Ubuntu ARM.
Well, the same X-org binary works well on the GTA04 - although I never have tested to heavily move around windows... The screen is small and there is no mouse :)
So what is different between the Xorg running on GTA04 and EVM? * CPU DM3730 -> OMAP5432 * more RAM * HDMI display driver * omapfb -> fbdev * screen/framebuffer dimensions * different input devices * anything else?
One of these must be the problem.
Or does TI have a rootfs available? I know they got Yocto, but do they have it recompiled or do you need to setup your own buildserver...?
I've looked at this and it's stack corruption of some sort in Xorg itself, somewhere in input handling code to be more precise, not in fbdev or some plugin. A function picks up address 0 from stack as return address, and jumps to it. This doesn't even give meaningful backtrace, so is really difficult to debug. It might be something that only shows up on ARM, so is not fixed upstream. Seems to be triggered by input events, not drawing related.
Well, it could be that drawing (moving a window) damages the stack and then the *next* input event fails. Or some race between window manager triggered events and input events from the mouse.
Maybe we can pick up another Xorg version from Debian stable
My test was with the standard Xorg from Debian Wheezy (stable).
So I think it is a bug that nobody else has or nobody has fixed so far.
or something, I'm not sure this can be fixed until the weekend.
We could try to make the omapfb work because it is most likely only a kernel/board file configuration issue that it is not loading.
Grazvydas _______________________________________________ Kernel mailing list Kernel@pyra-handheld.com http://pyra-handheld.com/cgi-bin/mailman/listinfo/kernel
-- Mit freundlichen Grüßen,
Michael Mrozek
OpenPandora GmbH Geschäftsführer: Michael Mrozek
Schäffbräustr. 11 85049 Ingolstadt Deutschland Tel.: 0841 / 13 177 33 http://www.openpandora.de/ HRB 4879, Amtsgericht Ingolstadt
ICQ: 15691903 MSN: EvilDragon1717@hotmail.com eMail: info@openpandora.de _______________________________________________ Kernel mailing list Kernel@pyra-handheld.com http://pyra-handheld.com/cgi-bin/mailman/listinfo/kernel
On Tue, Jan 28, 2014 at 8:05 AM, Dr. H. Nikolaus Schaller hns@goldelico.com wrote:
Am 28.01.2014 um 04:32 schrieb Michael Mrozek:
Am Tue, 28 Jan 2014 04:07:09 +0200 hat Grazvydas Ignotas grazvydas.ignotas@openpandora.org geschrieben:
I've looked at this and it's stack corruption of some sort in Xorg itself, somewhere in input handling code to be more precise, not in fbdev or some plugin. A function picks up address 0 from stack as return address, and jumps to it. This doesn't even give meaningful backtrace, so is really difficult to debug. It might be something that only shows up on ARM, so is not fixed upstream. Seems to be triggered by input events, not drawing related.
Well, it could be that drawing (moving a window) damages the stack and then the *next* input event fails. Or some race between window manager triggered events and input events from the mouse.
Yeah it's probably something like that. Actually what is crashing is some event dispatcher, so it might actually be not input event but some redraw event that triggers the corruption.
Gražvydas
On Tue, Jan 28, 2014 at 1:59 PM, Grazvydas Ignotas notasas@gmail.com wrote:
On Tue, Jan 28, 2014 at 8:05 AM, Dr. H. Nikolaus Schaller hns@goldelico.com wrote:
Am 28.01.2014 um 04:32 schrieb Michael Mrozek:
Am Tue, 28 Jan 2014 04:07:09 +0200 hat Grazvydas Ignotas grazvydas.ignotas@openpandora.org geschrieben:
I've looked at this and it's stack corruption of some sort in Xorg itself, somewhere in input handling code to be more precise, not in fbdev or some plugin. A function picks up address 0 from stack as return address, and jumps to it. This doesn't even give meaningful backtrace, so is really difficult to debug. It might be something that only shows up on ARM, so is not fixed upstream. Seems to be triggered by input events, not drawing related.
Well, it could be that drawing (moving a window) damages the stack and then the *next* input event fails. Or some race between window manager triggered events and input events from the mouse.
Yeah it's probably something like that. Actually what is crashing is some event dispatcher, so it might actually be not input event but some redraw event that triggers the corruption.
Gražvydas
I've compiled a version of Xorg with debug symbls and it works (writing this from the EVM), so the problem either got hidden away or there is a compiler bug in gcc Debian uses for armhf build, or something else..
Anyway this should be good solution for now, run unxz on this file and copy to /usr/bin/Xorg .
Gražvydas
Hi,
just a quick notice: Your patch did work, running a full desktop with heavy loaded stuff for a while now.
Thanks a lot :D
What power that PCB has... ever wanted to run KDE4, Gimp, LibreOffice, FireFox and PSX emulation at the same time on a handheld?
Okay, doesn't make sense - but it works (see attached image)
CPU usage isn't that bad either and the desktop stays response (well, as responsive as KDE4 is, as it's a lot slower than XFCE4 ;))
That's some nice presentation I can do at FOSDEM.
Hi, I have packed it into a debian package that can be installed over the existing xorg.
To install (e.g. after creating a new and fresh SD card)
apt-get update apt-get install xserver-xorg-core-evm
Since it is a stripped binary, it is even smaller (1.8 MB).
It just replaces /usr/bin/Xorg
BR, Nikolaus
PS: the .deb is here:
http://download.goldelico.com/gta04/debian/dists/experimental/main/binary-ar...
Am 28.01.2014 um 21:10 schrieb Grazvydas Ignotas:
On Tue, Jan 28, 2014 at 1:59 PM, Grazvydas Ignotas notasas@gmail.com wrote:
On Tue, Jan 28, 2014 at 8:05 AM, Dr. H. Nikolaus Schaller hns@goldelico.com wrote:
Am 28.01.2014 um 04:32 schrieb Michael Mrozek:
Am Tue, 28 Jan 2014 04:07:09 +0200 hat Grazvydas Ignotas grazvydas.ignotas@openpandora.org geschrieben:
I've looked at this and it's stack corruption of some sort in Xorg itself, somewhere in input handling code to be more precise, not in fbdev or some plugin. A function picks up address 0 from stack as return address, and jumps to it. This doesn't even give meaningful backtrace, so is really difficult to debug. It might be something that only shows up on ARM, so is not fixed upstream. Seems to be triggered by input events, not drawing related.
Well, it could be that drawing (moving a window) damages the stack and then the *next* input event fails. Or some race between window manager triggered events and input events from the mouse.
Yeah it's probably something like that. Actually what is crashing is some event dispatcher, so it might actually be not input event but some redraw event that triggers the corruption.
Gražvydas
I've compiled a version of Xorg with debug symbls and it works (writing this from the EVM), so the problem either got hidden away or there is a compiler bug in gcc Debian uses for armhf build, or something else..
Anyway this should be good solution for now, run unxz on this file and copy to /usr/bin/Xorg .
Gražvydas <Xorg.xz>
Okay, so my measurement device came and I was able to measure the heatsink.
With normal load (desktop, PSX Emulator), the heatsink had about 50°C. With full CPU load (stress test), it went up to about 64°C.
No idea how much the GPU can do, as I don't have a way to stresstest it as well.
Even though this is with heatsink, I don't think heating will be such a huge issue when we apply some small but wide area metal to the SoC.
Am 29.01.2014 um 12:47 schrieb Michael Mrozek:
Okay, so my measurement device came and I was able to measure the heatsink.
With normal load (desktop, PSX Emulator), the heatsink had about 50°C. With full CPU load (stress test), it went up to about 64°C.
That is pretty reasonable. I remember that the GTA04 chip comes to ~70C.
The typical temperature inside the GTA04 (case closed) is 45C with 22C outside..
No idea how much the GPU can do, as I don't have a way to stresstest it as well.
Even though this is with heatsink, I don't think heating will be such a huge issue when we apply some small but wide area metal to the SoC.
Maybe we should also try to measure the power demand. Since all Volt*Amperes must be converted into heat. The typical value of an OMAP3 is 3.7V*400-1000mA. I.e. 2-4W. Only if the OMAP5 system can draw significantly more, we have a problem.
BR, Nikolaus
-- Mit freundlichen Grüßen,
Michael Mrozek
OpenPandora GmbH Geschäftsführer: Michael Mrozek
Schäffbräustr. 11 85049 Ingolstadt Deutschland Tel.: 0841 / 13 177 33 http://www.openpandora.de/ HRB 4879, Amtsgericht Ingolstadt
ICQ: 15691903 MSN: EvilDragon1717@hotmail.com eMail: info@openpandora.de _______________________________________________ Kernel mailing list Kernel@pyra-handheld.com http://pyra-handheld.com/cgi-bin/mailman/listinfo/kernel
Am Wed, 29 Jan 2014 12:51:55 +0100 hat "Dr. H. Nikolaus Schaller" hns@goldelico.com geschrieben:
Well, it seems the OMAP5 heating sensor does work as well, and it seems to measure the direct heating (below the heating pipe).
The maximum we could get with stress test is 74°C with that one. As notaz mentioned already, we have no idea how accurate it is, but it goes up and down parallel with my measurements (it's just a bit hotter).
Am 29.01.2014 um 12:47 schrieb Michael Mrozek:
Okay, so my measurement device came and I was able to measure the heatsink.
With normal load (desktop, PSX Emulator), the heatsink had about 50°C. With full CPU load (stress test), it went up to about 64°C.
That is pretty reasonable. I remember that the GTA04 chip comes to ~70C.
The typical temperature inside the GTA04 (case closed) is 45C with 22C outside..
No idea how much the GPU can do, as I don't have a way to stresstest it as well.
Even though this is with heatsink, I don't think heating will be such a huge issue when we apply some small but wide area metal to the SoC.
Maybe we should also try to measure the power demand. Since all Volt*Amperes must be converted into heat. The typical value of an OMAP3 is 3.7V*400-1000mA. I.e. 2-4W. Only if the OMAP5 system can draw significantly more, we have a problem.
BR, Nikolaus
-- Mit freundlichen Grüßen,
Michael Mrozek
OpenPandora GmbH Geschäftsführer: Michael Mrozek
Schäffbräustr. 11 85049 Ingolstadt Deutschland Tel.: 0841 / 13 177 33 http://www.openpandora.de/ HRB 4879, Amtsgericht Ingolstadt
ICQ: 15691903 MSN: EvilDragon1717@hotmail.com eMail: info@openpandora.de _______________________________________________ Kernel mailing list Kernel@pyra-handheld.com http://pyra-handheld.com/cgi-bin/mailman/listinfo/kernel
Kernel mailing list Kernel@pyra-handheld.com http://pyra-handheld.com/cgi-bin/mailman/listinfo/kernel