Firmware & BIOS; x86 vs ARM etc. (rambling; curious)


Haraldur

Very Active Member
Joined
Mar 12, 2015
Messages
300
(This is not a topic about the choice of x86_64 vs ARM for the SoC of the Pyra -- I would mildly favour an x86_64 SoC, if possible, for the extra compatibility with my GNU/Linux PC system but, since I expect the form factor of the Pyra to be rather awkward for a significant number of the relevant programs (first-person shooters, real-time strategy, grand strategy (I cannot see there being a good experience with Crusader Kings 2 or Hearts of Iron 4 on such a small screen) or anything both fast-paced and mouse-based) I have no particular preference.) [/LISP]

Having read through the threads about the SoC choices & availability, I came across an argument, in favour of ARM rather than x86_64, that my Computer Science degree has not adequately prepared me for. That is the argument that x86_64 should be avoided, as one would have to write a BIOS for the processor to function, whereas for ARM one would not. I read this and I thought: "That is interesting, but why is it so?". I know almost nothing about ARM (other than that it is RISC and "cleaner than x86", allegedly) as I have never (until the Pandora, at least) had a non-x86(/_64) device for which I cared one jot, while I have been an enthusiastic (personal/home) computer user since childhood (no consoles, only DOS/Windows & GNU/Linux).

My understanding is that, for a GNU/Linux PC, the BIOS/(U)EFI is a program, loaded from flash memory on the motherboard, that checks a specific place (Master boot record for BIOS) for a pointer to another program to load. This other program is either an operating system kernel image or a bootloader that can further point to other images, typically operating system kernels. An operating system kernel then grabs data from a preconfigured position on a storage medium of some kind (in my setup, an SSD) that it treats as a filesystem and runs an init system (PID 1 in Unix-parlance) that runs other programs to build-up the user environment, including a login system, window manager, desktop environment etc.. The kernel also has access to/responsibility for restricted x86(_64) commands, virtual memory tables (needed by the CPU), process/thread CPU/core allocation tables (also needed by the CPU) and some other tasks. It is obvious how the x86(_64) system is dependent on the existence of an OS (but what about PC-booters?), but it is not so clear to me why the BIOS is required. It seems to me conceivable that there would be an x86(_64) machine that loaded an OS directly without going via BIOS.

Questions:

1. (For the above, please correct me if & where I am wrong/incomplete)

2. What is the ARM/Pandora/Pyra bootstrapping process? Why is it simpler -- why is the x86_64 one more difficult (and is that related to the PC hardware specification being relatively "open", with freely installable OSes etc.; alternately, is this why ARM devices tend to be so locked-down/crippled?)?

3. I am led to believe that there is a free(?) BIOS implementation called Coreboot/Libreboot. Is this relevant?

4. I have been fiddling with the programming language FORTH, recently (LISP/Scheme/Clojure is next!). It appears that there is some kind of bootloader/BIOS/firmware program based on this language, called Open Firmware. It was developed by Mitch Bradley for Sun Microsystems and used for the servers they sold (with, I believe, a variety of CPUs and Instruction Set Architectures in use: at least PowerPC and SPARC) and was more recently used by the One Laptop Per Child project (using an x86, I believe), with supposed advantages being interactive debugging with the Forth interpreter and mass-firmware-deployment. http://wiki.laptop.org/go/Open_Firmware http://wiki.laptop.org/go/OFW_FAQ https://en.wikipedia.org/wiki/Open_Firmware . Is this in any way relevant to the Pandora or the Pyra?

5. There is some terminology around the Pandora, the Pyra and the Nokia N900 that confuses me. Text describes "flashing the firmware" in terms that seem to me like installing an OS. Why is this? Is the OS truly on a normally-inaccessible storage medium? If so, why? Is there no way to insert a USB-storage device and boot from it with a menu like on a PC?

Thanks in advance. :)
 
Well this is a huge amount of topics to answer and a lot of it is a bit over my capacity to answer, but for item 5...

There is some terminology around the Pandora, the Pyra and the Nokia N900 that confuses me. Text describes "flashing the firmware" in terms that seem to me like installing an OS. Why is this? Is the OS truly on a normally-inaccessible storage medium? If so, why? Is there no way to insert a USB-storage device and boot from it with a menu like on a PC?
 

To me the term firmware is a big misnomer in my opinion.. It's really just writing the bootloader and OS filesystem to the internal NAND storage. I guess it's an easy term for new users...

The Pandora and Pyra/OMAP5 Devboard use a bootloader called u-boot.. The Pandora does have a boot menu that simplifies booting from a SD card, but it still a little more low level than what most are used to on a PC... There are two methods of booting off an SD card I know of,  One requires a multi-partition SD card. one partition is a FAT formatted boot partition that has the bootloader and supporting information on how to boot the Linux kernel and then the typical Linux EXT formatted partition that contains the Linux file system often called the (rootfs).  The other method which I've only seen used on the Pandora, but I'm sure it could work on other systems is a single EXT formatted partition that contains the Linux rootfs and a text file that resides in the root directory named autoboot.txt that contains the boot arguments, where to locate the Linux kernel to load and what not...  Configuring how your ARM device boots can be tricky, there seems to be several ways to approach it, I'm not really an expert, I've just been dealing with ARM based devices for 7 or 8 years now and picked up a few things. 

Edit: cleaned up some of my writing.
 
Last edited by a moderator:
I don't know the Pandora's boot process except as a user, but on previous ARM-based systems the boot process is:

1. Execute the instruction at address 0x00000000 on NAND (usually a branch instruction that jumps to the first line of the kernel)

2. Profit!

I suspect that's how the Pandora's uboot system launches itself, which then goes and looks for filesystems to launch a kernel from according to a boot.txt/autoboot.txt configuration file.  Once it's launched the kernel, it brings the system up much like any other SYSV-based linux system.

In contrast my PC's BIOS does a lot more, as I understand them.  The name gives the hint; basic input/output system.  It mediates access to things like the keyboard, and sets up accessors for those peripherals.  On an ARM system, devices map certain special addresses to their control registers, and you talk to them by writing over those addresses or reading them.

It's a lighter-weight system than the PC's BIOS system, and to be honest the PC BIOS hasn't really been very relevant since people started adding more busses to them, which AIUI the BIOS doesn't mediate.  UEFI controls a lot more though, and under my limited understanding, continues to run as an active process kind of like a hypervisor even after the OS boots. I don't own a system that uses UEFI yet though, so I haven't really researched that beyond idle curiosity.

It's theoretically possible to add a BIOS to an ARM system, but writing a boot/autoboot.txt file is simpler than writing a MBR/VBR, and you'd need to completely reconfigure the linux kernel to use the BIOS rather than its memory mapped registers.  I don't honestly see the point, especially as you'd need to define what an MBR looks like on an ARM system - on a PC it's 440 bytes of x86 code IIRC.
 
Last edited by a moderator:
The OMAP has a multi-stage boot process.

First is the boot rom. Boot ROM is stored on the chip itself in a tiny tiny bit of ROM. The OMAP contains a tiny amount of ROM and RAM inside the chip itself and when it first boots that is all it has access to. The bootrom sets the necessary flags for it to access the attached RAM and storage. This happens before you're aware the thing is even powered.

The second step is the X-loader. The last thing the bootrom does is load some chunk of data from the start of the storage device. The storage device may be selectable via input pins being high or low, but typically it is MMC1. The bootrom is too small to know much about file systems so it literally needs to be at the start of a FAT disk. The X-loader is loaded directly into the OMAPs internal memory and executed. The X-loader is a little larger, still limited, but it can do more: it sets up the UART for debugging, a bunch of other stuff, and most importantly it knows about file systems. If you have a debug terminal you should see some stuff from this. It can read from different file systems looking for the next step: the u-boot.

The u-boot is more complete. The X-loader is able to load it into onboard RAM instead of onchip, so it can be much larger. This is the Pandora's boot menu, where you press R-trigger when you start it up. It knows more about filesystems, it can load an EXT2/3 file, which is good because the last step is the kernel.

The kernel is the Linux kernel. It is a file somewhere on the storage medium which u-boot loads into memory. It then sets some parameters (such as root file system, very important if you want the kernel to actually do anything) and then executes that. And that kernel continues the process of loading the rest of the system.
 
If I understood ED correctly the term BIOS encompasses both the hard and software components that are required to get the processor to do anything useful (i.e. clock setup, microcode uploading, whatever). It seems to be the case that the available ARM SoCs integrate most of these components directly with the SoC while the X86 SoC under consideration (Intel something) requires quite a lot of these to be built by the system integrator (at least according to an unnamed Intel engineer to whom ED spoke on some fair or other). Due to the very low level nature of these components they require special know how that may be quite expensive to get hold of.


If correct this would imply that the BIOS problem isn't an inherent X86 problem but an indication that Intel SoCs are not (yet?) ready for Pandora like applicaions. The enthusiasm Intel seems to have shown when approached by ED may be an indication that they just don't know this market well enough instead of simply not caring, but who knows.
 
Last edited by a moderator:
The fundamental purposes of the BIOS are to initialize and test the system hardware components, and to load a boot loader or an operating system from a mass memory device. The BIOS additionally provides an abstraction layer for the hardware, i.e. a consistent way for application programs and operating systems to interact with the keyboard, display, and other input/output devices. Variations in the system hardware are hidden by the BIOS from programs that use BIOS services instead of directly accessing the hardware. MS-DOS (PC DOS), which was the dominant PC operating system from the early 1980s until the mid 1990s, relied on BIOS services for disk, keyboard, and text display functions. MS Windows NT, Linux, and other protected mode operating systems in general ignore the abstraction layer provided by the BIOS and do not use it after loading, instead accessing the hardware components directly.

Every BIOS implementation is specifically designed to work with a particular computer or motherboard model, by interfacing with various devices that make up the complementary system chipset. Originally, BIOS firmware was stored in a ROM chip on the PC motherboard; in modern computer systems, the BIOS contents are stored on flash memory so it can be rewritten without removing the chip from the motherboard.
 this from the wiki. what is in red is why a custom BIOS would have to be written to use any form of windows or DOS to work natively  with the pyra hardware.
 
As I understand it, the BIOS is the BootROM and X-Loader all in one, both of which in our case are provided by TI (the BootROM being burned to the chip already and the X-loader being available as source)

It does make me wonder why Intel doesn't have a BIOS reference that anyone can just use though. You'd think that would be in their best interest.
 
The OMAP has a multi-stage boot process.


First is the boot rom. Boot ROM is stored on the chip itself in a tiny tiny bit of ROM. The OMAP contains a tiny amount of ROM and RAM inside the chip itself and when it first boots that is all it has access to. The bootrom sets the necessary flags for it to access the attached RAM and storage. This happens before you're aware the thing is even powered.


The second step is the X-loader. The last thing the bootrom does is load some chunk of data from the start of the storage device. The storage device may be selectable via input pins being high or low, but typically it is MMC1. The bootrom is too small to know much about file systems so it literally needs to be at the start of a FAT disk. The X-loader is loaded directly into the OMAPs internal memory and executed. The X-loader is a little larger, still limited, but it can do more: it sets up the UART for debugging, a bunch of other stuff, and most importantly it knows about file systems. If you have a debug terminal you should see some stuff from this. It can read from different file systems looking for the next step: the u-boot.
So, to map that onto PC speak, it's a little as if the boot rom is the BIOS bootloader, X-loader is the MBR and uboot is GRUB/LILO/any other boot menu.  I'm sure there are specific differences (such as X-loader not being on a user disc), but they seem to map fairly well in terms of functionality and some constraints.
 
I guess a lot of the BIOS confusion arises from two usages for the word:

Any mechanism that bootstraps a machine to a state in which it's got a program read from (used to be, not any more) user modifiable memory can be referred to as a BIOS. Typically that thing can do more, like a limited set of SCSI, RAID, network configuration or simply be switched to boot automatically instead of requiring manual intervention (choosing bootable disk). Everyone used to have their own. But one of them actually called it simply BIOS. Just like one of them simply called the Other It DiskOperatingSystem. This was their idea of being user friendly...

However, now, most of the time when people refer to a BIOS they think "PC".

You can bootstrap an Intel processor without a traditional BIOS. If you don't have (example) a VGA/VESA type display, floppy disk controller based on the ancient crap IBM came up with, a hard disk controller backwards compatible with PATA, with IDE, with MFM... if you don't need any added stuff like ACPI, if you get all the chips uncommon in desktop motherboards and if you don't care jack about true Windows compatibility - it makes perfect sense to not use a "BIOS as in PC" (edit: should also have added the obvious ones here: memory controller might need some initing as well as any busses you intend to use). If you do, better get one of the commercial ready made ones.

But since a BIOS is relatively cheap compared to rolling your own, most manufacturers don't. That still does not guarantee Windows compliance, though. At least XP used to be really tight with regards to ACPI, for example...

Linux is another matter, you get it to boot, it runs quite happily. And if it doesn't, you fix the kernel.
 
Last edited by a moderator:
Thanks to everyone for the answers. :)

UEFI controls a lot more though, and under my limited understanding, continues to run as an active process kind of like a hypervisor even after the OS boots. I don't own a system that uses UEFI yet though, so I haven't really researched that beyond idle curiosity.
My instinctive reaction to that is one of horror -- my precious cycles! I notive every nanosecond it takes for Vim to start up. :)

That said, I have a UEFI system and have noticed nothing untoward. Although, I suppose there would be no easy way for me to tell: "top" would not show "UEFI" as a process.

If I understood ED correctly the term BIOS encompasses both the hard and software components that are required to get the processor to do anything useful (i.e. clock setup, microcode uploading, whatever). It seems to be the case that the available ARM SoCs integrate most of these components directly with the SoC while the X86 SoC under consideration (Intel something) requires quite a lot of these to be built by the system integrator (at least according to an unnamed Intel engineer to whom ED spoke on some fair or other). Due to the very low level nature of these components they require special know how that may be quite expensive to get hold of.

If correct this would imply that the BIOS problem isn't an inherent X86 problem but an indication that Intel SoCs are not (yet?) ready for Pandora like applicaions. The enthusiasm Intel seems to have shown when approached by ED may be an indication that they just don't know this market well enough instead of simply not caring, but who knows.
That is a mite puzzling, as, it seems to me, an SoC essentially contains the main elements of a motherboard and, for PCs, motherboard manufacturers provide the BIOS/UEFI.

The fundamental purposes of the BIOS are to initialize and test the system hardware components, and to load a boot loader or an operating system from a mass memory device. The BIOS additionally provides an abstraction layer for the hardware, i.e. a consistent way for application programs and operating systems to interact with the keyboard, display, and other input/output devices. Variations in the system hardware are hidden by the BIOS from programs that use BIOS services instead of directly accessing the hardware. MS-DOS (PC DOS), which was the dominant PC operating system from the early 1980s until the mid 1990s, relied on BIOS services for disk, keyboard, and text display functions. MS Windows NT, Linux, and other protected mode operating systems in general ignore the abstraction layer provided by the BIOS and do not use it after loading, instead accessing the hardware components directly.

Every BIOS implementation is specifically designed to work with a particular computer or motherboard model, by interfacing with various devices that make up the complementary system chipset. Originally, BIOS firmware was stored in a ROM chip on the PC motherboard; in modern computer systems, the BIOS contents are stored on flash memory so it can be rewritten without removing the chip from the motherboard.
 this from the wiki. what is in red is why a custom BIOS would have to be written to use any form of windows or DOS to work natively  with the pyra hardware.
Well, personally, I am not that interested in Windows or DOS use (though FreeDOS may be interesting to play with for a few minutes... if, for any reason, DOSBOX failed to suffice) as someone approaching the status of a GNU/Linux, FSF zealot (graphics drivers? I'll look the other way...). The main thing is that it is sometimes not possible to use (old) source code (it is lost, or dependent on machine-word size, or (partly) written in assembly) or it is much more convenient to take a binary (oh! I lack this library! And that one! I have the wrong (too new) version of X! Automake/autotools has failed, do I want to learn how this all works? No!).

As your quote says, Linux (and Windows NT) (mostly) ignores the BIOS abstraction layer, using its own drivers. I suppose that, with Wine (another compromise! -- as are the games...), programs would not care about the lack of a BIOS, or that Wine could be modified to fake the existence of a BIOS.

Quick question, can something like CoreBoot be applicable for a Pyra frame + x86 CPU board? I'm assuming it would not be straightforward because of the difference in hardware design, but could it used at all ? 

http://www.coreboot.org/
 I am curious about that, too, and also Open Firmware.

Everyone used to have their own. But one of them actually called it simply BIOS. Just like one of them simply called the Other It DiskOperatingSystem. This was their idea of being user friendly...

However, now, most of the time when people refer to a BIOS they think "PC".
It took me a long time to find out that GNOME's Document Viewer was really called Evince, so I could invoke it from the command-line -- annoying! About as bad as Microsoft's idiotic policy to hide file extensions in Windows by default.

At the very least it should be "Evince Document Viewer".

You can bootstrap an Intel processor without a traditional BIOS. If you don't have (example) a VGA/VESA type display, floppy disk controller based on the ancient crap IBM came up with, a hard disk controller backwards compatible with PATA, with IDE, with MFM... if you don't need any added stuff like ACPI, if you get all the chips uncommon in desktop motherboards and if you don't care jack about true Windows compatibility - it makes perfect sense to not use a "BIOS as in PC" (edit: should also have added the obvious ones here: memory controller might need some initing as well as any busses you intend to use). If you do, better get one of the commercial ready made ones.

But since a BIOS is relatively cheap compared to rolling your own, most manufacturers don't. That still does not guarantee Windows compliance, though. At least XP used to be really tight with regards to ACPI, for example...

Linux is another matter, you get it to boot, it runs quite happily. And if it doesn't, you fix the kernel.
Presumably drivers could be written for Linux to directly get at ACPI etc. should one wish it to. As far as Windows compatibility goes, as long as Wine works, I care not.
 
as someone approaching the status of a GNU+Linux, FSF zealot
FTFY
If I was dictator, Lignux. Or LiGNUx.


Or maybe GWL (GNU With Linux, pronounced like in Welsh, Gul (like Dukat!)).
"GNU(Linux)"

"GNU|Linux"

"GNU"

or just "GNU/Linux".

Or frankly, just "Linux". Or "Debian", "Ubuntu", etc. I rather emphasize "Free Software" (in the sense of Libre and as opposed to "just" open source), than to be picky about names.
 
Hm, so for an possible x86 Pyra CPU Module we would need an entire "real" PC BIOS then?
That was (part of) my question. I am not convinced that that is the case. It may be necessary for (some) versions of Microsoft Windows (who cares?), but it seems that the Linux kernel would be fine without it (perhaps with modification -- presumably WINE would be fine) as no-one has stated that the CPU itself is dependent on services provided by the BIOS (while it is dependent on services provided by the operating system).


Again, if I am wrong, I would like to be told so.



 



 




"GNU(Linux)"


"GNU|Linux"



"GNU"



 



or just "GNU/Linux".



 



Or frankly, just "Linux". Or "Debian", "Ubuntu", etc. I rather emphasize "Free Software" (in the sense of Libre and as opposed to "just" open source), than to be picky about names.





I prefer GNU/Linux to GNU+Linux (when I was younger I had a CD labelled something like Debian/XFCE/X/GNU/Linux  :rolleyes:  ) as it seems more commonly used -- though I pronounce it like "GNU Linux" or "GNU-Linux", as I am lazy.


 


I do use "Linux" alone quite often, but since Android is everywhere, I increasingly feel the need to discriminate. There are also some specialized userlands for embedded systems as well, using the Linux Kernel but no GNU tools. I would use "GNU" alone, but in most cases I would still have to mention Linux to make sure people understood me (and if the FSF/GNU Project do this, why shouldn't I?), even though there seems no viable alternative to the Linux kernel (indeed, do we need one?).


 


I do not like to use "Debian", "Ubuntu", "Arch" etc. alone as I find few things are so specific to those distributions, only. Besides, "Debian" is almost always really "Debian GNU/Linux" and "Arch" is officially (?) "Arch Linux" (and x86(_64) only). I also tend not to promote Ubuntu at this time, rather Linux Mint (though I am slightly irked by their update policy).
 
Gul (like Dukat!)).
:rolleyes:  There were a couple Cardassian episodes of TNG on here recently.  An odd character trait explored on DS9 was how O'Brien felt them.  I know other people that seem to use the same justification for being hateful toward others, and it seems especially out of place with him (I am pretty fond of engineers).

as someone approaching the status of a GNU+Linux, FSF zealot
FTFY
If I was dictator, Lignux. Or LiGNUx.


Or maybe GWL (GNU With Linux, pronounced like in Welsh, Gul (like Dukat!)).
"GNU(Linux)"

"GNU|Linux"

"GNU"

or just "GNU/Linux".

Or frankly, just "Linux". Or "Debian", "Ubuntu", etc. I rather emphasize "Free Software" (in the sense of Libre and as opposed to "just" open source), than to be picky about names.
The FSF crowd seem to want to put the emphasis not on GNU or Linux, but on the slash/plus/with.  I am a bit like you, as the free (libre) software  is much more important to me.  The only times the GNU/Linux thing has been an issue are the few times when I was trying to explain to people that Android has Linux, but it isn't Linux.  The first time was probably during an iPhone vs Android debate.  They pulled me in because I was the only Linux person they knew.  The Apple fans were kinda lost when I brought up BSD in response to their personal attacks against my OS choices, since they must not have known that what we used on our computers were cousins (I use "Linux" a lot, and BSD on some computers, amongst others).
 
Hm, so for an possible x86 Pyra CPU Module we would need an entire "real" PC BIOS then?
That was (part of) my question. I am not convinced that that is the case. It may be necessary for (some) versions of Microsoft Windows (who cares?), but it seems that the Linux kernel would be fine without it (perhaps with modification -- presumably WINE would be fine) as no-one has stated that the CPU itself is dependent on services provided by the BIOS (while it is dependent on services provided by the operating system).


Again, if I am wrong, I would like to be told so. 





Again: You are right: little modularity --> no strict NEED for even EFI much less for "PC BIOS". But tradeoff is: you need really smooth communication with all parties providing you HW (if only Intel does, great... but....) and you need time to roll your own. It just does not work that way.


Heck, yes, you could even cut out ACPI assuming manufacturers were happy to provide you with some pretty lowlevel details of their systems... That, however, is pretty much spot on why ACPI was nightmared up in the first place... they aren't: hello ACPI.


 


It's just that because it is easier to grab ready made things, this is not often done!


 


Now that you got me thinking I actually got curious about how the "Intel SOCs" based on Atom make things look from system point of view... It seems to be surprisingly difficult to get solid information on what exactly they expose (busses, connectors etc.). Most of the articles seem to just vent about "OMG-GHz-Cores-GPU-Teh-Works"... it appears they can run Windows, so they must "pose as a PC" at least optionally.


Simply logical thinking says if Intel had put "standardish" (from embedded industries) busses for communication, would they not say "lookee what we gots here - nice drop in ARM replacement". They aren't. The pessimist in me says those SoCs are PC-y contraptions bound to EFI-like solutions in practice.


If this rambling conjecture-filled musing of mine were to turn out accurate, the Intel CPU board for Pyra would be... interesting... it might well contain microcontrollers with some fairly amusing duties and those chips would take more space than the SoC ;-).


 


Also: correct me since I'm wrong, but hasn't Windows itself been bootable with EFI since 7? Not even need for "legacy BIOS" support from motherboard? Or do you feel EFI=BIOS enough?


 


Wine cares not what it runs on. BIOS is just 4 letters to Wine. But you knew that.


If you want a dependable system to natively BOOT your Windows XP, 2000, 98, 95: Then you need a BIOS and a damn robust one at that. This was what most of the hulabaloo threads always ended up to - those ole things would rock if they ran.


But really: get a new PC from Walmart, insert your XP install disc... dissappointment ahoy. The mythical Intel-Pyra would have been no different from those new Acers. I know I got an Acer that cannot boot into XP, though it does have a BIOS!
 
Back
Top