Looking for benchmarking tools


Grench

Forum Addict!
Joined
Oct 3, 2008
Messages
6,629
Looking for some utilities to do the following:


1. Track throughput over 802.11 and bluetooth networking.


2. Track disk (SD card) throughput (MB/sec) at the partitioned/formated level not the root deivice. I.e. I want to measure throughput to /dev/mmcblk0p1 ongoing - not /dev/mmcblk0.


I want to be able to compare throughput on partitions with different formatting.


I redid my whole Pandora recently as ext3 partitions. Frankly, it slowed down. A lot. Opening programs takes 2x+ as long. Even Vultures Eye can lag now.


So - I'm headed back to everything as ext2. However, I'd like to take a card, put 4 differently formatted partitions on it (fat32, ext2, ext3, fat(16/DOS)) on it and run some benchmarks.


Is there a utility out there or am I going to be stuck writing a bash script to print the time, copy a big file, print the time?


Thanks!
 
[...], I'd like to take a card, put 4 differently formatted partitions on it (fat32, ext2, ext3, fat(16/DOS)) on it and run some benchmarks.
It's extremely difficult to benchmark SD cards because most of them are made of proprietary black voodoo and may change their characteristics depending on how and how much they are being used.


What kind and brand of SD card did you use?
 
[...], I'd like to take a card, put 4 differently formatted partitions on it (fat32, ext2, ext3, fat(16/DOS)) on it and run some benchmarks.
It's extremely difficult to benchmark SD cards because most of them are made of proprietary black voodoo and may change their characteristics depending on how and how much they are being used.


What kind and brand of SD card did you use?

Kind and brand of card should be relatively irrelevant to this test. The objective is to see which disk format (Ext2, Ext3, Fat32, Fat) gets the best speed out of the Pandora.


I've used both Ext2 and Ext3 on the same cards (more than 1 brand/type) and have noticed that Ext3 is substantially slower than Ext2 - as in very noticeable from a user perspective.


But, since you asked, I plan to run these tests on a Patriot LX 32GB SDHC class 10 card. My 128GB Lexar card is already re-formatted into a single EXT2 partition for use as a media storage card. I tried it both as Ext3 and Ext2 - the difference was VERY noticeable with Ext2 being much much faster.


Your mileage may vary.

When I need to test I/O performance, I usually reach for Bonnie++.

Is there a PND or ARM/Pandora compiled version of that somewhere?
 
Is there a PND or ARM/Pandora compiled version of that somewhere?
My first place to look was StreaK's CLI love (wasn't in there). I don't know of a stand alone PND, but it does appear to be in Angstrom's Repo.


In general, ext3 should perform a bit slower than ext2 (journaling does that), but I didn't expect it to be too noticeable. I'm interested to see what your benchmarking results are.
 
Kind and brand of card should be relatively irrelevant to this test. The objective is to see which disk format (Ext2, Ext3, Fat32, Fat) gets the best speed out of the Pandora.
The raw flash in an SD card is abstracted by the SD card's controller. Thus the controller is always involved, and as a result the way the controller of a given SD card is working, as well as the particular type of NAND used, are of high relevance to the resulting speed. Some cards will work well with some file systems, others won't, and even others will work well in the beginning but eventually cause problems apparently out of nowhere.


The SD card market is very competitive, which means many SD card components will employ technical tricks to increase some qualities such as speed and lifespan. Sometimes some qualities, such as high lifespan, are sacrificed for qualities that are more easily marketed, such as higher (initial) speed (under "average" conditions), or more space per <MONEY>.


To my unrepresentative(!) experience, a speed difference of more than factor 2 between ext2 and ext3 indicates that the card being used likely has quite low quality NAND and/or uses some rather heavy file-system specific tricks, and thus any benchmarks made using that card would likely be quite useless.


Have you been using 'noatime'? If not, you could try whether using it noticeably affects the difference between ext2 and ext3, and then take further actions accordingly (such as looking into "alignment", or more robust SD cards).
 
Last edited by a moderator:
Kind and brand of card should be relatively irrelevant to this test. The objective is to see which disk format (Ext2, Ext3, Fat32, Fat) gets the best speed out of the Pandora.
The raw flash in an SD card is abstracted by the SD card's controller. Thus the controller is always involved, and as a result the way the controller of a given SD card is working, as well as the particular type of NAND used, are of high relevance to the resulting speed. Some cards will work well with some file systems, others won't, and even others will work well in the beginning but eventually cause problems apparently out of nowhere.


The SD card market is very competitive, which means many SD card components will employ technical tricks to increase some qualities such as speed and lifespan. Sometimes some qualities, such as high lifespan, are sacrificed for qualities that are more easily marketed, such as higher (initial) speed (under "average" conditions), or more space per <MONEY>.


To my unrepresentative(!) experience, a speed difference of more than factor 2 between ext2 and ext3 indicates that the card being used likely has quite low quality NAND and/or uses some rather heavy file-system specific tricks, and thus any benchmarks made using that card would likely be quite useless.
Mmm - very doubtful in this case. I'm seeing similar results over two cards where Ext3 is substantially slower than Ext2. The topic of this thread is to find a way to quantify what that difference is. My cards are both branded 'flagship' products. I seriously doubt that "low quality NAND" was used in either - but especially doubtful in the case of the 128GB card since it's on the forward edge of the technology curve.


Patriot LX 32GB SDHC class 10


Lexar Professional 128GB SDXC class 10 133X

Have you been using 'noatime'? If not, you could try whether using it noticeably affects the difference between ext2 and ext3, and then take further actions accordingly (such as looking into "alignment", or more robust SD cards).

I may have to include that as a variable in the testing - after I have a reliable way to test read/write to a partition.


I don't suppose anyone can port the disk benchmark utility out of Ubuntu to a PND? Graphical results are always a cool bonus.
 
My cards are both branded 'flagship' products. I seriously doubt that &quot;low quality NAND&quot; was used in either - but especially doubtful in the case of the 128GB card since it's on the forward edge of the technology curve.


Patriot LX 32GB SDHC class 10


Lexar Professional 128GB SDXC class 10 133X
Being "on the forward edge of the technology curve" is not necessarily a good thing -- as always, newer things are not automatically superior. For instance, from these benchmarks by Tom's Hardware (et seq.), it appears the "Lexar Professional" series card /might/ have difficulties with writing (which could explain the huge difference you have experienced between ext2 and ext3).


Or maybe you just had plain bad luck with the two SD cards in particular, or with the way you have used them. Or maybe I was just lucky with the cards I have used with ext2 and ext3. There is too much black voodoo to know for sure (without spending considerable amounts of resources, that is).

I may have to include that as a variable in the testing - after I have a reliable way to test read/write to a partition.
To my knowledge, there is no such thing as a good standardized SD card benchmark (please tell me in case you find one!) because SD card component manufacturers are able to adjust their products to any benchmarking procedures that become too common.
 
interesting. I just converted my PanData partition where I hold all of my PND files from ext3 to ext2.


ext3 loading a 950K savegame file for Vulture's Eye took 45 seconds.


ext2 loading a 950K savegame file for Vulture's Eye took sub-one-second.


Yes, those are reads. Yes, it is that dramatic. I know this is more seat-of-the-pants than a solid benchmark, but I don't need a finely tuned benchmark suite to tell me that the performance difference is gigantic.


Your results may vary.


Now I need to re-install my OS partition on ext2 instead of ext3. At my last re-install I'd put everything to ext3. Now the OS is the only thing that is still ext3 - everything else is ext2.


If you're running ext3, consider making your own little test to see which operates faster for you.
 
Can you format one of the cards with FAT32 and run the same tests. I am curious to see how it performs.
 
ext3 loading a 950K savegame file for Vulture's Eye took 45 seconds.


ext2 loading a 950K savegame file for Vulture's Eye took sub-one-second.

Can any local Linux Gurus explain this? I would think only writes would ever be affected by journaling. Might something else be causing this (kernel bug)?
 
Back
Top