Pandora Directory Structure Question

[alucart1984]

Hey, I've been arguing for the current implementation here, I just entertained the idea. His idea was to have all data, both pnd and the appdata under one subdirectory like so:

pandora/
pandora/apps/
pandora/apps/Quake/
pandora/apps/Quake/quake.pnd
pandora/apps/Quake/appdata/
pandora/apps/Quake/appdata/pak1.pak
pandora/apps/Supertux/
pandora/apps/Supertux/supertux.pnd
pandora/apps/Supertux/appdata/
pandora/apps/Supertux/appdata/customlevel.level   (or something)

So not just the pnd, but application-specific data directories too.

EDIT: emphasis on quote

I hate to be the one that says this but the PSP is already doing this and doing it well. Granted you can't really organize your games the way you want on the PSP but things work just the way you mention. Example

Games
/PSP/Game/Quake/
/PSP/Game/Quake/Quake.eboot
/PSP/Game/Quake/Data/Quake.pak

Music
/PSP/Music/Artist

Photos
/PSP/Photos/Ablum/photo.jpg

Works very well but the lack of organizing is the only issue.

 

[laurencevde]

I expect there to be a variable that defines where the appdata goes to. Set that to /pandora/apps and you'll get pretty close to what you want. All subdirectories of /pandora/apps are apparently searched through to find .PNDs, so you can then put your quake.pnd into /pandora/apps/quake/, along with its app-data. One problem though: depending on the amount of files/directories your apps create, the filesystem on your card, and the speed of your card, finding all .PNDs might become slow if it needs to look through all data.

Oh, and that desktop menu spec specifies both a standard for the menu-entries(.desktop-files), and a standard for configuration/layout of the menu(an xml-file).
The menu-entries just specify a set of (predefined) categories, menu-config specifies where those should be put into the menu. You can change the entire layout just by changing the menu-config.

 

[EvilDragon]

I expect there to be a variable that defines where the appdata goes to. Set that to /pandora/apps and you'll get pretty close to what you want. All subdirectories of /pandora/apps are apparently searched through to find .PNDs, so you can then put your quake.pnd into /pandora/apps/quake/, along with its app-data. One problem though: depending on the amount of files/directories your apps create, the filesystem on your card, and the speed of your card, finding all .PNDs might become slow if it needs to look through all data.

Yep, you can. It's a config file where you define all paths.

Crawling shouldn't take too long - I tried crawling through 50.000 files and it took 3 minutes.
So as long as you don't have too many files there... but of course, it does slow down discovery a bit.

 

[MDave]

There are good and bad points to both types of implementations that are argued for.

DaveC wants what has worked for years on Windows in terms of ease of use and knowing where everything is related to the app, one folder named after the app with all its contents in there.

EvilDragon wants something that works good if you have lots of folders and would like to remove config and other data without touching the actual apps.

I see EvilDragon's specific scenario not something that common. If you do a lot of formatting and installing apps over and over then yeah, I see the plus side to this.

But that isn't going to happen on the Pandora for your average Joe that just wants to play games and not mess with Linux distros on it, are they?

In that case, I'm in favour of what DaveC is talking about. It's worked great for PSP homebrew users, and more recently Wii homebrew users too.

 

[EvilDragon]

There are good and bad points to both types of implementations that are argued for.
DaveC wants what has worked for years on Windows in terms of ease of use and knowing where everything is related to the app, one folder named after the app with all its contents in there.
EvilDragon wants something that works good if you have lots of folders and would like to remove config and other data without touching the actual apps.
I see EvilDragon's specific scenario not something that common. If you do a lot of formatting and installing apps over and over then yeah, I see the plus side to this.
But that isn't going to happen on the Pandora for your average Joe that just wants to play games and not mess with Linux distros on it, are they?
In that case, I'm in favour of what DaveC is talking about. It's worked great for PSP homebrew users, and more recently Wii homebrew users too.

Oh, it doesn't work for Windows. Most Windows apps (and most Linux apps) do write their config settings either into the Home-Directory, the Local Settings or the Registry.
Most average Joe users don't know this, but it happens. If you delete the directory, you have the config files left.

Also... ermm..
The PND System is even easier for the average Joe user.

Compare:
To install a game on the Wii or PSP, you have to download it, extract it into specific folders onto your SD Card.
To install a game on the Pandora, you have to download it and place it into a specific folder onto your SD Card. It's the same, except for missing that extra "extract it"-Step.

To remove a game on the Wii or PSP, delete the whole folder. If you don't want to lose your settings / saves, etc. , you have to manually fiddle in the folder to find out your settings / saves / etc. files.
To remove a game on the PND, delete the .PND. You won't lose your settings. You want to delete the settings as well? Okay, simply delete ONE more folder.

To install a game like Quake on the Wii or PSP, you have to download it, extract it into specific folders onto your SD Card. Then you grab the Quake data files and place them into a directory as well.
To install a game like Quake on the Pandora, you download it, put the PND file into a specific folder. Then you grab the Quake data file and place it in the Quake-Subdirectory.

As PNDs can be compressed, they need much less space than uncompressed files. Loading is also faster for games that have many small files.
Putting games on the card is also faster. I remember a game on the GP2X that took approx. 2 hours to put on my SD Card as it had so many small files. When it was packaged up in a filesystem, it needed 5 minutes.

So... ermm... it's faster, smaller, is as easy to handle as the normal "put stuff into a directory".
But it's also more convenient.

 

[mindlord]

Keeping apps separate from data is the Linux "way" of doing things. Config files go in /etc or /home/user/.appname or (recently) /home/user/.config/appname, applications go in /bin, and data goes in /usr/share. This is not a foreign way of doing things at all, and I totally see the benefit for using this type of model for the Pandora.

It's not a bad way of doing things, it's not altogether confusing, and if allows you to quickly backup your data, and upgrade apps witout them undoing your custom configuration. It wouldn't be hard to write a "tidy" program that compares the PND's on the SD to the appdata folders and helps you figure out which ones are in use and which ones are cruft. LibPND might be able to help you when deleting PND's from the SD card (using the Pandora) by asking if you want to delete the appdata folder as well.

The current way PNDs work I think is pretty nice. Keeping appdata separate from the PND has tons of benefits and very few detriments, fast PND discovery being the most imporant, IMO.

 

[MDave]

There are good and bad points to both types of implementations that are argued for.
DaveC wants what has worked for years on Windows in terms of ease of use and knowing where everything is related to the app, one folder named after the app with all its contents in there.
EvilDragon wants something that works good if you have lots of folders and would like to remove config and other data without touching the actual apps.
I see EvilDragon's specific scenario not something that common. If you do a lot of formatting and installing apps over and over then yeah, I see the plus side to this.
But that isn't going to happen on the Pandora for your average Joe that just wants to play games and not mess with Linux distros on it, are they?
In that case, I'm in favour of what DaveC is talking about. It's worked great for PSP homebrew users, and more recently Wii homebrew users too.

Oh, it doesn't work for Windows. Most Windows apps (and most Linux apps) do write their config settings either into the Home-Directory, the Local Settings or the Registry.
Most average Joe users don't know this, but it happens. If you delete the directory, you have the config files left.

Also... ermm..
The PND System is even easier for the average Joe user.

Compare:
To install a game on the Wii or PSP, you have to download it, extract it into specific folders onto your SD Card.
To install a game on the Pandora, you have to download it and place it into a specific folder onto your SD Card. It's the same, except for missing that extra "extract it"-Step.

To remove a game on the Wii or PSP, delete the whole folder. If you don't want to lose your settings / saves, etc. , you have to manually fiddle in the folder to find out your settings / saves / etc. files.
To remove a game on the PND, delete the .PND. You won't lose your settings. You want to delete the settings as well? Okay, simply delete ONE more folder.

To install a game like Quake on the Wii or PSP, you have to download it, extract it into specific folders onto your SD Card. Then you grab the Quake data files and place them into a directory as well.
To install a game like Quake on the Pandora, you download it, put the PND file into a specific folder. Then you grab the Quake data file and place it in the Quake-Subdirectory.

As PNDs can be compressed, they need much less space than uncompressed files. Loading is also faster for games that have many small files.
Putting games on the card is also faster. I remember a game on the GP2X that took approx. 2 hours to put on my SD Card as it had so many small files. When it was packaged up in a filesystem, it needed 5 minutes.

So... ermm... it's faster, smaller, is as easy to handle as the normal "put stuff into a directory".
But it's also more convenient.

You raise some good points there although with the Windows example I was talking about how you can have an app anywhere you want in the file system, and games before the vista and 7 used to keep files in their app folder not in a home like directory. Still the same case for emulators too. Everything related to that app is all kept in the same place.

If you don't want to lose your settings / saves, etc. , you have to manually fiddle in the folder to find out your settings / saves / etc. files.

I don't think that is any different to having it in a separate folder just in a different directory all together. In fact this is more work for those that don't know where it is!

The PND files themselves I have no problem with, I never brought it up or anyone else I don't think quite similar to PSP EBOOT files where the main executable, icons and other info is stored in an archived format similar to PND's.

 

[DaveC]

A subdirectory for a PND?
What sense would that make?
You'd like to have a single PND in a single folder?

If the user wants to sort his PNDs into subdirectories (i.e. name them "crap games" or "cool games") to sort and move his PNDs there, he could still do so.
libpnd crawls through those directories, it would discover them as well.

So you can sort them if you like or put them all into one directory.
Pretty clean thing.

Well IF a pnd doesn't create anything else ever you don't HAVE to put it in a subdir you can have it in the GAMES dir for example as just a pnd. If you don't like that you could put in a subdir just to be consistant it is up to you. The thing is it is up to the user not up to some ridgid predetermined thing that you have to follow whether you like it or not.

I would bet that 95% of the time though there will be some kind of external data such as a save game file, config file, BIOS ROM etc. To not do something just because a few times it might not be needed is silly.

The other argument about how you just want to delete the pnd to save space and leave the appdata alone is also a rare case that is basically just to try to find an excuse for it. For one thing MANY times the data in appdata will be BIGGER than the pnd so just deleting the pnd like you suggest would not do much. QUAKE, Doom, Hexen etc etc where the data such as the PAK, WAD etc takes up more space than the pnd kind of shoots that down. You still have to hunt through multiple directories to do what you say anyway. If the pnd and it's appdata was in the same subdir all you would do is select the folder, cut, then paste on your computers drive. When you want to move back just do the same in reverse, it is actually easier than managing two different dirs in two different spots.

 

[WizardStan]

You raise some good points there although with the Windows example I was talking about how you can have an app anywhere you want in the file system, and games before the vista and 7 used to keep files in their app folder not in a home like directory. Still the same case for emulators too. Everything related to that app is all kept in the same place.

This might be true for apps that you just unpack and run, but not necessarily so. There was never any standard way, nothing which made applications write anything in a specific directory. I've seen applications that write their config files into the Windows directory; those are a right mess to clean up if you don't know it happened.
And then you get into games and applications that need to be installed. Then you can end up with files all over the place, in the install directory, in your documents and settings, in the registry, in the Windows directory.
Even Linux apps, which have a pseudo-standard, can still take some time to find everything. Did it create the config files in its own directory? Are they in ~/.appdata, or perhaps ~/.config/appdata? Or if you installed as root, maybe it's sitting in /etc and /usr/share?
The format proposed might involve two directories, but that's all it involves, just those two. PNDs in one, config and additional application data in the other. I can't imagine a cleaner way of doing it and making everyone happy.

 

[Caine]

Example motivation for operations on PND or appdata only:

• Share PNDs with a friend but not config and saves
• Remove PNDs to make space for new ones, but don't discard saves
• Fresh reinstall of applications (delete config only)

Two approaches:

1. appdata dir in application folder (windows way of structuring, sorta)
   Proposal by DaveC
2. application folder in appdata directory (linux way of structuring)
   Current implementation

Now consider the complexity of batch operations such as delete/move/copy on the two approaches.

batch operations on 1:

• Batch full delete: select app directories; delete
• Batch delete PND only: select PND of app 1; delete; select PND of app 2; delete; etc...
• Batch delete appdata only: select appdir of app 1; delete; select appdir of app 2; delete; etc...

batch operations on 2:

• Batch full delete: select PNDs; delete; select appdirs; delete
• Batch delete PND only: select PNDs; delete
• Batch delete appdata only: select appdata dirs; delete

Advantage/Disadvantage of 1:

• full removal/transfer takes (a little bit) less effort
• automount of PNDs is slower (more directories to scan). I think this is an important issue.

Advantages of 2:

• operations on config or PND only are a LOT easier
• in line with how apps are structured in the entire OS (it's the linux way)
• strong separation of read-only image (PND) and writable data (appdata)
  • easy to mount to optimized filesystems
  • easy to support multiple user profiles
  • more secure against misbehaving apps (access rights & quotas are easier)

For one thing MANY times the data in appdata will be BIGGER than the pnd so just deleting the pnd like you suggest would not do much.

This is incorrect. PNDs are ISO images. For most newly written apps ALL data is contained within these images. The appdata will consist mainly of savegames and configs.

 

[Dead1nside]

I see what you're both saying but I think there'd be an issue where if all the data was in one directory (perhaps with subdirectories) then a user might accidentally delete their save data.

Also, with a game like Quake where the WAD(?) file is external and needed to run the emulator, can you put that in /appdata too and just forget about it for all intents and purposes?

I don't see what's too hard about having one directory for the applications and one for the actual data. Isn't this currently how Windows Vista is doing things now, with games? It's certainly the Linux philosophy.

 

[skeezix]

Good explanation. One more query - what is a .pnd? A tar.gz of binary, lib and pnd.xml file?

It's an ISO that can also be squashfs'ed.

So games like SuperTux only need 7MB instead of 16MB on the card (as they don't need to be unpacked).
Or AlephOne with all datafiles is 97MB big. Unpacked (like you run stuff on the GP2X or WIZ), it does take about 200MB.

What? Since when is squashfs supported?
OK, I see that it's supported in the Git now, but you can't use *both* ISO and squashfs; you have to choose one or the other.

Also, skeezix has told me that the whole PND system should be revised after the release of the Pandora (because of the problems PXML has etc); we'll see what happens.

The filesystem in the pnd can be many things; iso is one, cramfs is others; its not all pnds are one type,.. its just a given pnd may be one type, and another pnd may be another type, up to developer preferences.

In theory, we can support dozens of filesystem types, but in practice we're only supporting a few right now. (its purely a 'magic' file thing, in pnd_run.sh, really.)

Even windows can make cramfs (we do it

jeff

As to post-release.. I'm not a pnd-nazi; libpnd we've made and LGPL'd, peopel can go nuts I'd liek sensible reviewed alteratins and additions .. no need to go hog wild and break it all day-2, but whatever. Extensions, changes, even long term changes.. all good.

 

[skeezix]

also..
You can just stick pnd files in any old directory you want; you don't have to use the menu or desktop auto-icon business at all .. you can stick them into /games on your SD, and then just use the file browser and double-click on the pnd you want in there to run it.

No one is forcing you into anything

What we are doing is making automagic instant easy to use behaviour an option ... seems desirable to me, but you don't have to use it

jeff

 

[skeezix]

Also, we're ignoring one of the big benefits.. pnd is drag and drop and play. PEople are really asking about.. wanting it more like windows? _Windows_? With dlls in one place, apps in another, your user data in a third place, your registry entries, and home-doir hidden stuff? And oyu have to run an installer and an uninstaller?

We have that beat Stick your pnd files in one place (or any place you want really), and your data all goes to one folder, with a subfolder per app. ITs easy to back up, easy to delete, and all automatic.

The only case I can see here, and its an interewsting idea, is to make the appdata-dir be the same dir as the pnd-file, as an option; that'd not be too hard a change, but lets leave that as a post-release feature request. Right now, all pnd user data will go to /pandora/appdata/app-name; in the future, if you want it so /games/foo.pnd writes its appdata to /games/foo/, we or someone can code it in; its a change to pnd_run.sh really, and maybe a conf file to toggle it.

As it is, its a very good system; whats going on here is 'bike shed' -- arguign over little details

jeff

 

[skeezix]

(sorry for my rush, too many posts by me lately from my phone

jeff

 

[MDave]

Also, we're ignoring one of the big benefits.. pnd is drag and drop and play. PEople are really asking about.. wanting it more like windows? _Windows_? With dlls in one place, apps in another, your user data in a third place, your registry entries, and home-doir hidden stuff? And oyu have to run an installer and an uninstaller?
jeff

Now now, don't go taking what people have said out of context! Nobody was asking for all that registry and installer stuff when comparisons were made with Windows. You and EvilDragon are picking on the commerical apps that do all those sorts of things to Windows, when I'm talking about the homebrew style games and emulators on that platform that use just a single folder for the executable and related application data. You can put the folder where you want them, and you know where everything is for that app. Linux can do the same thing, no?

Anyway, I'm not attacking the current setup, just defending what other options there are that do have their plus points to them

 

[#1 GameMaster]

Wait a second? WHAT are you all talking about?

The good thing about .PNDs is, that you put them all into one subdirectory and they keep their config files, etc. elsewhere.
That way, you don't need to install anything.

Want to delete the config (because you messed it up, maybe)? Then simply delete the folder of the app in the appdata directory.
Want to delete the app for a while but keep the config? Then delete the PND and keep the folder in the appdata directory.

It's that easy.

You want to have the datafiles and configs in the SAME directory as the PNDs?
Heck, if I have 50 PNDs in my menu directory, I surely don't want to have 50 config files in this directory... THAT would be messy and cluttered.
That would like putting ALL games and configs into ONE directory (without subdirectories)... is anyone doing that on the WIZ or GP2X??

This is no way anything like a DLL hell, as the app will have ALL THE DATA NEEDED in it's PND! (except for datafiles like Quake, etc. which we can't offer).

So if you delete the appfolder, the program won't stop working, it would just be as if you had just installed it fresh on the SD card.

You can simply download an app from the appstore, save it to the PND-folder - and run it. It works, I tried it. It appears instantly. And it does save its config files in the appdata-directory.

Here's how it works:
i.e. if you download SuperTux, you download the SuperTux.pnd and place it in the menu or the desktop folder. It will automatically appear in the menu or on the desktop.
You run it and change the configuration (button config, etc.).
It then creates a folder named "SuperTux" within the appdata directory. That's where your savedata is in.

Now, somebody created new maps you want to try out. Simply place them within the SuperTux-Directory in the appdata-Dir and run it. It works.
You can even create your own maps within that directory. Made a mistake and the maps don't work? Well, simply delete them and the original ones will be back, as they are stored within the .PND.

Easy.

So, of COURSE we need multiple directories. It's not like on the GP2X or WIZ. You don't EXTRACT PNDs. You simply COPY them onto the card and run them (you could compare it to placing ZIP-files on the WIZ or GP2X without extracting them).
But as config files (and datafiles for games like Quake) need to be put somewhere, they are being put into a subdirectory in the appdata folder. With a name so you know where you can find it.
The user doesn't even have to create the subdirectories - they will be created when you run a game. So as long as you don't need any special game data (which should be the case for 99% of all games), you don't have to do anything except for copying that PND file onto your SD Card.
And for emulators... well, it's up to the emulator dev where he allows your rom path. Most of them allow any path on the SD, so you don't need to use the appdata dir for that. But the emulator will save its config there.

The config or datafiles should NOT be put into the PND. If anything goes wrong, you'd need to redownload the PND. Here all you need to do is simply remove the folder in the appdata directory and restart the game. It couldn't be more simple.


So simply said, the directory structure is the following:

/pandora/menu/SuperTux.pnd
/pandora/menu/Quake.pnd
/pandora/desktop/SuperMarioWar.pnd

Then there will be automatic data dirs:
/pandora/appdata/SuperTux
/pandora/appdata/Quake
/pandora/appdata/SuperMarioWar

For SuperTux and SuperMarioWar (which are complete games), you don't need to worry about anything. They simply save their config file there. Delete the directory to restore the default settings. That's it.
For Quake, you place the datafiles into /pandora/appdata/Quake/
That's it.

It's not so hard, is it?

If anyone has any better ideas how to handle it, let me know. We thought A LOT about it.

I was getting ready to ask if that's how it works, because I actually like the idea of it working like that , but the way some of the people were talking about it it was starting to sound like it was going to be some kind of nightmare. :blink:

 

[EvilDragon]

Also, we're ignoring one of the big benefits.. pnd is drag and drop and play. PEople are really asking about.. wanting it more like windows? _Windows_? With dlls in one place, apps in another, your user data in a third place, your registry entries, and home-doir hidden stuff? And oyu have to run an installer and an uninstaller?
jeff

Now now, don't go taking what people have said out of context! Nobody was asking for all that registry and installer stuff when comparisons were made with Windows. You and EvilDragon are picking on the commerical apps that do all those sorts of things to Windows, when I'm talking about the homebrew style games and emulators on that platform that use just a single folder for the executable and related application data. You can put the folder where you want them, and you know where everything is for that app. Linux can do the same thing, no?

Anyway, I'm not attacking the current setup, just defending what other options there are that do have their plus points to them

Well, you HAVE the option. Of course you can put games onto your card, extract them and manually run them via a file manager. Same as Windows. But it means more manual work. Downloading, extracting, sorting.

I thought it was pretty cool going to the appstore, clicking on install and poooof - the game is installed and setup on your card.
Nothing more to do

 

[MDave]

I thought it was pretty cool going to the appstore, clicking on install and poooof - the game is installed and setup on your card.
Nothing more to do

Yeah that does sound good

But! There is going to be emulators that require the user to place bios files somewhere. Maybe the emulator is kind enough to let you search for it through a sort of file explorer, but all the ones I've used require them to be in a folder called bios in the apps folder. All the Quake, Doom and other commercial game ports are also going to require the user to put their legally owned game data into the app folders too, so it's not going to be all easy for those that don't know about the ins and outs of how the system works.

 

[dflemstr]

I thought it was pretty cool going to the appstore, clicking on install and poooof - the game is installed and setup on your card.
Nothing more to do

Yeah that does sound good

But! There is going to be emulators that require the user to place bios files somewhere. Maybe the emulator is kind enough to let you search for it through a sort of file explorer, but all the ones I've used require them to be in a folder called bios in the apps folder. All the Quake, Doom and other commercial game ports are also going to require the user to put their legally owned game data into the app folders too, so it's not going to be all easy for those that don't know about the ins and outs of how the system works.

I'm sure that emulators could have a "Load BIOS file" menu entry that would let you select a file. The emulator would then copy that file to wherever it wants it to go.