PXML specification


milkshake

Advanced Member
Joined
May 18, 2009
Messages
3,748
Age
40
Location
Rotherham, UK
As part of my work on my pnd/repo website I have been looknig at the PXML spec and I was wondering if there could/should be any alterations, and was wondering if we could have a descussion about it.


here are a couple which have cropped up during development of my site which could be usefull, although if anyone has any other suggestions maybe they should add them here too.


It would be benificial to have an extra entry in the version element of the xml like so.



Code:
<version major='number' minor='number' release=number build='number' flags='beta,alpha,final' />


so we can seperate final builds from alpha's and beta's.


Others have mentioned about maybe an aditional dilemiter to the version number i.e.


major.minor.release.build.pndver


pnd version for use where software is ported the software version might remain the same but you might fix a bug in the port so the pndver would then incrament instead.


finally it might be beneficial to include an optional element for a software licence? i.e. BSD, GNU GPL etc and source code uri?


anyway what do people think, could anyone from OPT comment/discuss?
 
Last edited by a moderator:
My suggestion, make it <version major='number' minor='number' release='number' build='number' application='string'/>


PNDs become standardized with major.minor.release.build versions referring specifically to the version of the PND, making it easy to upgrade a single PND as it gets updated, and the application version is simply whatever string the application has used for a version number. You shouldn't need to programatically compare the specific applications string, though it could be useful for end users, especially if there are multiple PND files with different versions of an application. Converting "1.34alpha.xyz" into a number wouldn't be easy, and what if someone else has ported "1.34alpha.jim". It's not obvious which is better, or if either is better, nor how those values should be applied if the PND file itself gets updated.


First revision of each PND starts at 1.0.0.1, application="1.34.alpha.whatever...", thus you can programatically update PNDs as they are updated, and leave the decision of which version of the application they actually want to the user.


Case in point, consider Mame. We've got 3 versions of it, and all have their own purposes. It could make sense for a user to have all three. So major, minor, release, and build refer to the specific version of the PND, and application is just the string form of the release of Mame being included for display purposes only.
 
That is a very good option.


what about the flag for alpha,beta,final? this would primerily help sort out pnd's into catories based on wether its a final build or not.


this would help my site also put them into catagories based on this.
 
Last edited by a moderator:
Before developers started labeling their programs as alpha/beta/release, I remember that release happened at version 1.0. 0.x.y was alpha/beta, and when it was finally ready for release, 1.x!


I can't remember how long ago that was though. I guess the current trend is to have "version 1.0alpha" which gets updated until it is fairly complete and becomes "1.0beta", until it's ready for release where it becomes "1.0release". I don't like current trend, but can't argue against it.
 
I belive that is still the case however as lots of apps are made by people that dont always make programs for a living I would say it would easier for them to flag their version as alpha, beta or final, its just much simpler and cleaner maybe?
 
Last edited by a moderator:
I diddn't want to say any more about the version tag, but still.. why would you reserve a 4-part version number to the PND, which is nothing more than a wrapper? it would be reset to 1.0.0.1 as soon as the application updates (because it's the first PND of that app - if it wouldn't reset, it would be the app version, in a way, again) and for homebrew, the PND version and the app version would be the same anyway, so that can be disregarded. soo.. I see no sense of this because just for the one or two bugfixes you wouldn't need a 4-part version number.. not even a 2 part one..
 
I think you greatly underestimate what goes into a PND. I've got Java based on open-jdk 6b18-1.8-r10.6.6. Suppose I call this PND version 1.0.0 Bug in my installer, I call the new version 1.0.1. If I add plugin support, it's a new feature, so that becomes version 1.1.0. Eventually I upgrade the whole thing to 6b18-1.8-r10.7.6: that becomes PND version 2.0.0.


There are a lot of other examples if you look of small changes to the PND (things within the PXML, icon changes, default settings, etc...) that warrant a new version that have nothing to do with the application being included.
 
I'd like to point out that this conversation is also going on on the other board.


skeezix suggested having a "release-tag" option that can be "alpha", "beta", or "release" (and if not specified, "release" is assumed), as well as a "release-scope" option, allowing for "public" or "private" (where "public" is the default).


milkshake also suggested having a "license" element, and possibly a "source" element, containing a URI to the source. sebt3 voiced approval.


I suggest we try to keep this conversation in one place.
 
Last edited by a moderator:
I linked to this conversation from GP32x the only reason i posted there also is because not everyone uses both boards, sorry if I did wrong.
 
Last edited by a moderator:
problem with that is... when im at work i cant access gp32x its blocked but i can access here so during the day I roam these streets and at night I roam both :) , I know most active devs still go gp32x though.
 
problem with that is... when im at work i cant access gp32x its blocked but i can access here so during the day I roam these streets and at night I roam both :) , I know most active devs still go gp32x though.
Going to gp32x has the additional plus of me not being there to go on your nerves :p
 
This is no proposal for the PXML spec but for the repository one. I think it fits best in this thread.


-----------------------------


OK, I thought about it a bit more and I think there's one additional tag we need. That tag could be generated on the server and doesn't have to be in the PXML.


I have to first explain the setup so this makes sense, please bear with me.


Let's say in the future, we have at least 2 repositories, let's call them RepoA and RepoB. Now, what happens when he client stumbles across a program (same ID, same version) that's on both repositories?


The easy way out, and the way that wouldn't need any change to the repo definition, is to just priorize sources in the client, for example the higher up a repo is in the settings, the higher priority it has. This way, when stumbling over an app with the same ID and version on multiple repos, you just take the title, description, etc from the higher priorized one and eventually download the package from the same.


But now, the efficient way to handle it is.. when you have the same program on multiple servers, which you will have since most people would probably upload their stuff to all repos and the repos would synchronize with each other, then you would balance the downloading between servers. Download app 1 from RepoA, App 2 from RepoB, app 3 again from RepoA etc. One could even go the way of some download managers and get the first part of an app from the first repository and the second part of it from another repository.


I'm sure you all agree that this way would be preferable, after all, it would serve as a client side load balancing, which is easier on the servers and will probably result in faster download speeds for the end user. But this approach has two downfalls:


1. when using the download manager approach, you have to be absolutely sure that the file on all repositories is the same on a binary level. Now, if whoever made the app uploaded it on RepoA and then repackaged it without changing it for some reason and then uploaded it to RepoB, the app, while being the same one, wouldn't be the same on a binary level, so the dl-manager-approach would fail. Now, I do not know if the server needs to implement something special for the client to be able to download only the second part of a file, so this approach would maybe not work anyway, but that brings us to..


2. what happens when the two apps, both with the same ID and the same version tag, weren't the same app? Sure, the chances of two devs choosing the same ID and version and uploading it on different repos are rather slim, but it is possible. Also, never underestimate the douchebagginess of humans. Say GuyA uploads his prog to RepoA. GuyB hates the guts of GuyA and therefore makes a small malicious app (or a really buggy one) and intentionally uses the ID and the version tag of the one GuyA made and uploads it to RepoB. This way, GuyB could be sure that half of the time, people would get his malicious version instead of the real one but blame GuyA for it.


So, long story short, what could be done about those two problems? The answer brings us back to the start of this post and the tag I want to propose. The tag would be a hash (CRC, MD5, doesn't really matter). If one had a hash tag, the client can act like this: If a program with the same ID and version is on multiple repositories, get the hash tag of the one in the highest-priorized repository. Then, balance the downloading between that highest-priority repo and all other repos which have the same program (again, id and version) with the same hash tag, but ignore all repositories that, while having the same ID and version, do not have the same hash tag.


This would solve both problems mentioned above.


1: The hash ensures that all the repos you download from have the files in exactly the same (on a binary level) version, so repackaging for whatever reason would no longer be a concern for you.


2: The hash ensures that you will get the right app and not some other one that's just an impostor. Sure, if the malicious version was on your highest-priorized repo, you would still get it, but one can assume that you would priorize the largest repo (or the offizial one, if there will be one) at the top and that, in turn, the original dev of the app you want would upload it on the largest/official repo, so the impostor would have to resort to another, smaller repo.


TL/DR:


I propose a hash tag that is generated repository side at upload time.


Holy canolli, this post grew huge. Why the fuck do I always use so many words to explain something so simple, in the process probably making it harder to understand than if I just made it short?
 
I kinda like this idea. Not only would it allow for client-side load balancing (as you put it), it would also allow for PNDs to be validated after download. I think this could actually be the more useful side-effect. My work on the client already supports multiple prioritized sources, and I don't have the knowledge to implement any interesting download tricks (other than random selection of download source), but I could certainly do hash verification without trouble.


Still, in order to do those interesting download tricks, all repos involved would have to use the same hash function. If we have to pick one, I'm a stickler for security: SHA-256 or SHA-512 would be nicer than MD5.
 
Still, in order to do those interesting download tricks, all repos involved would have to use the same hash function. If we have to pick one, I'm a stickler for security: SHA-256 or SHA-512 would be nicer than MD5.
I'm also a fool for security, but I don't think you'd really need a cryptography-grade hash algorithm for that - generating a file with a valid PXML that has the same hash as another one is pretty impossible even with plain old CRC
 
well my repo already stores MD5 hash's of the files that are uploaded, what would need to be implaments from an upload point of view would be to validate the hash stored in the PXML with the actual file hash of the binary that the PXML came from to make sure it matches.


but who would decide which hash is the valid non meliciouse version without manually checking them first?
 
Last edited by a moderator:
I think you misunderstood Klaue's suggestion. In this case, he's suggesting no change to the PXML, only a change to the repository specification. The idea is that, for each application listed in the JSON repo, you would add an extra field, "md5", with the value of the MD5 hash you calculated, preferably in hexadecimal. Then, my client downloads the JSON file, including all the MD5 hashes; when the client downloads a PND, it calculates the hash of the PND and compares it to the hash listed in the JSON. That way the client knows that the PND was downloaded properly without errors.
 
ah ok, I could easilty do this now (I have all the md5 file hash's stored for duplication comparison pursposes).


We should add this to the repo spec, and since no one is using the spec except us (atm) maybe we should be the ones to define it?


obviously get some sort of nod from OPT first though.
 
Last edited by a moderator:
I'd say we're the current authorities on the subject, yes :) And I've been hesitant to make changes so far just because the spec hasn't given us problems yet and I'm afraid of screwing things up. But this change is so minor and has such obvious benefits that I'm okay with doing it.


Of course, more people's approval would be nice. Where's skeezix when you need him?
 
Back
Top