Pandora Pandora Store/online Package Installer?


eyecreate

Member
Joined
Feb 3, 2009
Messages
253
Website
git.eyecreate.org
I've been redesigning a website that will (hopefully) turn into a store/portal for software installation/buying on the Pandora. I've made some progress recently and thought people might want to see. Below is the URL for the test app. It currently queries my database which now has a list if apps from the app-install-data package of debian. The interface is a simple drag-drop approach and and search-as-you-type too. I hope you think this is a good idea, but also, please don't try to mess it up yet :D , I'm still finding bugs.(also, it may be down if I decide to do maintenance.) Hope to get something up for "launch".

http://pandorastoretest.eyecreate.org/databasetest.zul
 
Hey, I've been working on something very similar (if you have been reading my PXML spam posts recently you'd know ;) ) although it won't support money transactions/buying packages (which it seems that yours will).
However, it >supports< (99% finished) automatic PND data parsing (aka loading of PXMLs and filling in fields automatically), support for the PND screenshot spec, very easy-to-use package searching, an exposed XML API (this is not quite done yet) if you want to make front-ends similar to apt-get or something to install applications and so much more! :p
You can try it by following these instructions (running a local server): http://wiki.github.com/dflemstr/pndmanager...nd-installation
My source repository also contains test-PND-packages if you don't have any yourself.
(@ other forum users: don't try it out yet, it isn't ready for public consumption yet; I'll announce it in a few days)

EDIT: Oh, and I guess for those who are curious but impatient, here are some screenies.

I've gotten really far with my application, and am planning to announce it once I've fixed a few issues. Maybe we could cooperate on something here?
 
Click link, shows empty list and a search option. *fear*

Are you looking into ipkg or pnd or PXML-apps, or just debian apt database?

dflemstr is also working on something, as well as others; people should get a forum together :)

jeff
 
@dflemstr: I think it's important we do some collaboration, as it will put more ideas together and thing might go faster. Not sure where to start, but I guess I could start by trying out the code you mentioned to me. Let me know if you have an idea.
EDIT:after looking at the pictures first, it seems our ideas have alot in common when it comes to presentation, the main difference being that yours mostly just dealt with PXML packages, where some of my ideas for what I expected was a mix between what you have with the "product page" look and existing Appnr. Also, I planned to have a "client" that would allow repo packages and PXML to be downloaded to the device(not on the website, though I was going to leave the option on PXML packages), which it seems is not the case on your website from the pics.

@skeezix:I plan or doing the following: Allow existing packages to be installed that are in the repos while integrating the PXML and other package files for the pandora specifically into the mix so they all seem "equal". The client software will then install each as is necessary. After seeing dflemstr is also doing some work, I'd like to see where we can work together. Also, not sure where your fear of an empty list/search box comes from, but I will say I only put up some bare items needed to interact, nothing decorative has really been done much.
 
, which it seems is not the case on your website from the pics.
What do you mean, "existing packages"? I don't know of any existing packages... And of course I only deal with PND packages; these will probably be the only packages officially supported by the Pandora team so why would I cover something else?
EDIT: Oh, you're thinking about Debian packages... Those won't be usable on Ångström, however, and wouldn't be installable anyways since there isn't any persistent storage on the Pandora. So yeah, no use in making the system compatible with that.

Anyways, I should maybe clarify what I have done and plan to do with my application.

I have:
  • Created a VERY fast web application. It's so fast that you'd be amazed. I filled the database with 4096 packages, and got 2ms response times on searches (granted, I used the H2 RAM-based SQL database and ran the server on localhost but still ;) ). I use Scala + Lift, with extreme compilation optimizations, on Tomcat 6 or Jetty, so the result is that you basically cannot create anything faster than this short of by writing a web server in C :p
  • Integrated AJAX, Comet, jQuery etc to get an application that almost feels like and responds like a local application. My goal is that it should be VERY user friendly; no learning curves at all.
  • Made the application parse PND files and populate the database with meta data on-the-fly when uploading files. No tedious forms on the web page.
  • Made a REST-based API that enables you to make front-ends for the application (like apt-get or anything). (So yes, I have thought about this and this was one of my primary goals) This also makes it possible to do auto-updates. Note that it's impossible to make PNDs compatible with, say, APT, or most other package management systems, without extreme hacks, so we will need a completely separate/independent system; don't think that anything else would be possible.
  • Integrated OpenID with the application, so that I get an user management system, thus giving users the right to upload packages directly themselves and not having to submit them to some maintainer.
  • Made the application completely platform-independent and environment-independent. Just look at my installation instruction page: you can download and run the application locally with two lines on the command line. It's also compatible with all SQL databases known to man (aka known to the JDBC system :p), runs without requiring access to a file system at all, and does extreme multi threading (thread pool of up to 512 threads, with an event system working side-by-side with it to reduce the number of needed threads to a minimum)

I will:
(updates are Italic)
  • Done: Add user administration, so that it's possible to moderate packages
  • Done: Add Spam-protection systems
  • Done: Extend the OpenID system so that users don't have to fill in their user preferences; just use the OpenID global preferences!
  • Done, just need translators: Translate the whole page into all languages known to man (PNDs have translation meta data so it'd be a shame if we wouldn't use it, and German meta data looks out of place on an English web page)
  • Skip; too CPU expensive: Extend the REST API to support things like statistics in the client
  • Done: Extend the web page statistics system to create diagrams etc of the package status
  • Improve the UI; I use too many CSS hacks in it :p
 
Last edited by a moderator:
Well, I guess what I had read so far on the forums confused me, as i read somewhere that Ångström did have packages and that would be one way to install "regular" apps in most distros into the Pandora. If that's the case, then, and also after reading your post on what you've done, it seems that you have most things already covered. I'll just save my code for another project I was going to end up doing some other time so I can reuse it. Otherwise, I'd be happy to either help add the idea of making it easy for those wanting to sell their games to do so, or I can create a Packagekit like client to your website like I would have after I made the website.

If neither of those things seem something you need/want and you don't know of something you want, I can just go back to finishing my XML to Python/pygame compiler. :p

EDIT:I was curious about the language/framework you said you used, and I'll say it looks pretty neat(I may try it out myself at some point). Reminds me of Python, in a good way. :D
 
. Reminds me of Python, in a good way. :D
Oh, Scala? It's become my hallmark on this forum in some ways that I'm the guy always talking about Scala. Basically, it's a language that has all the features you could dream of while still being a statically typed. If you combine Python, Ruby, Erlang, lisp, OCaml and C++ you might cover most of them, but never all of them. I mean, come on, is there any other language that has pattern matching on XML literals? Syntax-checked custom DSL languages that are type-safe? Multiple inheritance that is foolproof through mixins? A language that supports infinitely large thread pools with its Actor system? A language that enables you to override singleton object method fields (you won't need it, ever, but it's a cool feature)? A language that supports closures, continuations, monads and pure function values? Didn't think so.

So I basically use Scala in combination with the Lift web framework and am able to make a web application in under 1000 lines of code that would take a PHP programmer 10000 lines of code to come close to, without doubt. It's very nice. However, it has more of a learning curve than Python (kind of), but it's worth it in the end I think.

If you're interested, here's how you can get started with Scala:
Install the very basic Scala system:
→ Ubuntu: "sudo aptitude install scala"
→ Fedora: "su -c 'yum install scala'"
→ other systems: the 12-step tutorial below shows you how to install Scala; in Windows just download the Scala zip and double-click the bin/scala.bat script to start the interpreter

Scala 12-step tutorial: http://www.artima.com/scalazine/articles/steps.html
Scala learning resources: http://www.scala-lang.org/node/1305
Scala book: http://www.artima.com/shop/programming_in_scala

Lift tutorial: http://liftweb.net/docs/StartingWithLift.pdf
Note that Lift is a VERY new framework, and it can be difficult to find learning material for it.
 
Last edited by a moderator:
eyecreate -- strikes me before you get deep into writing an app, you should ask some Q's or look into existing information that drastiocly alters your perception ;)

Angstrom distro uses (like numerous others) ipkg as its native package manager; most distros assume a hard drive and such so the packages slide right into the installation no problem; we don't really have that luxury, nor infinite space, etc. (ie: If users go plugging in dozens of packages, where do you think they will go? flash is only so big, and you dont' want user data files writing and re-writing over flash, since all flash media has limited number of writes before it dies.) So we created the 'pxml' and 'pnd' systems, which is our own packaging system that allows apps to be found and executable without even being installed, right off SD cards (or flash if the user wishes to copy them into flash.)

We also have toyed with the idea of making it so writes to OS directories in flash will get redirected to SD (say), so that a normal package system would work fine (it'd look normal to the OS, but actually be on SD) (theres more to it than this, I'm summarizing.) But we'll see how much time we have to test and get everythign ready .. once real units start coming together, people will want them *bad* so we'll have limited time to finalize everything in the software level I'm sure :) (I'm not an official voice, just a contributor mind you ;)

Anyway, you have to think it through and you'll see some of the implications; normal package manages will work (like ipkg, apt etc), but you have to consider implications for a flash based handheld. We still will use such things for firmware updates, mind you, but for applications/games/etc we rather expect people to use pnd or pxml apps.

There should be lots of info floating around, but I admit, its mostly scattered.

jeff
 
QUOTE
Anyway, you have to think it through and you'll see some of the implications; normal package manages will work (like ipkg, apt etc), but you have to consider implications for a flash based handheld. We still will use such things for firmware updates, mind you, but for applications/games/etc we rather expect people to use pnd or pxml apps.


I think its very important to manage this properly. Will I be able to create PND/PXML packages on the Pandora itself? If not, for sure, its needed. As a developer, my dream is that I can use the Pandora to compile my package, build the package containers, and upload the package From The Pandora Device Itself to a public repo for distribution. Please, please, please consider this carefully: if the Pandora is fully self-contained and contains everything from the factory needed to get an app out to the other Pandora users, it will be a *HUGE* leap forward.
 
, but you have to consider implications for a flash based handheld. We still will use such things for firmware updates, mind you, but for applications/games/etc we rather expect people to use pnd or pxml apps.
I think its very important to manage this properly. Will I be able to create PND/PXML packages on the Pandora itself? If not, for sure, its needed. As a developer, my dream is that I can use the Pandora to compile my package, build the package containers, and upload the package From The Pandora Device Itself to a public repo for distribution. Please, please, please consider this carefully: if the Pandora is fully self-contained and contains everything from the factory needed to get an app out to the other Pandora users, it will be a *HUGE* leap forward.

1. You compile your app using gcc/whatever.
2. You assemble all the resources, files and binaries that your application needs in a folder somewhere.
3. You use mkisofs or your favourite ISO tool to create an ISO image from that folder. (there might also be ZIP support later iirc)
4. You write a PXML file in a text editor and use cat to append it to the ISO file:
cat myiso.iso PXML.xml > mypnd.pnd
5. You take a screenshot of your app, save it as a PNG image and append that the same way as above.
cat mypnd.pnd screenshot.png > final.pnd
6. If you use my PND Manager, you can log in on the web frontend and upload the file there.
All done.

No custom tools needed, no tweaks, no nothing. If you just manage to make a valid PXML file (which isn't difficult), everything should go well. And of course you can do this on the Pandora; the tools are basic tools that are available everywhere.
 
Last edited by a moderator:
And of course you can do this on the Pandora; the tools are basic tools that are available everywhere.
I think its vital that a script or set of tools be shipped *on the Pandora* to make this happen on the device as simply and as easily as possible. If people can really truly ship from their Pandora to the community it will serve to suppress a *lot* of fracture.
 
Last edited by a moderator:
dflem -- we suggest using tools to do the work (ie: sure its appending pxml to the iso, but using a script lets us update this behaviour in the future without risking incompatibilities. ie: we've often gabbed about putting icons in the append list, and index files to show the offsets to appended bits, and all sorts of _madness_ but in the the end we went with simple, first.. but be nice to refer to the sh as 'faith' for most people.) But yeah, what dflem said :) dflem knows his shit, listen to him :)

1) Apps can be pxml apps -- a directory with a PXML.xml file; naturally you can do that anywhere with zero effort
2) Apps can be .pnd files, and we include a simple script to assemble them (which does work on pandora.) We expect peopel to hack out a pile of toosl to do this on whatever platform in easy ways

I'm sure people will pump out tools to make PXML.xml files, and maybe I or someone shoudl make a sh scirpt that just makes a dumb template nearly-empty one.

Hell, I don't expect anyone to use any of the code we're supplkying realy, but its all made relatively simple and most of the ewokr is standard linux stuff, so no biggy, and yes, you can do what you want :)

The one thign I've been mulling over is .. maybe I or someone needs to start writing up some scant details in the wiki (rather than our private dev wiki), so informatiion begins to be centralized. Sadly, I hadn't anticipated having 3-4 months of life get destroyed by work (yay for layoffs meaning us non-laid-off are working two jobs at once effectively, and I'm not the only one in the volunteer dev team in this boat :/) I'll get to it ;)

The libpnd and associated is all in the git mind you though, so you can see the pnd_run.sh and pnd_make.sh and such right now if you like. (libpnd invokes pnd_run to run apps, and .desktop files will refer to pnd_run.sh in their Exec lines to run the apps. pnd_run runs pnd and pxml.xml dir apps, with all the clever mounting magic we've talked about. pnd_make.sh just does some iso and cat magic to turn a PXML.xml app dir into a .pnd file, and the system shoudl more or less treat app-dir and .pnd apps as the same.

libpnd includs pndnotifyd a little daemon hack I made which watches certain (configured, everythign is configured out the wazoo) paths, and when you change those paths (drop new files in, or remove some, say) it will cause the various launchers/etc to rescan for apps, and auto generate .desktop files and all that business.

All works on the pandora itself, or you can make pnds and such on your deskotp, or whatever :)

jeff
 
I tried downloading your server to play with it, and I get this error when logging in with my Openid from Launchpad.
CODE
Error

* We could not log in, because: org.openid4java.consumer.VerificationResult@29da5d65
 
I tried downloading your server to play with it, and I get this error when logging in with my Openid from Launchpad.
CODE
Error
* We could not log in, because: org.openid4java.consumer.VerificationResult@29da5d65

Congratulations; you raised an exception that I thought was impossible to raise! It's good to have external testers of the application, is it not?
Anyways, I patched the application to create a more human-readable error message (do a git pull to get the patch); if you could try to reproduce your problem and give me the error message (because I know where the error comes from, but not what it is, from your provided information) I'd be very grateful.
 
Last edited by a moderator:
That looks pretty impressive, and features a screenshot from an JME-Demo. Nice. :)
 
That looks pretty impressive, and features a screenshot from an JME-Demo. Nice. :)

(incredibly offtopic:)
That screenshot actually DOES NOT come from a jME demo; I needed a depth of field blur shader that was better than the really bad one that is included in the jME distribution and made one using one of the test scenes in jME to test it. Then, when I made my test data PND files, I just grabbed some random image and attached it.
(slightly more on-topic:)
The primary goal of me attaching that screen shot was that I wanted to test the "lossless" (aka artifact-free) bi-cubic size scaling algorithm that resizes all screen shots when they are uploaded to PND Manager; the actual attached screen shot is 800x600 pixels big; if you open the PND with a hex editor or something, you'll see that this is the case.
(incredibly on-topic)
I just wanted to put the question out there:
Would anyone want to help me with testing this application? I'll be done very soonish, but there might be some bugs that just have to be ironed out before I release the app for public consumption. If someone would take his/her time and really try through all the options, settings, pages, graphs etc on the page, I'd be very grateful.
There are some bugs that I'm aware of currently, such as that the site memory monitor diagram makes weird jumps sometimes while updating, and that its legend is stretched out to 550px for some reason. However, if anyone notices anything that I'm NOT currently aware of, please don't hesitate to tell me about it :)
 
Last edited by a moderator:
alright, got that extra info you wanted.
CODE
Error

* We could not log in, because: Nonce verificaton failed.

Could it be related to these?
QUOTE
http://groups.google.com/group/openid4java...1989985e6d536cc
http://groups.google.com/group/openid4java...48c9ff2a0def791

Also, since many people I think will want to log in the site right away, it might be nice to make the log-in access more than just the front page <a> link or User menu. I'd say a button under the menu(or something) would seem to be easy quick click for those on a rampage to find/change/upload a pnd.
Also, in case you wanted to know, I'm testing this on Kubuntu 9.04 non-standard kernel(aka 2.6.29) using SwiftWeasel 3.5pre(rc2 x86_64) as my browser.
When viewing the site stats page, idk how it's supposed to look, but my screen image looks off.
http://imagebin.ca/view/qmuuaA3r.html
I will keep a look out for more things.

EDIT:also,what did you compress the pnd tests in? I tried to use ark and it asked me what kind it was and zip didn't work.
 
alright, got that extra info you wanted.
CODE
Error
* We could not log in, because: Nonce verificaton failed.

Could it be related to these?
QUOTE
http://groups.google.com/group/openid4java...1989985e6d536cc
http://groups.google.com/group/openid4java...48c9ff2a0def791

Also, since many people I think will want to log in the site right away, it might be nice to make the log-in access more than just the front page <a> link or User menu. I'd say a button under the menu(or something) would seem to be easy quick click for those on a rampage to find/change/upload a pnd.
Also, in case you wanted to know, I'm testing this on Kubuntu 9.04 non-standard kernel(aka 2.6.29) using SwiftWeasel 3.5pre(rc2 x86_64) as my browser.
When viewing the site stats page, idk how it's supposed to look, but my screen image looks off.
http://imagebin.ca/view/qmuuaA3r.html
I will keep a look out for more things.

EDIT:also,what did you compress the pnd tests in? I tried to use ark and it asked me what kind it was and zip didn't work.

I'll have to do some more research about that error...
Login box? Sure, why not! But you can use the full power of the PND manager without logging in; the only thing missing is the ability to upload packages.
The site statistics graph is broken and the problem lies somewhere upstream; I'll have to repackage lift-widgets myself (or make a mailing list post that can take ages to create any ripples) :(. However, since the statistics page only really matters to developers, I might as well remove it.

The PND tests are regular PND files: standard ISO image files with some meta information appended at the end. Ark can't load them because it isn't able to read ISO files (tried it a minute ago; heh, I also happen to be on Kubuntu 9.04 - KDE 4.3 atm). Try to do the usual "mount -o loop bla.pnd ~/mountpoint" and see if that works better. The PND files are only test files, however, taken from the official libpnd repository, so they don't contain anything exciting :p

EDIT: oh and BTW, I'm currently doing some major refactoring that brings regression and havoc, so it'll take some time before I'll commit something back tot he repository.
 
Last edited by a moderator:
BUMP

I'm in need of a competent HTML/jQuery web designer to create a better template for the pndmanager application.
It should either be someone who has a design idea and is willing to implement it, or someone who can provide a very detailed and feasible mockup design so that I can write the theme myself.

The following things need to be done (one way or another):
  • Create a new global template. This template is applied to all pages in the web application.
  • Create templates for individual pages if they need them. This should not be needed, however.
  • Create a new template especially for the "Browse Packages" template. It looks nice as it is, but seems kind of misaligned in the rich view, and could have a much more elegant design.
If you decide that you are up for it, please see this page for more information: http://wiki.github.com/dflemstr/pndmanager...nt-instructions
...and of course, make a post here.
The application is slowly getting ready for 1.0 release (I just plan on adding some statistics diagrams and then it's done) so if I don't get anyone to do it now, I'll just leave my own crappy theme on there and just announce the application. It's open source, after all, so it can be modified later.
 
Back
Top