Standardized Control Hints


foxblock

Asleep
Joined
Jun 17, 2009
Messages
1,563
Location
Germany
While, over here, we are discussing a set of standardized (default) buttons to control full-screen applications (such as emulators, games, launchers) another important topic has come up:


Button control hints.


See this screenshot of Mario Party - as a veteran N64 Gamecube player you will immediately spot the little pictograms used to show what buttons control the current scene. As a new player you will also spot those and even without knowing the controller-layout it will help you to easily find out how to control the game.


This is immensely helpful (maybe even more than a standardized set of buttons) and most consoles had their own set of graphics and methods (read: layout, text, etc.) to display these.


Now this thread is to discuss this topic in regards with the Pandora. (for some more details on the topic read the "Standardized controls" thread at the top - especially posts by Christoph.Krn).


I have gone forward and created a few mock-up images of how graphics could look like:

Pandora_buttons_all.png
Pandora_buttons_B.png
http://foxblock.pira...a_buttons_Y.png
SVG: ALL - B - Y

For a single button I display the immediate neighbour buttons as the Pandora has no coloured buttons to make it easy to distinguish them just by a short look.


Sadly the 3D "effect" introduced some artefacts, maybe a skilled artist can pick up this idea if it is commonly accepted.


(Also transparency is missing, oh well... mock-ups)


Also there is this nice control-mapping image mcobit did for rRootage, which could be turned into a template (though a top-down view might be better):

rrootage_controls.png



Please share your ideas on the matter.


(This topic was created rather quickly and without a real solution read to split off-topic discussion in the other thread, sorry about that)
 
Last edited by a moderator:
The following tries to take apart some of the reasons why and where the Pandora has low usability, why it is so difficult to do something about it, and how the situation can hopefully be improved.


"1)" Always refers to a menu control scheme standard


"2)" always refers to standardized control hints


(It is the most important part of what I've written in the other thread as it was on 30oct2011; any changes will be made to the actual post in the other thread. It was never intended to be a thread starter, and in this role is not of high quality. Also, the following effects that I talked about have to be declared as "Yet to be proven correct." for the sake of precision.)


------------------------------------------------------------------------

If I understand this post of yours correctly, then you highlight as main issue the following:


There is confusion amongst users as to which buttons are mapped to which action. This is caused by

  • 1) Lack of standardization causes every application to utilize a different key for a different action.
  • 2) There is no standardized way to communicate this binding to the end-user. You cite the Wii menu as an example.


The topic of this thread was to address issue 1., not issue 2. In the ideal case issue two would be handled by the launcher, e.g. by querying the applications. It is however, from a technical point of view, a totally different challenge which warrants a discussion of its own.
Yes except that 2) is more like a tool chest for developers, plus there is something else to these two points.

  • The definition of "confusion" I use here is a very specific one. It means conscious confusion that is caused by an increased overhead of /non-conscious/ processing. The non-conscious nature of this overhead makes it difficult to see.
  • "games" refers to most full-screen applications, including things like Minimenu or Rockbox.
  • 2) should not be implemented through something like the Wii home menu, but rather:
    a) In-menu hints through standardized graphics (much like in this screenshot of Mario Party 5). Some games already have these (random examples: PandoraPanic! and Panorama) but do not always use them consistently, and the graphics often do not make much sense, mainly because of the similarities between the game buttons of the Pandora, the SNES, and the PlayStation. A standardized set of in-game hint graphics of high quality that are easy to put into an application would be very useful. (Any standard on how these graphics should be used can prevent their use and should be created later - developers cannot be forced to adhere to it anyway.)
  • B) A Visual one-screen overview (much like in one of the screenshots at http://www.videogamesblogger.com/2009/10/06/metroid-prime-review-samus-returns-to-gaming-in-glorious-3d-from-an-all-new-perspective.htm , search for "layout" there, or in the screenshots at http://forza.norm302.com/fm/features8.html ). As a first step, this does not necessarily have to be implemented by the launcher so that developers just hand the button assignments over and the user can access it whenever hints are not available with a single press of the Pandora button, where the overview is integrated along the other functions of that launcher screen. As a first step, the developers can hardwire this overview into their games and make it accessible through a button that they hint to at some point (e.g. in the Start/Pause menu). This does mean that these games will not automatically use newer standardized graphics, but this is not avoidable anyway because of the in-menu hints that can not automatically update either (it does not make sense for them to do; this might break something in some games and at the very least alters the game experience). In case this will ever be handled by the launchers, developers can remove the legacy hardwired overview and, if necessary, replace it with a hint to the new function. This is still better than the current nothing.


[*]In the context of 2), it is important to know that design is not what it looks and feels like, it's how it works. Every part that plays into design in its various definitions is potentially of importance when trying to make sure that 1) will succeed in reducing confusion.


[*]The impact of the menu navigation standards as proposed here (the "common denominator") is more limited than one might think. There are many reasons for this, but probably most importantly, the menus typically used most frequently are likely to use remaining buttons for random other tasks, thereby causing cognitive overload (==confusion) regardless of the standard. If more buttons are standardized in an effort to mitigate this problem, then the menus that had used the previously remaining free buttons will now have to either break standard compliance, or somehow put the functions somewhere else. If more button _functions_ (environment variables) are being introduced, they will lead to disharmony between people and thus greatly decrease the chance that any further efforts will succeed. If no more button standards or functions are added, then the cognitive overload remains.


[*]There are many reasons control standardization efforts in general can fail in an open ecosystem, and many of them are not controllable. A developer might not support 1) because it does not make sense for her application, or simply because she prefers something else, or for a large number of other reasons. The more immersive 1), the worse these effects; the less immersive 1), the less sure it is that it will manage to keep cognitive overload down without a little help from 2).


[*]All of the suggestions that were made to the very basic "common denominator" standard share a very limited number of some extremely basic concepts, like B to accept, X to cancel, Start to pause. However, even the discussion about all other parts of the suggestions, like "Select to exit?" or "RShoulder==DPadUp?", indicates how much of a non-conscious "experience" these other parts are that people might not realize.


-> _Example:_ Space as an overlay menu standard is mainly helpful to those who /already know that there is an overlay menu and who already know that they have to press space to open it up/. It makes sense to those that create the standard, but not to those who are lost in an emulated game, do not know how to get out, and maybe do not even know that there is an overlay menu. One might say that "Novice users first have to learn that there is an overlay menu, the standard is not there to teach them", but that's not true. The user might not even be a novice user at all - he's not stupid, he just doesn't know. The confusion from that situation is not very different from the one experienced when a PND's developer failed to communicate how to quit his application, and it is similar to the confusion one experiences in a foreign menu (there will always be situations like these since `1)' is very limited). The more logical (==less confusing) solution for an emulator overlay menu would be to _put_the_overlay_menu_on_Esc_ and to provide some hint to the Esc key's function, with a "quit" option in that menu. This would make sense and in turn incite other developers to do the same, which is exactly what is required. (A developer who did this would break compatibility to the currently proposed standard!) Again, this is just an example.


[*]2) is very /visible/ to the user, who is a human being and as such typically uses vision as his primary means to get information from the world. This increases general awareness of 1) through the existence of 2). The result is, hopefully, an increased incentive to support and help improve 1) (as well as `2)') on both the side of the developers as well as the side of the users. This incentive is crucial in this open development environment.





There are many loose ends that come together.


1) is unlikely to be very helpful in an open environment such as the Pandoras, but 2) can be used to reduce remaining problems and make 1) successful anyway. I hope this makes a little more sense. It splits the potential success of this standardized controls effort here into standardization, graphic design, software design, and psychology, (all part of usability) and might still be difficult to understand. Please tell me if something is unclear.
 
Last edited by a moderator:
The other thread is taking a bit of a turn to technically making the gaming controls first order citizens to applications and to provide a technical mechanism to a user to set his/her own (global) preferences for those controls. Right now, applications are not aware of the existance of those controls other than through a default mapping of regular keyboard keys.


So let's explore the options for control hints here. Should these two topics eventually need to converge then I recommend we summarize a combined standard/proposal on the wiki.

..."games" refers to most full-screen applications, including things like Minimenu or Rockbox.
I agree, though running such full-screen applications in a window would not exclude them from the definition. I'm not entirely sure what the exact definition of "games" should be, but I think we are on the same page.

Foxblock made a first mockup for these in his post above. The first goal would be to perfect these and make them available as a developer resource to whoever wants to provide control hints to the user, either during gameplay (where a is more suitable) or in an overview menu (where b is more suitable). In a second stage we can investigate a solution which provides a simple way for applications to communicate such bindings back to the OS such that an external, uniform display can be presented. The visual one-screen overview could be utilized for that.


I'm a bit at a loss as to what exactly needs to happen for the first stage aside from creation of these graphics. For the second stage applications should be able to report e.g. (key,value) pairs to the OS containing button/action-description. When the Pandora key is pressed, the configuration of the focused application could be fetched and drawn as an overlay over the one-screen overview.


Another concern which I noticed in some novice users around me is that they struggle immensely with the layered controls in emulators. I.e. the emulated game refers to non-existing controls. It is not possible to detect any such references and I'm not entirely sure how to address this or if this even should be addressed at all (I have the impression that our niche audience can deal with it).
 
Last edited by a moderator:
I'm not entirely sure what the exact definition of "games" should be, but I think we are on the same page.
The term "games", in this context, can be seen as a substitute for "Pandora specific fullscreen applications such as multimedia applications, games, launchers and similar". What I proposed does not necessarily have to be limited to these things, but it makes most sense there.

Foxblock made a first mockup for these in his post above. The first goal would be to perfect these and make them available as a developer resource to whoever wants to provide control hints to the user, [...]
With no personal offense intended: these are not very well suited for the intended purpose for a large number of reasons.


I do actually already have some months-old sketches (as well as some more things) related to my ideas as I was already working on the standards, but making something that is of really high quality (which standards should always be) typically requires a lot of time for design, implementation and testing. This also means that I would not really have started a thread on my own at this point. Just like I already said in the other thread, this thread here is now not maximally likely to result in high-quality results. Instead it is now much more likely that, out of all the proposals that are being submitted in a hurry (and therefore are very unlikely to be of high quality), the one that people perceive as the least bad will be "poll"ed. (I just wrote this because that makes this scenario less likely, but then again this does not help with the fact that it's unlikely that there will be any more proposals at all.)

[...] provide control hints to the user, either during gameplay (where a is more suitable) or in an overview menu (where b is more suitable).
I assume it's unlikely that you mixed up a and b there, so I assume that there is some kind of communication failure here.


For a menu, small hints (a) are better suited. These can be put at the bottom of the screen, in front of menu entries or into buttons. During gameplay, a button map ( B) is is the only solution that works in nearly all cases (hints may not fit the gameplay experience) and it should be accessible easily, e.g. it can be shown in the start menu (hardwired as long as there is no launcher support).

I'm a bit at a loss as to what exactly needs to happen for the first stage aside from creation of these graphics.
Well, nothing actually, that's the greates thing about them. As long as the hinting graphics are high-quality, they will be self-explanatory for developers and work in any situation that they are used in. As suggested by my previous post, any further obstacles for developers are likely to be non-beneficial at this point (unless I'm missing something here).

For the second stage applications should be able to report e.g. (key,value) pairs to the OS containing button/action-description. When the Pandora key is pressed, the configuration of the focused application could be fetched and drawn as an overlay over the one-screen overview.
Applications should also be offered some common mappings, like "D-Pad to select" instead of reporting every single D-Pad direction to the user. Another thing to do would be to create a good way of displaying key combinations. And just in case, the displayed visual representation should be completely overridable by the application.


Very likely there are more things like these that can be done.

Another concern which I noticed in some novice users around me is that they struggle immensely with the layered controls in emulators. I.e. the emulated game refers to non-existing controls. It is not possible to detect any such references and I'm not entirely sure how to address this or if this even should be addressed at all (I have the impression that our niche audience can deal with it).
People who know how to use emulators (know how to install an emulator; know what an emulator is there for) also are unbelievably likely to know that different emulated games can have different control schemes. They probably understand any (most, at least) associated problems. Those people who do NOT know how to use emulators are the ones who have been pointed to the emulator by other people, and are likely to ask those about any related urgent questions. The problem mentioned does not only apply to the controls of emulated games, it's a result of a more generic thing.


Therefore, communicating to the user which of the Pandora's buttons is mapped to which button of the emulated controller is sufficient. This can be done using a button map (`(b)', the "Visual one-screen overview") in the emulator's overlay-menu. Any further problems, such as games that refer to obscure gaming system peripherals or software that can emulate a number of systems, can likely be mitigated through a short explanatory note next to the button map that simply says: "These are the emulated controls:".
 
Last edited by a moderator:
With no personal offense intended: these are not very well suited for the intended purpose for a large number of reasons.


I do actually already have some months-old sketches (as well as some more things) related to my ideas as I was already working on the standards, but making something that is of really high quality (which standards should always be) typically requires a lot of time for design, implementation and testing. This also means that I would not really have started a thread on my own at this point. Just like I already said in the other thread, this thread here is now not maximally likely to result in high-quality results. Instead it is now much more likely that, out of all the proposals that are being submitted in a hurry (and therefore are very unlikely to be of high quality), the one that people perceive as the least bad will be "poll"ed. (I just wrote this because that makes this scenario less likely, but then again this does not help with the fact that it's unlikely that there will be any more proposals at all.)
I am not offended, but could you please share those reasons, because only that way those graphics can be improved or re-done.


Additionally there are a few quite skilled graphic artists in the community, I am pretty sure they can come up with some high-quality work (in terms of graphics, but also layout).


We have time, there is no need to rush this decision (some people might make it look like that, but there really isn't), so I don't see your anxiety of a rushed solution.


This thread has the benefit of bringing multiple (highly intelligent and skilled) people together to share ideas. One guy in his basement is very unlikely to come up with the perfect solution, firstly because he usually is not involved in all parts of development such as games, office applications, music players, launchers, etc. (I am not exactly sure about your role here, I have never seen you release anything, so I don't know your knowledge on this fields), secondly because he has no metrics or user-reports or observation of use-cases and lastly because he is just one mind and people tend to not think of everything (just like the other thread and proposed control schemas show).


So if you find the time, I think everyone would appreciate if you shared the sketches and ideas you have - even if they are not fully fleshed-out yet (yes, this created problems in the other thread, but the reason was less the state of the ideas than the fact that they were slightly unrelated, which lead to the creation of this thread in the first place).


Ideas are not any good if they are only used to catch dust.
 
Last edited by a moderator:
With no personal offense intended: these are not very well suited for the intended purpose for a large number of reasons.
The design of these lies outside my area of expertise. I get the impression you have a very clear idea in your mind of what is required and why, yet every post you make about it leaves me a bit confused and I need to read it at least twice before I get what you are talking about.


Perhaps you could sketch some criteria and metrics for these images, so I can understand your motivation and reasoning.

I do actually already have some months-old sketches (as well as some more things) related to my ideas as I was already working on the standards, but making something that is of really high quality (which standards should always be) typically requires a lot of time for design, implementation and testing.
Ok, take your time. However, you do not need to design, implement and test everything by yourself. If you outline your vision, others will think with you and provide input and feedback.

Instead it is now much more likely that, out of all the proposals that are being submitted in a hurry (and therefore are very unlikely to be of high quality), the one that people perceive as the least bad will be "poll"ed. (I just wrote this because that makes this scenario less likely, but then again this does not help with the fact that it's unlikely that there will be any more proposals at all.)
I agree that as long as there are serious open issues a standard should not be pushed. On the other hand, we should also not get lost in unachievable dreams. You are the originator of the idea for standardized control hints, if you want to take the lead, go ahead. I'm interested to see more about this as it has the potential to make a giant leap in reducing the barrier of entry.

[...] provide control hints to the user, either during gameplay (where a is more suitable) or in an overview menu (where b is more suitable).
I assume it's unlikely that you mixed up a and b there, so I assume that there is some kind of communication failure here.
I was thinking of the small hints appearing on the screen like they would in e.g. quick-time events (I hate those, but it's an example). You would not pop-up an image of a full controller during the action.

As suggested by my previous post, any further obstacles for developers are likely to be non-beneficial at this point (unless I'm missing something here).
No, I think that is pretty much spot on. For the developers the ease of support is important. Often having access to high quality artwork is an important issue as often developers are lousy artists/designers (naturally there are plenty of exceptions). On the other hand, if there is an extremely accessible API for accessing a standard mechanism, then using that might be actually less work. For the time being, the artwork is priority as without that, all this is moot anyway.

Therefore, communicating to the user which of the Pandora's buttons is mapped to which button of the emulated controller is sufficient. This can be done using a button map (`(b)', the "Visual one-screen overview") in the emulator's overlay-menu.
Yes, that makes sense.
 
[...], you do not need to design, implement and test everything by yourself.
Doing so leads to accumulating knowledge from many fields. If I hadn't done so when I programmed games a "long" time ago, I would have hardly any knowledge about graphic design, and likely also less knowledge about usability. I am well aware that I do not need to do everything on my own, and actually often I deliberately help initialize something I will not partake in (if it serves a superordinate purpose). Thank you for your attempt to point this out to me though.


I think through how to improve things all of the time whatever I'm doing, so the ideas I shared here actually have to be declared a by-product (that I have shared too early). Sadly, I cannot spend time on all of these by-products, especially with a high quality approach. But this here by-product does have a high priority. By making an attempt to explain it too early, time was now actually _lost_.


An openpandora.org subdomain is already requested.


You have already understood part of why there is more to the hints than just graphics. The most unusual ideas are by nature not average, which means that they are not easily understood. If many things associated with the idea are required for many other things to successfully happen, then the effort required to make people understand that web of interconnections (often a requirement for asking for support) explodes, as you have experienced youself. If the emphasis of key points is out of balance because some points are higlighted too much, many people will fail to see the web beneath, especially when these people are focused on surfing the internet and not on listening to random ideas. The required effort is greatly reduced if an implementation that impersonates the ideas is created first, and then simply shown to others who will then hopefully eventually see all the things it does and eventually improve upon it. You may now think that this process is a lot of work, and, yes, it is indeed a lot of work, but that's a good thing.

I was thinking of the small hints appearing on the screen like they would in e.g. quick-time events (I hate those, but it's an example). You would not pop-up an image of a full controller during the action.
Ah, instead of "overview menu" I read "menu overview". That explains it.

I am not offended, but could you please share those reasons, because only that way those graphics can be improved or re-done.
Creating good icons is an art in itself. I don't think I'm a master of this art, I especially lack training and experience, but even if I was, it wouldn't help if I pointed out only the current problems of your proposal. The internet holds quite a bit of information about what not to do in icon creation, so you could:

  • Read some internet resources, like for instance 7 Principles of Effective Icon Design
    Additionally, keep in mind that the hints have to fit into a large number of diverse games without looking alien.
  • Additionally, keep in mind that the hints may look very different on the average Pandora screen than they do on your monitor: contrast, saturation, hue, fidelity, size, everything.
  • Additionally, reading Dieter Rams: ten principles for good design might cause additional helpful insights.
 
Back
Top