While browsing through the questions in UX.SE, I noticed that that a lot of answers to questions don't have links to actual tests/studies. According to this answer that's OK, and we shouldn't do anything to stop it.

However, there was one answer to that question that said something about a lack of good studies to use as reference for these questions. I'm not sure how true that is, but it got me thinking. It would be interesting if the UX.SE community got together and ran a few UX studies and tests of our own, preferably to provide real data to answer a popular question on UX.SE.

UPDATE: Here is a more detailed proposal (feel free to comment):

  • Every week (or every two weeks, or every month, depending on how many people are interested), a question on the UX meta would be created asking people to submit proposals for possible UX studies that we could run. The proposals would detail what data the studies would collect, how that data would be collected, and why it would be useful. The person who submitted the proposal would be responsible for managing the entire study, though he or she could get other people to help.
  • The proposal with the most up votes will be selected to be used as our study of the week (or whatever time interval we decide on).
  • Once the study has been completed, we will post the results on a blog (see this post)

An example proposal of a possible UX study:

Please Note: This is just a draft of an example proposal.

Are links with images more likely to be clicked on?

Based off of this question (please read it so you will understand the proposal): Are links with images more likely to be clicked on?

  • This test would consist of three webpages: one with a link with a small thumbnail (as described in the question), one with the link described in the above, but with a large thumbnail; and one with the same link described above, only without the thumbnail.
  • There would be some javascript on every page that would count what portion of the visitors clicked the link, and if they did, how long it took them to do so.
  • The data would show whether having a thumbnail improves click through rates, and whether the size of the thumbnail matters.

This is the more old proposal: (only it's not much of a proposal)

If people are interested, then what we need to do is:

  • We need to know how we are going to run these studies. Ideally, then should let anyone do them from a computer at home, so that everybody can participate (unfortunately, that means no fancy equipment).
  • We are going to need to decide who is going to make, or program, the studies. Perhaps alternate every week?
  • Decide if we are going to do studies to answer popular questions (on UX.SE), or do studies to find out random statistics.
    • If we are going to do studies to answer popular questions, we need to decide which questions we are going to answer.
  • We are going to need a chatroom or some other sort of hq (I'll create it once I know people are interested).
  • I was thinking that we would publish the studies as answered to popular questions, but if we decide to just do studies about random but interesting UX related things, then that wont work. We could use a blog (see this answer on the stack overflow meta).
    • If we are going to studies to answer popular questions (on UX.SE), then a blog might still be a cool "supplementary" publishing platform.

Let me know what you think :) Questions and Comments are welcome.

share
    
The question has been updated to answer some of the questions raised in this post: meta.ux.stackexchange.com/questions/355/… –  Christofian May 22 '11 at 15:47
add comment

4 Answers

I think it's an interesting idea and you should press on. Don't ask for permission. Don't form a committee. Just go ahead and put together a study and let us know where you need help.

share
4  
Oh. Yeah. This. This is the only way to get things done. –  Rahul May 26 '11 at 22:44
add comment

A UX study is possibly something Stack Exchange, Inc could underwrite.

We do have some nascent community grant programs underway, so feel free to hammer this into a detailed proposal!

share
    
I'm waiting to see how many people are interested first. –  Christofian May 21 '11 at 14:38
    
a good sponsored study from UX.SE would be double rainbow –  Naoise Golden Jan 2 '12 at 17:28
add comment

I'm kind of wary of this as I feel like it raises more questions than it answers.

What exactly would we study? What would be the goal? Just providing backup to certain answers to questions? Which questions? How would we pick which to pursue? Would it be worth it, given that the most popular of our questions so far only have 3k views? Wouldn't it be better to invest resources in increasing awareness of StackUX before we start doing research (or would doing the research be a way of achieving that)?

How would we do the research in practice? Who would do it and how would we decide? How would we ensure that it's reported accurately and objectively?

I don't mean to be a party pooper but right now it feels like this would be hard to get off the ground. Perhaps we need a more concrete plan? I agree that we should gauge interest before making a decision either way, though.

share
1  
My sentiments exactly. Design-by-committee is a very inefficient process, let alone design-perform-and-then-analyze-a-study. I doubt that this way we can pull off a study with any degree of validity. –  Vitaly Mijiritsky May 22 '11 at 5:51
    
Well, I did have some ideas, though I was waiting to see if anyone is interested, and to get some community feedback on the proposal; before submitting them. However, I am guessing that this proposal needs more detail before that can happen, so I will be adding more detail to it as soon as I can find the time. –  Christofian May 22 '11 at 14:37
1  
To answer one of your questions: doing research would increase awareness of UX.SE. If we produced lots of useful studies, the studies would get attention from the ux world, and UX.SE, as the source of the studies, would also get attention. –  Christofian May 22 '11 at 14:54
    
@Rahul I have updated the question to include answers to your questions. Please tell me if I didn't answer anything, and I will add it. –  Christofian May 22 '11 at 16:43
    
@Christofian Still not convinced. How do we know the person submitting the study is someone who knows how to do studies? How will they perform it? When/where? Also, I don't think meta.UX is frequented enough for there to be interest in this from enough people to make it worthwhile. –  Rahul May 23 '11 at 12:44
    
Also, I don't think meta.UX is frequented enough for there to be interest in this from enough people to make it worthwhile. If we ran something similar to community ads, then more people might notice the topic. –  Christofian May 23 '11 at 21:33
    
How will they perform it? When/where? Take a look at the example test proposal for an idea on how the tests would be run. –  Christofian May 23 '11 at 21:34
    
How do we know the person submitting the study is someone who knows how to do studies? I really don't know. We would probably only let people run studies who have a bit of reputation already, but that isn't a guarantee. I guess we could review the code before the study starts to make sure it works, and host it on se so we know that they don't manipulate the data. If anyone has any ideas, please speak up (I should think of something soon :) ). –  Christofian May 23 '11 at 21:38
1  
I think this might be a good idea once we have a bigger userbase and there are some people who'd offer to help you on this. Perhaps you should find some people who will vouch for doing studies with you as a start, and leave the "people can create a question when they want to do a study" for later, when there's already proven interest? –  Rahul May 24 '11 at 10:14
add comment

I'm 1000% in for bringing forward more "tangible" studies on UX. You are right when you say right now most answers are pure top-of-the-head opinion, which in some cases can be based on experience or on real knowledge with no inmediatelly available links to sources, which is acceptable, but I've got the feeling this is the exception.

Nielsen Norman Group have some good studies, but they go at $300+ a pop. We've got some good UX/UI Guidelines (Yahoo, Apple, Android, Microsoft), but following them is an exercise of faith, because they don't publish sources to what they assert.

And then we can do a search on the uncountable analytics, marketing, design, etc. firms that publish their investigations.

Every extra "serious" investigations on UX will always be good and I will participate.

I am sure most of us have sites/apps that could be used in "experiments".

That said, I feel it's a bit naive to have it organized as a community, this things tend to work when there's an interest behind, individually or as a company, and interest normally translates in money. Sad but true. Thus many questions rise as other answers state.

Nevertheless:

A proposal

  1. Have some kind of election to choose which UX:SE questions should be studied, like the Community Promotion Ads posts here in meta are elected.

  2. A very concise type of study should be set, like Google Analytics' A/B Testings (since most of us use GA) and which sites/apps qualify for testing.

  3. After picking some winners all interested users should run the tests in their own and come back with the results.

  4. Results should be peer-review for quality. Then published in some neutral site. Since it is in UX.SE's interest, having a special section for these studies would be a good thing, it would also give legitimacy and notoriety.

Some bonus Amazon vouchers would also go a long way to reclute volunteers ;)

share
add comment

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .