User behavior in SERPs. Eye tracking study July 2010

This fantastic study was done by Mari Carmen Marcos @mcmarcos and Cristina González Caro at Pompeu Fabra University in Barcelona and published July 2010 by magazine El profesional de la Información (copyright owners).

It can be downloaded here in Spanish but as I found it interesting for SEO consultants and there is no English translation I want to summarize and put it into my bad English.

Eye tracking experiment design

There are some previous studies analyzing the search engines results pages (SERPs) but this is the first one trying to determine whether the intention behind queries affects the way people browse the results page, the relationship between gaze patterns and intentionality while performing queries.

Participants attempted four different types of queries, informational, navigational, transactional or multimedia.

  • Informational: the user wants to obtain information such as phone number of a hospital, the temperature of a city or the biography of a singer
    Question: What is the schedule of the Louvre museum in Paris?
    Query: louvre hours
  • Navigational: the user wants find a particular website, such as a university where he wants to study, a company's one where he will do a job interview or a newspaper site he would like to read
    Question: Search the official website of the National Spanish TV channel
    Query: "televisión española"
  • Transactional: the user wants to perform an action, such as a software download, buy a flight ticket or request a certificate
    Question: Find a website where you can book a table at a restaurant in downtown Barcelona
    Query: barcelona restaurant
  • Multimedia: the user wants to find a photo or video
    Question: Find a picture of the Alhambra at sunset
    Query: Alhambra at sunset

Search engines
Google, Google Images, Yahoo! and Yahoo! Images

Participants
58 people who regularly use search engines
Between 18 and 55 years old, 80% of them between 20 and 30
25 men (43%), 33 women (57%)
30 minutes by test made individually, not in group

Tasks
22 tasks in total per person, 10 informational, 3 navigational, 4 transactional and 5 multimedia.

Variables
Type of query: informational, navigational, transactional or multimedia.
Type of results: organic or sponsored (paid adds)
Areas of interest (AOIs): titles, snippets, urls and images

Metrics
The number of fixations on each AOI
Time of those fixations in milliseconds for any of those AOIs

Organic and sponsored results
From the total amount of queries performed only 39% had sponsored adds.
This percentage is divided as follows: 36% on informational queries (77 pages), 43% on navigational (25 pages), 76% on transactional (56 pages), 0% on multimedia (0 pages).

Conclusions by query type

Some numbers of the study:

SERPs fixations

User behavior in SERPs, data table

Informational intent queries

Users performing these queries focused more on the snippet element trying to decide whether or not this result is consistent with the information they are looking for. The title was in second position as also offers an important insight of content after the click.

Relevancy by AOI:

  • Title: 37% of fixations, 34% of fixations time
  • Snippet: 51% of fixations, 53% of fixations time
  • URL: 12% of fixations, 13% of fixations time

Navigational intent queries

The order in relevance is the same by fixation time but it changes in favour of title and URL by number of fixations.

Relevancy by AOI:

  • Title: 38% of fixations, 31% of fixation time
  • Snipet: 42% of fixations, 49% of fixation time
  • URL: 13% of fixations, 15% of fixation time

Although keeping a low relevancy, the number of fixations at sponsored results represents 5%, close to double when compared to informational queries, 2,8%.

Transactional intent queries

Organic results are still getting most of the attention, 82%, but a growing user interest for sponsored links is confirmed, 17%.

Relevancy by AOI:

  • Title: 42% of fixations, 28% of fixation time
  • Snipet: 43% of fixations, 56% of fixation time
  • URL: 15% of fixations, 16% of fixation time

Number of fixations at sponsored results is 9,8%, the highest rate compared to 2,8% at informational and 5% at navigational.

The distribution of fixations by area for sponsored results is 79% for the upper area and 21% for the ones on the right column.

Regarding the variables for sponsored, title gets 43% of the time of fixations, snippet 28% and URL 29%.

Multimedia intent queries

By number of fixations, 76%, and time of fixation 71% almost all the attention is focused on images.

Users set their attention mainly in the first row of results, so these images are usually the ones that user comes to evaluate.

User behavior in SERPs Gazeplot Images
Gazeplot for multimedia queries

Conclusions summary

In general terms, the results demonstrate that a relationship exists between the user's intention and their behavior when they browse the results page.

User behavior in SERPs by intention type
Heatmap and gazeplot for different types of queries

If user's intention is to obtain information or to reach a particular website, organic results are the most relevant. Instead, sponsored links are taken into account only if the intention is to perform an action.

Time of fixation by AOI:

  • Organic results: snippet 44%, title 39%, URL 17%
  • Sponsored results: title 43%, snippet 28%, URL 29%

For informational, navigational and transactional queries the most observed element is the snippet followed closely by title in organic results. Snippet is the element where user expect to find the information that will help them decide if they click on the result.

For transactional queries, sponsored results are receiving a bigger number of fixations but in this case the order of relevance of elements is title, snippet and URL.

Finally, for multimedia queries, image queries receive most of the fixations.

Sponsored links does not arouse interest in the search users whose intention is not transactional, and the other way round.

The wording of the title must be carefully thought to reflect the content of the page

Ads at the top have more fixations than those found on the right side column may be because the upper zone more easily confused with organic results or because it is an area where user expects sponsored advertising.

My take on the study

This data is not revealing nothing absolutely new to professional SEO consultants but it is important to have some scientific or numeric evidence to support the practise of this discipline.

We knew the role of snippet as a factor to increase click through rate and the study evidences even more how important can be to balance clicks in favour of one result or another.

Writing carefully titles and meta descriptions is beyond discussion but intentionality adds a new variable to the equation here so try to anticipate user's intention adapting your wording according to the type of service your site offers.

What's your take?

Aug 24, 2010
Written by:
Filed under: Analytics






40 comments
Aug 24, 2010
Posted by:
Bill Slawski #1

Thank you very much for translating the highlights of this study into English, Ani.

I've been hoping to see some data about the differences between different types of query intents using an eyetracking study for a while, and the variation in attention between different parts of an individual search result do fit what I was thinking they might - but it's terrific to see information backing those assumptions.

Aug 24, 2010
Posted by:
Ani Lopez #2

Thanks Bill,
Credit goes to the authors, I just exported their work outside the borders of our country.

Aug 24, 2010
Posted by:
Steve Dossett #3

Thanks for sharing. This is great - I passed it on to a client or two.

Love the text above this comment box by the way - haha. :)

Aug 25, 2010
Posted by:
Stefan #4

Thanks for this! So don't forget to use meta description for popular searches.

Aug 25, 2010
Posted by:
Peter #5

Interesting research!

PS. Its a pity that you presume that all users on the Internet are male!

Aug 25, 2010
Posted by:
William Vicary #6

Thanks so much for translating this, these sort of heat maps are great for showing clients the difference that additional rank can make!

Aug 25, 2010
Posted by:
Ryan Mull #7

Great article... one piece I am wondering about.

The transactional eye tracking piece (the far right two images) looks like a local search, and clearly almost all of the attention is on the paid search ads, and yet the findings say the focus is on organic. Am I missing something here? This finding and oversight seems pretty significant.

Aug 25, 2010
Posted by:
Geno Prussakov #8

Excellent summary of a truly interesting study. It is definitely good to see a scientific study on SEO. I wish we had more research like this being published online.

I would also like to echo Ryan's above question on PPC ads. Was this piece of the study relating to something else (e.g.: importance/role of PPC ad copy)?

Aug 25, 2010
Posted by:
Randy Pickard #9

User Centric conducted a similar eye tracking study focused exclusively on the PPC Ads and the importance of the display URL. A summary article was published by at
http://www.seomoz.org/blog/do-not-overlook-the-importance-of-the-display-urls-in-ppc-ads

Thanks for translating this article. Fascinating to see eye tracking data on how intent influences fixations.

Aug 25, 2010
Posted by:
Barrie Adams #10

Hi Ani, thanks for taking the time to do this, I'm working on getting rich snippets into the index and also concentrating efforts generally to improving snippets for an ecommerce site. It's great to have real data I can show clients to prove the work is worth the development time, and eye tracking studies are like gold dust.

Aug 25, 2010

Big thanks for taking the time to translate this Ani. It's a great look inside one of the most important factors we should watch. You nailed it when you said professional SEO's already know the value of this item, but this type of work (and your effort in translating

Aug 26, 2010
Posted by:
Ani Lopez #12

@Peter: I don't think the authors of the study presume all Internet users are male in fact people participating were more women 57% than men 43%, if this is what you are talking about.

@Ryan Mull and @Geno Prussakov: You are right saying that for transactional, heatmap/gazeplot seems to indicate PPC is getting more attention than organic while in text/stats it says the opposite. I'll ask the authors about.

Thanks everybody for come and comment, I hope you enjoy the rest of the articles.

Aug 26, 2010
Posted by:
Leili #13

very intresting article.

Aug 26, 2010

Nice data to use for SEO and SEM.

Aug 26, 2010


Genial Ani, gracias por acercar este genial estudio al mundo, y arriba en Sphinn , eres un crack :)

Aug 26, 2010
Posted by:
Lee Davies #16

A very interesting study, thanks for translating from the spanish version.

Aug 26, 2010
Posted by:
Mike #17

This is a great study and will be helpful to many professional in the SEO field! And I agree, i see a lot of attention on paid search part of the results. Makes sense since that's where Google is making their money.

Aug 27, 2010
Posted by:
Shari Thurow #18

I think this data, though interesting, can easily be misinterpreted. A snippet has more text in it than a title. Therefore, people will probably spend more time looking at a snippet merely due to allocation of screen real estate rather than genuine interest.

That is a core problem with eyetracking studies - a study might tell you how people look at things, but not why they look at things.

They might look at things because they are distracted from completing a task. And they might look at something due to genuine interest. Again, the "why" part is extremely important.

Aug 27, 2010
Posted by:
Shari Thurow #19

Oh, and one more thing. I hope that people do not overgeneralize this study.

People from different countries, and even from different regions of the same country, look at elements on web pages differently.

Microsoft has done some incredible work on eyetracking, and their data does not match this data. It might be due to the fact that test participants were located in different geographic regions.

Again, everyone, please keep this in perspective. It is very easy to overgeneralize, and misinterpret, eyetracking data.

Aug 27, 2010
Posted by:
Ani Lopez #20

@Shari Thurow interesting notes.
"A snippet has more text in it than a title" right but a title has a bigger text size and different color compensating that.

Do you have any link to the Microsoft study you mentioned? It would not surprise me their conclusions not matching these ones, Microsoft does not match nothing rational :)

You are right preventing not to overgeneralize but what's your take on it?

Aug 27, 2010

Thanks to everybody for these useful comments, we'll take them into account for the following study (we are working hard)

@Peter: "you presume that all users on the Internet are male!": As Ani has answered, we have almost 50% of each genre, and aged from 18 to 61

@Ani @Ryan Mull and @Geno Prussakov. About "for transactional, heatmap/gazeplot seems to indicate PPC is getting more attention than organic while in text/stats it says the opposite": the maps chosen to show are just an example of some maps we have collected, they don't fit exactly to the data but can represent in some way what we have found. Please, take them just as an example, not to generalize

@shari: About "A snippet has more text in it than a title. Therefore, people will probably spend more time looking at a snippet merely due to allocation of screen real estate rather than genuine interest": we say the same in the conclusions in the original paper, but we consider that the data are valid because even the snippet is longer than the title, the title has a bigger font and a blue color, so it is more probable for people to look at the title. If we consider length we should consider other variables, and it is not possible to measure them

@Shari: I agree, this study have been run in Spain and should be run in other region to generalize

Thanks everybody, and especially to Ani!

Aug 31, 2010
Posted by:
Mr Ohidul #22

This was a big issue on the news. I seen it and was talking it out with my partner. Lets see what happens!

Sep 01, 2010
Posted by:
Roshan #23

http://research.microsoft.com/pubs/70395/tr-2007-01.pdf
link to the research - Microsoft eye tracking for web search which shows the significance of the search result optimization and user action on the search results.

Sep 02, 2010
Posted by:
sangil lee #24

hi
Very interesting exp.
I have been studying the psychology of perception

Sep 02, 2010
Posted by:
sangil lee #25

questions about this 1. How did you intend category? info.. navi.. transac.. multi.. Are there any standards?

Sep 02, 2010
Posted by:
sangil lee #26

it be.. Can u tell me about the intent category standards?
2. Except for the results, what can i get other eye movement data?
wait ur email or reply.. thank u so much

Sep 02, 2010
Posted by:
Ted Carroll #27

Studies that I have been involved with show that users attach a higher weighting to the title compared to the snippet. But this can still be compatible with the conclusions of this study i.e. while the user spends longer reading the snippet compared to the title (because it contains more text), they may still attach a higher weighting to the title. One reason for the length of time viewing the snippet may be because the autogenerated snippet is difficult to interpret, for instance!
However, a very useful study.

Sep 06, 2010
Posted by:
Yanniv #28

Nice data to use for SEO and SEM. Interesting points on local vs paid and the moment of consumers.

Sep 08, 2010
Posted by:
Shari Thurow #29

Hi again-

Many people know users/searchers look at the title in the snippet due to the fact that the title is the hypertext link in the snippet and also tends to have more text in it than other links on a search engine results page (SERP). With all due respect... that is a "duh" (obvious) observation.

I do not think you should discount Microsoft's eye-tracking research because you simply do not like them. That is not a very objective, scientific point of view.

I appreciate that you did a lot of work for this test. But I also understand that eye-tracking is tricky, and the mind-eye hypothesis is highly questionable...it is a hypothesis after all.

People pay attention to items with peripheral vision, too. Based on how questions were presented, people look at different areas of the screen. And the query words? "barcelona restaurant" isn't necessarily a transactional keyword phrase. The hours for the Louvre can be found on other sites than the official Louvre website...but the word "Louvre" can also indicate navigational intent.

I'm sorry if I respectfully feel that this research, and conclusions, are overgeneralized in some places and obvious in other places. And some of these conclusions might be incorrect.

Sep 08, 2010
Posted by:
Ani Lopez #30

Dear Shari,
First I would like to mention that I follow your posts and respect you as professional. I would not like you bother for what or how I expressed myself. Said that let me go deep in my comments.

"Many people know users/searchers look at the title in the snippet due to the fact that the title is the hypertext link in the snippet" Fine to me but that is the purpose of these studies, bring some facts and figures to our subjective assumptions.

Some studies are not 100% accurate, some others only applicable under certain circumstances depending how they were made and, although you are 110% right saying they can be misinterpreted, I'll allays put more credibility on facts, figures and evidences.

What I said about Microsoft's eye-tracking research was a bad joke. Those around here know I'm a open-source minded guy. Anyway, any study deserves respect if done to discover the reality besides opinions.

I did a very few work for the test, nothing in fact, just summarize and translate and do not feel offended by your comments but, to be honest, knowing the professional you are I was expecting something more from you than just a "do not misinterpret". Yes, some of these conclusions might be incorrect. Please, bring these errors to the surface, that would be much more helpful.

Now you are explaining better your point of view, don't feel sorry about that at all, I'm sure it is going to be very interesting to the authors to improve next studies in that field.

Ironic, all this Google Instant thing seems is going to turn upside down this particular discussion.
From a UX perspective I hate it but that's another story.

Sep 10, 2010
Posted by:
Mark Simon #31

First of thanks to those who did all the hard work putting together and executing this study. Also, thank you for translating. It is very nice to have data to back up gut feelings and subjective opinions.

Oct 22, 2010
Posted by:
John Alston #32

At User First, we found this study to be a great example of how eye tracking can be used to fill in gaps in knowledge available from other UX techniques such as analytics (getting us closer to Shari's "why"). We weren't as concerned with weighing the implications for SEO ourselves as we are interested in this study as a practical demonstration of eye tracking's unique value. You can read more of our thoughts on our post on our blog below.

http://www.userfirst.com/our-blog/2010/10/15/eye-tracking-study-analyzes-serps/

Ani, we appreciate you translating and summarizing this study for the English-reading world. And, of course, thanks to the actual researchers too!

Oct 22, 2010
Posted by:
Ani Lopez #33

Hi John
Thanks for your kind words, I'm glad you found it useful.
I appreciate any take on this study.

Oct 23, 2010
Posted by:
Henrik Hansen #34

Great study of Mari Carmen Marcos and Cristina González Caro, thanks! And thanks to you, Ani for sharing the "highlights" in english.

I have been studying user-behavior for more than 10 years know, first mostly at the websites and later from a user-holistic point of view, especially search behavior. (And years before "online", of course...)

We have to get closer and closer to the users needs to understand why they do as the do - and why not. It is not good enough to check Bounce Rates/one page/10 sec-behavior on the SERPS (or the website). After 10 sec. the user allready are going on with the task. Therefore - great to see a study that went that close - secounds and millisecound.... :-)

Henrik, Denmark

Oct 23, 2010
Posted by:
Ani Lopez #35

Hi Henrik, thanks for come and comment.
This kind of studies are more and more important every day specially since search engines are going wild lately with so many changes.

Nov 30, 2010
Posted by:
Matthew Hunt #36

Thank-you for doing this! I am very excited to find updated eye tracking information (especially something with Google Maps and other universal search results).

Jan 20, 2011
Posted by:
Dan #37

Some great points with your stats on tracking. I'm just starting to get to that point, but have a bit to go with translating the stats.

Aug 18, 2011
Posted by:
Jason #38

It's amazing to see search change overtime. The eye-tracking information you provided is great and so useful in light of blended search. You almost need to do a study every month as Google continues to change the location of at least one aspect of the SERPs. I would love to see the eye-tracking data compared with the latest Click through study mentioned here - http://www.local-business-marketing-specialists.com/blog/the-latest-greatest-google-click-through-rates-for-the-serps/.
Be interesting to see of we can determine where the nearly 50% of clicks are going that aren't going to the organic results. I think we'd be able to determine some good general percentages given where the eye-tracking data hits.

Thanks again for your post.

Feb 18, 2013
Posted by:
Pete #39

Is this study still a reference in 2013?

Feb 18, 2013
Posted by:
Ani Lopez #40

No idea Pete, most likely yes.

Have your say
Submit
twitter @anilopez

Articles I write for other sites

On Paella and Semantic Markup for recipies

In plain words, it does not work fine most of the cases. It's a bad idea. I'll explain why while I teach you how to cook an authentic Paella.

Analytics Tribulations Of An SEO

The art of measure is never easy but when it comes to SEO it's even worst

Challenges of Spanish Language on Search Marketing @ Multilingual Search

'Standard Spanish' is something that I don’t buy into when it comes to international scenarios. I'll explain to you why and some tips to start facing correctly your Spanish strategy.

Handling Multilingual Sites for Humans & Search Engines @ Bruce Clay Blog

The logic behind the scenes to show all content to bots and the right language to users

Mobile detection issues & Google Instant Previews @ Cardinal Path blog

Mobile web represents the bigger headache ever for those wanting to target the small but growing audience they represent nowadays. check your Instant Previews for possible indexation issues.