featured post

Does the Google Bot index css hidden divs?

The Google Bot loves WordPress

We are testing a theory at my work (Clickstop) to see if the Google Bot indexes css hidden divs.  I have hidden a secret word in a div which I hid via the css syle=”display:hidden” style=”display:none” on the home page.

After a discussion we asked the question “Does the Google Bot index css hidden div’s divs?”

We will see what happens and report back…more to come.

UPDATE:

{2 minutes later…thank you WordPress.}

Simply amazing…Google indexed the site in less that 2 minutes and the results are in.  The Google Bot did index the information in the hidden div – which is why we tested it.  (Seriously – we checked it with 90 seconds and it was already there.)

We use these divs as informational pop-ups to define and display more information about products to users of our site.

Example:  We have a page which shows all of our E-Track products (E-Track tie down kits are used in moving vans, trucks and semi-trailers to help tie down and secure loads.)

We don’t put every bit for information on every page.  If we did – the page would be difficult to navigate and miles long.  So we use information pop-ups.  (Not old school javascript pop-ups – but cool, new jquery/colorbox action.)
[asa]059615707X[/asa]

Conclusion:

Does the Google Bot index css hidden div’s divs?  Yes.

The Google Bot does index hidden div’s on websites.  That being said – you better use relevant content…otherwise Google’s pet Panda will come a knocking and you will find yourself in an SEO death spiral – but it is very good to know that, for useability’s sake, hidden divs can be useful.

I love the Google Bot, I love WordPress and this is the crap that gets me excited.

 

Discussion

29 Responses to “Does the Google Bot index css hidden divs?”

  1. It will be interesting to see if that remains in the index over time or gets removed. Google claim to be very good at finding hidden text etc so it would be good for the experiment to update this after a number of days to see if subsequent crawls remove it.

    Posted by Steve Evans | September 28, 2011, 10:00 am
  2. Why do you like WordPress? Does Google index it well?

    Posted by Rudiger | September 28, 2011, 10:03 am
  3. I tested that a few years ago and googlebot would index nearly instantly after I would visit my own webpage, as they used the “malware prevention service” that most browsers ping when visiting an url, to scan the web. (heck that’s one efficient way to scour the web, much much better than following links :P)

    i’ve now disabled it as i feel it violates my privacy a lot

    Posted by zou | September 28, 2011, 10:07 am
  4. Did you read this article? It indexed my site in less than 1 minute.

    So, to answer your question. Yes.

    Posted by tom.altman | September 28, 2011, 10:16 am
  5. Did you honestly believe that because you changed a style tag Google would suddenly skip that part of the file? It’s only hidden in the browser, it’s not magic.

    I mean damn, Google has been saying for years that if you try to hide keywords in hidden divs that user’s will never see, you’d get in trouble.

    Posted by swearfu | September 28, 2011, 10:21 am
  6. I talked about hiding irrelevant content – and I don’t condone it. It was a question we had about hiding content we know is relevant to that page – but that we don’t want to show to everyone.

    It’s not about trying to “pull one over on Google” – it is a question we had. To see if this supplemental content could be hidden until needed – but yet relevant to the Google Bot.

    I said in the post: “you better use relevant content”

    Posted by tom.altman | September 28, 2011, 10:29 am
  7. It is spelled “divs”.

    You use “div’s” when you are referring to something that involves the div, such as, “The div’s width is…” or “The div’s purpose is to…”.

    I had to get that off my chest ;)

    As you were.

    Posted by Sverri | September 28, 2011, 10:48 am
  8. I will correct…my bad! :) Thank you Sverri!

    Posted by tom.altman | September 28, 2011, 10:49 am
  9. Seriously, you wondered this?

    Here’s what you need to do to find out what Google sees. Use Firefox Developer Tools to disable CSS and JS, and view your site. Everything you see there is what Google indexes. Obviously your “hidden” content won’t be hidden without CSS or JS.

    Posted by Mark | September 28, 2011, 10:56 am
  10. I assumed I knew…but had never tested it. This is a real life test – but more interested, as Steve Evans (comment above) pointed out. How long will it last?

    But seriously – do you think the Google Bot is “dumb”? I don’t think so – they are looking at a lot more than just the code left sans css and js.

    Posted by tom.altman | September 28, 2011, 10:58 am
  11. Hi Tom,

    `display:hidden` isn’t valid CSS (and doesn’t hide anything) .. maybe you used something different?

    I’d be interested in … hiding with `display:none` and `visibility:hidden` both within inline styles and via a stylesheet. Seeing if those results are any different.

    Posted by Paul Irish | September 28, 2011, 10:58 am
  12. I’m sorry – you are correct.

    I used: <div style=”display:none”>

    Thank you Paul for catching that error.

    Posted by tom.altman | September 28, 2011, 11:00 am
  13. François, I’m not advocating this for results SPAM purposes – I’m just saying this is possible and showing ways to improve usability. Like the example I gave above.

    Posted by tom.altman | September 28, 2011, 3:11 pm
  14. I’m sorry but how is this news? Even back when I was a teenager everyone and their brother knew this about google.

    Posted by Bill | September 28, 2011, 4:19 pm
  15. An HTTP Get request returns all of a page’s HTML before your browser parses any instructions in associated JS or CSS files (or inline instructions). In essence, your browser says “Show me the file so I can decide whether I should hide some of it.” It shouldn’t surprise you that a special-purpose browser like the Googlebot can ignore presentational ephemera and index everything.

    Posted by Bob Sutton | September 28, 2011, 6:12 pm
  16. “How is this news?”, “You didn’t know that?”, “Obviously”, “Everybody knew that” … Are you guys serious??

    Google changes algorithms *all the time* both secretly and publicly in a global cat & mouse game with spammers, scammers, link farms, black hats, and regular honest folk.

    I’m left assuming all these obnoxious comments are from a bunch of halfwit PHP coding creationists with no concept of the scientific method.

    Thanks for testing this and writing up and sharing your results. Ignore the haters.

    Posted by Manny Hernandez | September 28, 2011, 6:25 pm
  17. I think I love you Manny. :)

    Posted by tom.altman | September 28, 2011, 6:27 pm
  18. Two related tips!

    FIRST: Note that if you explicitly want to hide content visually but have it show up for both search engines and disabled (blind) users, the recommended technique is to position it off-screen, to the left.

    .screen_reader {
    position:absolute;
    left:-10000px;
    top:auto;
    width:1px;
    height:1px;
    overflow:hidden;
    }

    See also: WebAIM’s discussion of this technique. http://webaim.org/techniques/css/invisiblecontent/#techniques

    SECOND: If you want to include content in a page *WITHOUT* showing it to any screen readers or search engines, you can place it in a SCRIPT tag with a ‘type’ attribute of ‘text/html’. (This makes sense for things like error messages, especially.) Then you can grab the content via e.g. `.innerHTML` and place it in your lightbox or use it for templating, etc. See also: ICanHaz.js.

    Posted by Alan Hogan | September 28, 2011, 6:56 pm
  19. Sorry the formatting in that post is poor. Not sure the best way to format comments here.

    Posted by Alan Hogan | September 28, 2011, 6:56 pm
  20. No problem – thanks for sharing.

    Posted by tom.altman | September 28, 2011, 6:58 pm
  21. I noticed this and use Ajax to load any hidden info I want to pop up, cycle, etc. Even if it preloads as soon as the page loads, as long as it’s a separate request it’s not indexed with the rest of the page.

    Posted by Eric Mason | September 28, 2011, 8:23 pm
  22. Thank you for sharing this useful research. I’ve been wondering about for quite a long time

    Posted by Tobias Wagner | May 4, 2012, 1:40 pm
  23. thanks , intresting

    Posted by roritm | July 10, 2012, 8:21 pm
  24. People are such jerks sometimes. Your post was very informative and and helpful to me, as were SOME of the comments.

    I, like some, believed this to be the case all along, but I’m thankful you shared your findings because it proved (at least at the time of your test) that the content is indexed. I use hidden DIVs all the time for relevant content, but content that would clutter the page if displayed by default. This gives me confidence that those DIVs are in fact inexed.

    Alan’s comment on positioning the content off the page is interesting. Are you saying this gives you a better chance of it being indexed, or that it gives the content more relevance than if the content was hidden (display:none, visibility:none)?

    Posted by Troy | August 11, 2012, 9:40 am
  25. “Alan’s comment on positioning the content off the page is interesting. Are you saying this gives you a better chance of it being indexed, or that it gives the content more relevance than if the content was hidden (display:none, visibility:none)?”

    What does “off page” mean – if you can see it in the source code it is seen by the Google bot – so its really irrelevant. I’m not sure Google cares either way.

    A good addition to this test would be to have a display:none vs. visibility:none – I’ll add that to the list.

    Thanks for the comment Troy.
    tom

    Posted by tom | August 11, 2012, 9:57 am
  26. This should do the trick :
    .hide-from-user {left: -9999px;
    position:absolute;}

    Posted by steve chan | September 21, 2012, 9:58 am
  27. I’m not sure that is a good idea steve. We would need to assume Google doesn’t care about the -9999 positioning.

    Maybe a risk – maybe something we should test. :)

    Posted by tom | September 21, 2012, 10:18 am

Trackbacks/Pingbacks

  1. [...] A: add HTML commented links on all pages of the site.  (view source on a Google bot css hidden div page to see these [...]

Post a comment