Supplemental/omitted results - Google
I'm still trying to come to terms with "omitted" results in Google and how important (or unimportant) it actually is.
Google is doing some odd stuff at the moment - for several of my sites if I type in the full www url the search comes back with some real old content and page titles. The pages and titles concerned have often been defunct for several months and yet G is still retrieving this. The current pages (as retrieved up until recently) are now either not showing at all or show in the "omitted results" index. It's almost as though G has lost its deep crawl information and reverted to old data whilst it is thinking what to do next - could this be part of the "Big Daddy" infrastructure roll-out?
What is noticeable is that the pages in the "omitted" results section all have similar meta descriptions - the meta is showing in the returned results and not the deep crawl data as retrieved up until recently. Odd.
I have also seen a few reports of "bad data push" generated during recent Google crawls - is it possible that something has gone awry during recent updates and attempts to roll our new search architecture? Have they had to resort to rolling back historical data? The apparent loss of deep crawl data for several of my sites, a bunch of pages dropping into the omitted results section and real old page titles reappearing might support this.
I might be accused of trying to over analyse some of this but for sure something's up at Google Towers.
For now I'll be revisiting some of the affected sites and making the meta descriptions on key pages unique. I'm also off to Italy for a couple of weeks so maybe things will have reverted to "normal" on my return. Bloody search engines - not my favourite topic right now but nothing a few glasses of Chianti won't cure 
Dave
Comment by ray — July 15, 2006 @ 2:37 am
Burt,
You are on to something because I have been experiencing some problems with Google also. My XsitePro niche sites were getting crawled just about everyday by Google, but now it has stopped or the spider only visits once in a while. I don't think I 've been banned as I don't practice any blackhat or use dupe content exclusively.
I hope they straighten it out soon.
Regards
Comment by James — July 15, 2006 @ 6:32 am
Over the last week, my sites have gone to index page only, fully indexed, and not indexed at Google. Sometimes with www sometimes not. It seems to change nearly every day this week. Fun.
Comment by Dave — July 15, 2006 @ 6:41 am
Ahh, I don't care what it's doing. Can't really do anything about it anyways. I won't lose sleep over it.
I'm a little miffed that my blog has shown no signs of pagerank movement though. Not that it matters.
Oh, and it was Dave(not me, the other Dave) that posted, not Burt.
Comment by sarahG — July 15, 2006 @ 1:34 pm
Google definitely still has problems. I've watched a 2000+ page site go from ranked to just 1 page back to the 2000 odd, with cache dates of 2005.
The omitted results issue is possibly down to a similar description, be it the meta or what google has chosen (it can come from the ODP too). If it's easy enough, get a unique meta description on each page, that will get them out of the omitted status.
As for being in it one day and dropped the next… Google just still seems to be having issues. I've found it's easier to get a new site indexed that an old site sorted out! Plus remember to do your checks on the same Google IP so that the results are an accurate comparison.
But yes, Google is still a bit temperamental!
Comment by ray — July 15, 2006 @ 2:58 pm
Oops, sorry about that Dave, " not the other Dave", I did notice your name at the bottom of the post.
Regards
Comment by uKnowWho — July 15, 2006 @ 7:11 pm
Ever since BD came into being, they've seemingly been doing roll-back after roll-back. I suppose it's to be expected when rolling-out a new massive infrastructure.
It also seems Google is still having [URL=http://www.webmasterworld.com/forum30/35058.htm]problems with canonicals and 301 redirects[/URL] which might be worth looking at in your own sites (everyone).