June

02

by Kaj Kandler

I can’t believe what I just found on the Yahoo!Search Blog about removing pages from a website. The author says “The best way to remove dead URLs from the Yahoo! Search index is to return an HTTP Error 404 when our crawler requests the page.”

Are they serious, really serious?

The HTTP spec clearly says return code 404 is “Not Found” temporarily and 410 is “Gone” permanently. They even say in th explanation for code 404 “The 410 (Gone) status code SHOULD be used if the server knows, through some internally configurable mechanism, that an old resource is permanently unavailable and has no forwarding address.”

Yahoo slurp is free to treat a 404 page as if removed although I don’t think it serves the searching public well. However, I can’t understand why the Yahoo!Search blog teaches webmasters to send a 404 if a 410 return code is appropriate.

Just needed to rant about this, because this blog has for sure a wide readership.

One Response to “Webmaster’s rant”

  1. Don't listen to the Yahoo!Search Blog - WebProWorld Says:

    […] use HTML return code 404 (Not Found) for deleted pages. Why do they teach the wrong information? The correct answer would be return code 410 (Gone). Check the W3C specification if you don’t believe me. K<o> __________________ Technical […]

Leave a Reply

You must be logged in to post a comment.