Get exclusive CAP network offers from top brands

View CAP Offers

WH robots.txt

asked 4 years ago
Check this out. It’s kind of interesting.
http://www.whitehouse.gov/robots.txt
2 Answers
Dominique answered 4 years ago
Hmm, lots of stuff they don’t want spidered.

Lookis like all of their Iraq stuff isn’t going to be spidered.

bart3k answered 4 years ago
Interesting indeed. But why would they want to put all that stuff up on the web in the first place if they don’t want people reading it?

Bart