LHMark
Posts: 7048
Joined: Fri Jan 21, 2000 2:18 am

How Do I Hide A Website From Google?

Thu Aug 03, 2006 5:02 am

I have just built a website for a once very high-profile band. It's the media and contacts guide for their upcoming tour, and it's accessed as a subdirectory off their main URL (www.[bandsname].com/media).

The problem is, the band's fans are googling the band name and finding the media site which contains direct contact information, financials, and tour specifics. How can I stop this from happening?
"Sympathy is something that shouldn't be bestowed on the Yankees. Apparently it angers them." - Bob Feller
 
EWS
Posts: 3369
Joined: Tue Mar 07, 2006 6:41 am

RE: How Do I Hide A Website From Google?

Thu Aug 03, 2006 5:03 am

Quoting LHMARK (Thread starter):
The problem is, the band's fans are googling the band name and finding the media site which contains direct contact information, financials, and tour specifics. How can I stop this from happening?

Password protect the /media directory and give access to those only who need it?

Lew
 
LHMark
Posts: 7048
Joined: Fri Jan 21, 2000 2:18 am

RE: How Do I Hide A Website From Google?

Thu Aug 03, 2006 5:10 am

Can't. We've got to make it accessible to journalists all over the world, not all of whom we'll have contact with.
"Sympathy is something that shouldn't be bestowed on the Yankees. Apparently it angers them." - Bob Feller
 
trekster
Posts: 4319
Joined: Fri Dec 26, 2003 2:47 am

RE: How Do I Hide A Website From Google?

Thu Aug 03, 2006 5:14 am

how about put ur own ctc details on the site, and the info in the site create a pdf document then email it once a request is received
Where does the time go???
 
airbusA346
Posts: 7284
Joined: Fri Dec 10, 2004 7:05 am

RE: How Do I Hide A Website From Google?

Thu Aug 03, 2006 5:20 am

Quoting Trekster (Reply 3):

That is a very good Idea, and then the information wouldn't end up in anybodys hands.

Tom.
Tom Walker '086' First Officer of a A318/A319 for Air Lambert - Hours Flown: 17 hour 05 minutes (last updated 24/12/05).
 
trekster
Posts: 4319
Joined: Fri Dec 26, 2003 2:47 am

RE: How Do I Hide A Website From Google?

Thu Aug 03, 2006 5:21 am

Exactly.

Someone would need to keep up to date with all the emails though.

How about sending the PDF to all the major news agencys, might work as well
Where does the time go???
 
LHMark
Posts: 7048
Joined: Fri Jan 21, 2000 2:18 am

RE: How Do I Hide A Website From Google?

Thu Aug 03, 2006 5:27 am

I think I just found a meta tag that'll do the job [meta name="robots" content="none"]. Can anyone tell me if that will work?
"Sympathy is something that shouldn't be bestowed on the Yankees. Apparently it angers them." - Bob Feller
 
AeroWesty
Posts: 19551
Joined: Sat Oct 30, 2004 7:37 am

RE: How Do I Hide A Website From Google?

Thu Aug 03, 2006 5:42 am

In the upper level folder, create: robots.txt

Type in the text document and save -

User-agent: *
Disallow: /
International Homo of Mystery
 
diamond
Posts: 3000
Joined: Sun Apr 11, 2004 8:01 am

RE: How Do I Hide A Website From Google?

Thu Aug 03, 2006 5:44 am

Isn't this the sort of thing that "Aircraft" specializes in?
Blank.
 
DrDeke
Posts: 805
Joined: Sat Jun 11, 2005 7:13 am

RE: How Do I Hide A Website From Google?

Thu Aug 03, 2006 5:44 am

Quoting AeroWesty (Reply 7):
In the upper level folder, create: robots.txt

Type in the text document and save -

User-agent: *
Disallow: /

 checkmark 

This will prevent all well-behaved search engines from indexing your site.

-DrDeke
If you don't want it known, don't say it on a phone.
 
Zudnic
Posts: 59
Joined: Sun Oct 23, 2005 12:42 am

RE: How Do I Hide A Website From Google?

Thu Aug 03, 2006 6:51 am

Quoting AeroWesty (Reply 7):
In the upper level folder, create: robots.txt

Type in the text document and save -

User-agent: *
Disallow: /

Correct answer. Works like a charm.

However, it may be too late. You should put the directory you want to disallow on robots.txt BEFORE uploading the content.

Finally, it sounds like you have some rabid fans out there. If I were you I would password protect your content. Not sure if there are any hosting services out there that would do it for you, but it's a relatively simple Perl script to ask for a password then set an access cookie.

Edited: Content, clarity.

[Edited 2006-08-02 23:55:10]
 
NoUFO
Posts: 7397
Joined: Tue Apr 17, 2001 7:40 am

RE: How Do I Hide A Website From Google?

Thu Aug 03, 2006 7:19 am

Quoting AeroWesty (Reply 7):
Type in the text document and save -

User-agent: *
Disallow: /

Better yet:

User-agent: *
Disallow: /media/

to keep spiders from indexing the media folder - as long as other content can be listed.
I support the right to arm bears
 
NoUFO
Posts: 7397
Joined: Tue Apr 17, 2001 7:40 am

RE: How Do I Hide A Website From Google?

Thu Aug 03, 2006 7:29 am

Quoting LHMARK (Thread starter):
The problem is, the band's fans are googling the band name and finding the media site which contains direct contact information, financials, and tour specifics. How can I stop this from happening?

You definitely need to protect the folder with a password!!

As an interims measure, embed a flash movie where visitors need to enter a password first before proceeding AND use the AS Obfuscator to protect the code. Yes, I can be of assistance here.

Edit: Oh well, that way you could protect the folder itself. But no password = no protection, really.

Quoting LHMARK (Reply 2):
Can't. We've got to make it accessible to journalists all over the world, not all of whom we'll have contact with.

How so if none of the journalists knows of this little media-folder?
From what I know, journalists have to ask the management for additional information and then receive either a link with some log-in data or a CD-ROM.

[Edited 2006-08-03 00:32:29]
I support the right to arm bears
 
aircraft
Posts: 266
Joined: Mon Jan 15, 2007 9:18 am

RE: How Do I Hide A Website From Google?

Thu Aug 03, 2006 9:50 am

Quoting AeroWesty (Reply 7):
In the upper level folder, create: robots.txt

Type in the text document and save -

User-agent: *
Disallow: /

 checkmark 

There are many other options you can mess with, but this is the quickest and easiest way.

If you want to restrict any current traffic that comes from search engines, you can make use of the Environment Variables of the web server. Grab the HTTP_REFERER (it's actually spelled 'referer, not 'referrer' in this case). If the HTTP_REFERER variable contains a url containing google.com, msn.com, ask.com, askjeeves.com, yahoo.com, etc., then you can take measures to prevent the page from showing, or you can choose to redirect the traffic elsewhere.

Send me a note if you need clarification.

Quoting Diamond (Reply 8):
Isn't this the sort of thing that "Aircraft" specializes in?

Yep  Smile
 
Klaus
Posts: 20622
Joined: Wed Jul 11, 2001 7:41 am

RE: How Do I Hide A Website From Google?

Fri Aug 04, 2006 1:55 am

Simply keeping the sub-directory un-linked should do the trick already - disallow directory listing (should be standard anyway) and mail out the link but never place it on any link-reachable page.

Google et al "crawl" along existing links; If your pages are only linked among themselves, the crawlers shouldn't find them. You might want to refrain from using Google Mail to e-mail the link to be certain, however...

Who is online

Users browsing this forum: No registered users and 4 guests