Unscrupulous SEO

By Peter Cornish | July 31st, 2016

Some time ago I wrote an exposѐ on an unscrupulous SEO agency I stumbled over and this might be another – or maybe they are just incompetent. You decide.

No-one can find my site

In this case the client had complained to the digital agency that created their web site that no-one could find it in Google, even though it had been many, many months since the site had been launched.

The client tells me the agency investigated, then responded with a proposal for SEO services ranging $500 to $1000 per month. The frustrated client asked a few friends for their opinion, and this is how I came to be involved.

Testing your site's Googlism

I used simple techniques described in Steps 1 and 2 in my SEO Website Review Wizard to determine the site’s ‘Google state’.

The site was completely ‘unknown’ to Google. It didn’t matter what anybody searched – the site would not appear because Google didn’t know the site existed in the first place.

Blocked by robots.txt

Digging deeper I noticed the site’s robots.txt was blocking all search engines. This important website control file tells search engines how they should treat the site, and in this case it was saying ‘ignore me!’

You can see your robots.txt (if one exists) by typing your website address, then /robots.txt
eg: your-web-address/robots.txt

If it looks like the one below, you’ve got a problem, and that’s exactly what happened in this case!

User-agent *
Disallow: /

Blocking a WordPress site

In WordPress this robots.txt setting can be activated through the website dashboard user at:

Dashboard | Settings | Reading

Tick the ‘Search Engine Visibility’ box and WordPress updates the robots.txt to ‘disallow’.

Google and all other search engines will then discard any record of your website leaving it invisible on the ‘net’.

SEO problems miraculously fixed themselves

Miraculously, after I requested access to the client’s site, the robots.txt disallow spontaneously disappeared.

I undertook a couple of other SEO related activities, like registering the site with Google Search Console, submitted an XML Sitemap etc. In a couple of weeks the site will be fully ‘indexed’ by Google and feature in search results.

Robots.txt is a powerful file for SEO, but when misused either maliciously or through incompetence can create real problems.

I'll let you decide if the SEO Agency in question was incompetent or malicious.