Squeezing the most out of your network

If Voltron taught us anything at all, it was that there's power in numbers. Watching TV as a kid and seeing those lions combined to make Voltron and defend the universe taught me two things:

1. The sum is greater than the parts
2. Nothing trumps a combination of five amorphic robotic lions
   

Now a days, I don't much travel in packs, let alone packs of lions, but the first lesson still applies to lots of stuff I encounter in my day-to-day. More to the point, it applies perfectly to the practice of SEO. Many of you have created networks to achieve whatever ends you are after, whether its arbitrage, dominating a niche, or just plain old splog networks. For the sake of this post, let say we have created a network of sites to target a specific niche. How about Christian Religion (you can find some free goodies for this niche in the download section, BTW)

Now, you've spent some considerable time building a great network that addresses every level of your niche. If you're targeting the Christian Religion niche, off the top of my head I'd say you'll have at least 5 fully developed sites in your network. Remember, you want to target as many niche segments as possible so you can rank for a large handful of keywords and keyword phrases. I am not going to delve too deep into setting up your network and how to position and interlink your individual sites. If you want to read a really great post on that subject, check out Blue Hat Seo. For the sake of this post, I want to address how to squeeze the most interlinking (and thus indexing) power out of your network.

By this point, you've probably SEO'd the living hell out of each site in your network. Hopefully you'll have set up a keyword rich mod rewrite scheme for all your sites. And you'll also have set up individual site maps for each site, and have interlinked all the landing pages of each site and even linked across sites in your network. Well if you have covered all the steps just mentioned plus some of the more common sense techniques, you've probably set yourself up a nice little network. The thing is, while you've got 5 excellent little lions purring away, they aren't really making a Voltron, you follow? Sure you'll have inter-linked the sites in your network, but there's allot more you can do in terms using your network to its maximum potential.

If you're using the Bible database listed in the downloads section, you've already got a site with at least a couple thousand pages. Depending on what else is in your network (a forum?) you've probably got more than one large site. In this niche, subject coverage is pretty easy to achieve, but getting all those pages - all that coverage - indexed can be tricky. Here's my suggestion on how to achieve maximum indexing power for your network.

We may have a great inner linking scheme, but there are always going to be large amounts of pages that just slip through the cracks if you don't cover your bases. What we need is a way to get the sum total of all of our pages on all of our sites in the entire network and spit them out in a nice list (or lists). Why? I'll get to that.

First, I want to tell you that Google has a handy little Sitemap Generator script. Its an open source python script that you place on your server and configure it to crawl your folders, index their contents, and output pretty little site maps that you can point search engine's too. Truth is, this little script should already be a cron job in all your networks. This is a great tool that should be part of your repertoire, but out-of-the-box, it doesn't precisely do what we need it to do, which is spit out a list of all the urls in our website. I have created a modified version of this script to crawl your websites log files and wrap each url in a <loc> tag for easy extraction later. It is currently configured to crawl your LOG files because this is effective for dynamic sites (yet has its limitations, say if certain parts of your network get zero traffic). You can reconfigure the script as you see fit. I'll post the download link at the end of this post.

I am not going to go into depth about how to use this script, as it is used EXACTLY the same way you use the original scripts as-is from Google. So if you have any problems or questions in that regard, refer to Google.

So what we have now is a modified version of the Google site map script on a cron job for each of our websites in our network. This little cron job will chug away constantly updating a .xml file for each website that can easily be called by a php script on other websites in our network. If you did things right and set up a keyword rich mod-rewrite scheme for your urls, then not only do you have a list of every URL in your network, but you'll also have its respective keywords!

From this point, it's an easy task to go about creating a php function that calls out other sites in your network, pulls the link out of the xml and displays it in the context of your current website.

So there you go! You've just gotten one step closer to harnessing the power of Voltron! I say 'one-step' because not only can you take this technique alot further than I am letting on, but there's a few more techniques to squeezing the most out of your network that I'd like to address in later posts.

With that said, I'm off to go download vintage cartoons.

Download the log scraper here