How Do Search Engines And Web Crawlers Work

It is the search engines that finally bring your website to the notice of the prospective customers. Hence it is better to know how these search engines actually work and how they present information to the customer initiating a search.

There are basically two types of search engines. The first is by robots called crawlers or spiders.

Search Engines use spiders to index websites. When you submit your website pages to a search engine by completing their required submission page, the search engine spider will index your entire site. A ‘spider’ is an automated program that is run by the search engine system. Spider visits a web site, read the content on the actual site, the site’s Meta tags and also follow the links that the site connects. The spider then returns all that information back to a central depository, where the data is indexed. It will visit each link you have on your website and index those sites as well. Some spiders will only index a certain number of pages on your site, so don’t create a site with 500 pages!

The spider will periodically return to the sites to check for any information that has changed. The frequency with which this happens is determined by the moderators of the search engine.

A spider is almost like a book where it contains the table of contents, the actual content and the links and references for all the websites it finds during its search, and it may index up to a million pages a day.

Example: Excite, Lycos, AltaVista and Google.

When you ask a search engine to locate information, it is actually searching through the index which it has created and not actually searching the Web. Different search engines produce different rankings because not every search engine uses the same algorithm to search through the indices.

One of the things that a search engine algorithm scans for is the frequency and location of keywords on a web page, but it can also detect artificial keyword stuffing or spamdexing. Then the algorithms analyze the way that pages link to other pages in the Web. By checking how pages link to each other, an engine can both determine what a page is about, if the keywords of the linked pages are similar to the keywords on the original page.

20 Analytics Tools For Blogs

There are a lot of data points that can be meaningful for tracking blog effectiveness. That is, tracking what happens when visitors arrive at and engage with your blog content. It really comes down to the purpose of your blog. Metrics for a blog that’s focused on making a web site more search engine friendly by adding crawlable content and attracting links is quite different than a blog that’s meant to build thought leadership or brand credibility.

Many of the metrics tools used for blogs are also used for basic web site analytics. That makes sense because many blog initiatives do not have the same kind of budget as web site marketing programs do. Therefore, the analytics employed tend to be low(er) or no cost.

Regardless of the purpose, I’ve assembled a list below of the various tools we use, or have tested to report onsite blog metrics. Pick the service or tool you like the most from the list below or something new for your unique purpose and please share in the comments. The list is in no particular order.

  • 103bees - Free web stats (ad supported) up to 100k visits per month, then it’s $9
  • Enquisite - Free, extremely detailed web stats
  • Hittail - Provides suggested topics for your blog by keywords used in referral traffic
  • Crazy Egg - Provides overlay, list and heat map web stats
  • RobotReplay - Lets you record visitor actions on your site and play them back
  • Clicky - Web stats plus feed and Feedburner stats
  • Google Analytics - Web stats, not really the best for blogs but it’s free
  • StatCounter - Free web stats
  • Co.mments - Track comment threads starting on your blog and follow them elsewhere in a feed
  • Blog Tracker - Free from IceRocket but limited functionality
  • Performancing Metrics - Basic is Free, or $3.99 to $16.99 per month for more features
  • Site Meter - Basic Free and Premium versions $6.95 and up
  • Mint - Popular web stats with bloggers for $30 per site
  • MyBlogLog - Basic blog visitor stats and social networking. Free and paid versions.
  • Feedburner Stats - StandardStats Free, TotalStats $4.99/mo
  • Wordpress Stats - Free basic blog stats plugin for Wordpress blogs
  • Google Analytics and Feedburner Stats - Free plugin for Wordpress blogs
  • eXTReMe Tracking - Free web stats with a paid version for $4.50 per month
  • Web Stat - Many web stats features for $5/mo
  • TraceWatch - Free but you need access to your server which should be running PHP/MySQL

What are your favorite analytics tools for blogs?

MSN Optimization

Google and Yahoo , the most popular search engines uses almost same algorithm to display search result. Micorosoft search engine uses different algorithm to display search result. To optimize for MSN search engine, note the following points :

1) Regularly update your website using fresh content.
2) Check the tags used in the webpages. The tag should properly enclosed.
3) Check all the links added on the pages. All links properly work.
4) Avoid keyword stuffing. Also donot include irrelevant keyword to target the page.
5) Alt tag must be relevant name to the image and no keyword stuffing in alt.
6) No hidden links or text. Links or text should visible to users.
7) Donot purchase links from link farms. Directory submission or article distribution are beneficial to increase links.
8) If possible include 1-2 lines text at the top of all the important pages. Add relevent keyword phrases.
9) Use links of some internal pages in the bottom of the page related to the page. These links will work as keyword for the page and help to increase keyword density of the page.
10) Include the text in title and description used in heading tag.
11) Choose relevant name for the pages. If the page contains the information on rates of hotels then file name should be "hotels-rate.html" not "sample.html" or "5674.html".

You cannot impress MSN to collect lots of inbound links. You must have fresh and unique content including target keyword inside the content.

Read the MSN guidelines.

Beginners guide to social media strategies


Stage 1 - Research

The first stage of a successful social media strategy should involve identifying industry related areas of the web that will provide opportunities to create a buzz online for your clients industry. These include looking at niche social bookmarking sites, blogs, forums and social networking sites. You should have records kept on file of what sites work well for which industries, this will save you time in the long run, this file will eventually get very large (trust me!).

Stage 2 – Registrations

Once sites have been identified that meet the projects criteria the long registration process entering clients details in profiles takes place. This will usually involve entering a “homepage” URL, company logo and a brief summary of what the company’s goals are. Any additional promotional material would also be useful to help build up the brand within the different communities.

Stage 3 – Building trust

The key to creating a successful social media strategy is the relationships you build and the trust that you earn. When suggesting sites or information it should not immediately be the clients site or link bait, it should consist of industry related material. Commenting on related blogs, starting forum threads about the industry should also be created. This should be maintained for a period of time before moving on to the next stage. You should take a genuine interest what you comment on and write about, no-one likes a fibber!

Stage 4 – Release link bait

Once there is a level of trust associated with the company profile that is the time when the link bait or urls should be posted. Using this method will ensure that the information that is posted will not be labeled as spam by the various site administrators or site owners. By this time the profiles created will already hold some weight within the various communities which will ensure a level of traffic.

Stage 5 – Generate traffic + links

Once the urls and link bait have been posted this should immediately create incoming traffic to the client’s site or landing page. The secondary goal of such a strategy or possibly primary goal depending on client will be the natural links generated as a result of such a strategy. If like minded users find the link bait interesting then by all accounts they should then bookmark the link on their own social media site, be it a blog, social bookmarking site or forum. This will additionally create more incoming traffic to the site and generate a number of links that will help with the clients search engine positions for selected keywords.

Stages 3, 4 and 5 should be maintained over a longer period of time to keep impacting the search engine results positions. If this is not repeated throughout an extended time period then the increase in search engine positions will only be temporary.

Need Of Validating the XHTML/HTML and CSS Codes


Defining Validation

If you are unfamiliar with Validation then you may about it by going through the definition:
Validation is process of refinement of HTML or CSS code and assurance to that coding is free from errors and making it as that it can meet the standards set by the W3 Consortium.

There are various types of validators used in which some may check only errors, others may suggest you about your code and tell what the way of writing code is.

Need Of validation:

There are many persons who are against the validation because of some of the reasons as:

* Validation does not always guarantee that page will work.
* It is time consuming process, because making every pages error free consumes much time.
* An average visitor does not see your page source code.

But, there are some reasons for adopting the validation of HTML and CSS codes some of them are:

1. It Helps in Resolving Compatibility Issues

There are various types of browsers used in the industry and there is a huge scope for new software’s and tools. So, the code should correct and compatible for all of the browsers and should be updated with technology and software coming in next era.

Validation of the XHTML, HTML and CSS codes does not always ensure the compatibility to the browsers but it propels you towards right direction. If you are still reviewing your files in multiple browsers but, since I am here dealing only with structure, the browser compatibility issue will not rear its ugly head as it will with CSS.

2. Increased Search Engine Visibility:

Google attract towards the well formed and well structured site. Here we may talk about the search engine spiders, they crawl through web and index the pages of your site and show visibility of your site in the search engines. So, you must use those attributes which may easily crawled by the search engine spiders. Use codes and content for your site as much possible. If you are using Image anywhere must use the title of image and tags in code will make it easy to crawl. So, the error free codes are important for your site for increasing visibility to search engines.

3. Learn Think and Apply:

Once you starting validating your files you’ll find the process to be an invaluable learning experience. Validation itself will teach you what the best ways to create your XHTML/HTML and CSS codes are. The more you use validation, the faster you’ll be able to create well-formed documents because you won’t be making the same mistakes over and over again.

YouTube Launched The Mobile Version

YouTube launched the mobile version of their site today. Here are all the details.

The first thing I saw as I navigated to the site using my mobile was the following informational warning message:

Information: YouTube Mobile is a data intensive application. We highly recommend that you upgrade to an unlimited data plan with your mobile service provider to avoid additional charges.

The mobile interface is very compact and in fact more limited (though for obvious reasons) than I had expected - there is no way to 'directly' upload videos taken from your mobile phone to your YouTube profile, or in fact even sign in to your profile for whatever purpose. While this is a concern, it isn't one that will affect a large pool of people since most of YouTube users just watch and a very small percentage actually uploads videos. Even from those that do upload not many of the videos are taken from mobile phones. (You can still use MMS to mail videos to have them uploaded to your account).

For just viewing videos, however, the site works just fine. The recently added videos are displayed at the very top and clicking on menu takes you to a list of options including featured, most viewed, top rated, and top favorites.

msaleem_mtube1.png

msaleem_mtube2.png

The FAQ points out: "You need a handset that supports streaming video (RTSP/3GP with H263/AMR)." Considering the increasing importance of the mobile marketplace and the mobile page views this is a logical though limited first step for YouTube. The move is a premature because of the lack of a large number of people with the appropriate data plan and the mobile handset compatibility requirements.

Making MySpace Effective

Have you ever seen a company's MySpace profile? I have noticed that more and more large, medium and small businesses are creating profiles in an effort to leverage the MySpace community. It seems like most of them think that by popping up a decently designed profile and adding thousands of friends their business will grow to a whole new level, but the problem is that this in itself is not too effective.

Creating a nice profile with thousands of friends may help with branding, but in most cases it does not work because business objectives and goals are not kept in mind when creating these profiles. If your overall goal is to increase your brand's recognition make sure you have your logo all over the profile as well a call to action that draws people to your website. If your goal is to appeal / market to these MySpace users make sure you promote to all of your friends using the bulletin feature (without spamming them). If your goal is to promote a band just don't add their music to the profile, also make sure you add a buy button so people can buy the CD online with ease.

A MySpace profile can be an effective means to reach your target audience, as long as you have some clear objectives in mind. Don't just design a nice profile and start adding tons of friends. Make sure you keep your goals in mind and design your profile accordingly.

Google Soon to Recognize Text in Images?

Information Week has reported that in June of 07 Google filed a patent application, which has just become available, outlining a “method of optical character recognition in digital images.” The application seems to cover both static images as well as video. The ability to do this could radically change a number of existing Google services as well as again change the way the Internet marketing world interacts with images and video.

Being able to identify text in images and video would be an extremely large leap in search engine indexing technology. Being able to index the content of videos and other new social media would most likely either force a change in the way search engines have to build their SERPs or provide opportunities to build highly relevant image and video search functions that could generate significant relevancy from the text content within the media.

Clearly this would not be a complete solution for new service development, as many images and video do not contain text to index.

Meshly Mashes Twitter and Digg

Meshly is another platform for publishing quick posts to your 'space' via instant messages. The service is based on a very easy three-step approach that ultimately combines features from both Twitter and socially driven sites like Digg.

What I immediately liked about the service was the ability to sign up, change your settings, and post content, all through your choice of instant messaging client (though some may find the lack of browser or mobile-platform support a little restricting). The entire process of using Meshly is exactly as easy as it is described on their website.

msaleem_meshly1.png

msaleem_meshly2.png

msaleem_meshly3.png

Using Meshly is like you are talking to an actual person on the other end who is helping you use a number of easy-to-follow prompts and a list of yes or no questions. Once your account has been created you are given a simple, three-option menu with all the commands that the Meshly IM bot understands.

msaleem_meshly5.png

Typing in 'help' at any point brings you back to this menu, 'post' walks you through the steps to make a post on your Meshly space, and 'settings' allows you to change your password.

While I really enjoyed the IM-based content posting, Twitter has had that feature since last October. Furthermore, as far as the socially driven aspect of Meshly is concerned, that space is already so saturated that I don't feel Meshly will be able to really get a strong foothold. In the end, though all the components (the IM-feature, Twitter like posting, and Digg-like social aspects) are great, the whole of Meshly is not greater than the sum of its parts.

Adsense Getting Start Tips

AdSense is the program that allows publishers to put Google Ads on their own sites - and generate revenue from them. It’s popular because it’s so easy to do and because the ads are context sensitive. Whatever the subject of your web site, Google will crawl it and automatically choose ads that match. And with thousands upon thousands of advertisers, the chances are you’ll get relevant ads on your site.

Getting AdSense up and running is fairly easy - just paste the Javascript code into your blog template - but if you want to make the most of Google ads on your site, here’s a few quick tips:

Placement

The placement of ads on your page (where you put them) is possibly the most important factor if you want to get maximum revenue from AdSense. Try to place ads “above the fold” - i.e. near the top of the page, so a reader doesn’t have to scroll down in order to see them. If you’re ads are right at the bottom of the page, nobody will ever see them, never mind click them.

Ad Sizes

Different sized ads can perform very differently, so it’s worth experimenting with different sizes to see which works for you. Many people find large leaderboard ads at the top of a page, or 300×250 ads embedded in the content perform best.

But you also need to consider how the ads fit into your blog template. If an ad won’t physically fit, try a different size or placement. You might even find that you need to try a completely new blog template in order to fit ads in without detracting from the content.

Colours

You might not think it, but the colour of your ads can have a huge impact on your revenue. Most people find that colours that blend in with your own site work best. In particular, look at removing borders (by setting them to the same colour as the background).

Change one thing at a time

When tweaking your ads, rather than changing ad size, placement and colour all at once, change one thing at a time and wait at least a day to see how that change affects your stats. This is by far the best way of seeing how a particular change is affecting your revenue, and you won’t have to spend time figuring out what it was you need to put back if your earnings suddenly fall through the floor.

Use channels

Google provides custom “channels” that enable you to track ads on specific pages or placements to see how they perform. This is really useful if you want to see how one ad is performing in relation to another, and it’s absolutely vital if you have more than one site. Spend some time setting up channels and getting a feel for how they work, and you could end up saving yourself a lot of time and money in the future.

Experiment with image ads

For some ad sizes, Google lets you specify if you want to see text ads, image ads, or both. Often, letting Google’s algorithm decide works best, but in some circumstances, using only image ads can improve revenue. If you have a very text-heavy site with few other graphics, an image ad can provide an eye-catching focal point to your page. On other sites, it just looks overly busy and doesn’t work. Experiment!

Don’t Click!

Never, ever, ever be tempted to click your own ads or encourage your readers to click ads on your site. Not only is this against the rules, but Google has some pretty sophisticated fraud detection algorithms, and if you’re caught clicking or encouraging readers to click ads, the chances are your account will be frozen and you’ll lose all your earnings.

It’s also worth remembering that someone - a real, living, breathing advertiser - is paying for every ad click. If you’re clicking your own ads, you are effectively stealing their money. So don’t do it!

Track your changes

You’ll be surprised how a small change in layout or placement can affect the amount of money you can make from AdSense, but sometimes, you might find a sudden jump or fall in revenue and think “What did I do???”.

For More Information Visit Google Adsense.

How to look BIG in online marketing

Generally, internet marketers would jump into flame of fire after running internet marketing biz. In summary to be sucessful in online market business need to adopt following steps

1. It takes focus! - it is easy ESPECIALLY when you work online to get distracted by emails, instant messages, fun sites (that can kill a lot fo your time). So stay focused, If need be don’t check your email during the day, turn off any instant message clients so you can accomplish the tasks at hand. In affiliate marketing don’t get overwhelmed by the number of offers just go after a few and target those untill you make them work.

2. It takes perseverance! - You have to be the type of person that even though you are losng money you keep trying. Even when Google Delists you, you keep trying. You can’t give up in online marketing, as it is constantly changing there are new battles daily with victories and defeats you have to be able to take both in stribe.

3. Don’t get Complacent! - Yes its great when that offer starts earning you a 100 or 1000 a day but don’t get too happy and just sit back. Cause that offer or campaign can be pulled tomorrow or your rankings can take a dive… what then?

Google Is Good At Marketing

Yahoo and Microsoft compete badly against Google

Search is so consolidated that it is uncomfortable being an SEO. If Google decides to profile you or kill your sites there is not much you can do, especially because Yahoo! and Microsoft are losing marketshare month after month. Why are Yahoo! and Microsoft losing marketshare? Their bad marketing coupled with Google's good marketing:

The toolbar has NEVER given anyone any real information. BUT, it has given the world a perception that whatever Google ranks that must be right. It has also helped to give Google millions of devout followers and millions more who are willing to give access to a lot of private data just to see a little green go a little farther to the right. BRILLIANT. So brilliant in fact I have never understood why MSN and Yahoo haven’t done their own white paper and give away a little blue bar to "prove" it works. BUT, that is why Yahoo and MSN seem to be losing market share. Not because their results aren’t as good. It is because they aren’t as good at marketing.

To appreciate how bad Yahoo's marketing is, consider the following:

  • Overture Keyword Selector - their public facing keyword tool is unreliable and does not even promote their own brand or their own network on it.
  • Want to sign in to Yahoo search marketing? Go to sem.smallbusiness.yahoo.com. Yup...2/3 of the companies revenues come through a subdomain of a subdomain.
  • Yahoo powers millions of domain landing pageviews every day, and are afraid to put their brand on it, all while Google puts their brand (and typically search box) on everything they touch.

Given that the difference between Google and Yahoo is largely one of marketing, and given that Google has been doing more manual editing of the search results, that lends credibility to Wikia's human search model.

What happens when Wikipedia has a "search powered by you" box promoting an engine other than Google on every page of Wikipedia? Does Wikipedia keep ranking #1 for everything? Does this create another viable search channel for marketers? Does this competition make Google less arrogent and harsh in their webmaster relations department?

Here is Wikias' whitelist, and here is a blog tracking Wikia's progress. And here are some of Google's human review documents that were leaked a couple years ago.

Microsoft, Viacom Saddle Up New Deal

Probably safe to rule out Google and Viacom making up now

Advertising and content distribution should add up to a $500 million deal over five years as Viacom and Microsoft unite with a broad strategic agreement.

The actual figures could vary from the $500 million being suggested as a likely base value for the arrangement. As the two companies disclosed their deal in a statement today, one item leaped out and demanded attention:

Microsoft's Atlas division will become the ad server for Viacom's U.S. websites and Microsoft will have the exclusive right to sell remnant display advertising inventory on Viacom's U.S. websites.

Viacom runs MTV and Comedy Central, two well-known entertainment brands. This gives Microsoft exposure to audiences from the Daily Show and The Real World, among others, for multimedia advertising.

Their agreement also extends what content is currently available to Xbox 360 players through the Xbox Live Marketplace. Distribution also extends to Microsoft's MSN portal, where Viacom will provide short-form and long-form content.

The pairing comes amid Viacom's running battle with Google over the presence of Viacom shows on YouTube, as uploaded by users of the video sharing service. Viacom thinks this infringement is worth $1 billion and has sued Google for this.

Search Engine News

Loading...