Ashesh Saini | Blog

What is Robots.txt??

The robots.txt file is used to instruct search engine robots about what pages on your website should be crawled and consequently indexed. Most websites have files and folders that are not relevant for search engines (like images or admin files) therefore creating a robots.txt file can actually improve your website indexation. It also provides you a way to hide your content from search engine.

Robots.txt Syntax

# comment
User-agent: [robot-names][(*)Wild card character]
Disallow:[(/)all] [specific directory] [specific file Location]

User-agent

The value of this field is the name of the robot the record is describing access policy for. If more than one User-agent field is present the record describes an identical access policy for more than one robot. At least one field needs to be present per record. You can multiple ( more than 1) User-Agents in one entry.

Disallow

The value of this field specifies a partial URL that is not to be visited. This can be a full path, or a partial path; any URL that starts with this value will not be retrieved. For example, Disallow: /help disallows both /help.html and /help/index.html, whereas Disallow: /help/ would disallow /help/index.html but allow /help.html. Any empty value indicates that all URLs can be retrieved. At least one Disallow field needs to be present in a record.

Things to remember while writing robots.txt:

1. Robots.txt should be written in a plain text editor like Notepad. Do not use MS-Word or any other text editor to create robots.txt. The bottom line is this file should have the extension “”.txt”" else it will be useless.
2. A robots.txt file is always stored in the root of your site, and is always named in lower case. Spiders will always search for it in the root directory (e.g. http://www.example.com/robots.txt)
3. There can only be one instruction per line,
4. You should avoid putting spaces before the instructions (recommended to avoid making mistakes).
5. For security reasons, be aware while preventing spiders from accidentally indexing sensitive and private areas of your site, as anybody at all can view your robots.txt file.

Read more…

A cheat sheet is a concise set of notes used for quick reference. “Cheat sheet”(sometimes also rendered as “cheatsheet” or “cheat-sheet”) can be used like a memory refreshing tool. Cheat sheets are popular in any area where a quick reference is useful, not just for examinations. Like computer industry has many cheat sheets due to the high number of commands, actions, procedures, terms and concepts one must learn to use programming languages and interfaces. In Information Technology, a cheat sheet is a reference tool that provides simple, brief instructions for accomplishing a specific task. Below you will find the Various professionally prepared and Must have cheat sheets on most of topics from computer software and internet world. As you cant remeber the URL of every page which have cheat sheet, So Please feel free to bookmark this post to get all under one umbralla. Now keep all the information at your fingertips. Do you have/know a useful cheat sheet that you’d like me to consider posting? Let me know! Read more…

Today while browsing, I found a hack about two most popular instant messengers (chatting clients); Yahoo Messenger and Google Talk. This Simple Hack allows you to Run multiple instances (more then one) of Yahoo Messenger and Google talk simultaneously on one PC. Yes!! I am not joking.. You can open more than one instance of any of them, working in parallel.

FOR YAHOO MESSENGER

• Go to Start >> Run >>
• Type regedit >> press Enter
• Go to HKEY_CURRENT_USER >> Software >> Yahoo >> pager >> Test
• Then Right click on test >> choose New >> Dword value .
• Rename it as ‘Plural’.
• Double click it >> assign a decimal value of 1.
• Close the registry and Restart yahoo messenger.

Hey Its All Done!! Now you can open ‘N’ instances of yahoo messenger and login into each from different yahoo ID.

FOR GOOGLE TALK

• Create a shortcut of Google Talk messenger on your desktop or any other location.
• Right click on the Google Talk messenger icon and select properties option
• Modify target location text from this “c:\program files\google\google talk\googletalk.exe” /startmenu to “c:\program files\google\google talk\googletalk.exe” /nomutex
• Then Click OK!!

Now you can run multiple instances of Google talk . As far as system security is concerned, I don’t think any of the above tricks can harm the computer. But still you can have some precautionary measure before trying it.

There are some other methods are also if you dont want to take any chance on your system. Use PIDGIN. Pidgin is a desktop chatting client and very easily configurable, After installing it you will not need Yahoo Messenger , MSN messnger or Gtalk etc. Just use Pidgin for signing into everyone. And Ofcourse you can have 2 or more yahoo accounts running simultaneously.
Choice is All Yours.

You must know about your neighbor websites, hosted on your Webserver/ Shared IP. Shared hosting is good in many ways for service providers as multiple website can be hosted on same webserver under the one IP address. But Some web hosts put many hundreds of websites under one IP address. If few of your neighbor sites are using high CPU resources, It can slowdown the speed of other sites too. Webserver remain busy serving the resource hungry sites and the overall response time get increased. So always ask your webhost about how many sites they host on one webserver. Here I am providing you a link, from which you can check that who else are in your neighborhood.

Reverse IP DNS Check Tool: http://www.myipneighbors.com/

This tool will find other domain names hosted on your IP address. Just enter a domain (www.example.com) or an IP address (127.0.0.1) to get a list of domain names that share the same IP address. Though Its not a perfect method to know about how much load a webserver has, Because Servers cab be configured to host multiple IP addresses. Means Thing can be worst like below:

So you can never Know how busy a webserver actually is. Lesser the sites hosted on a server, Better the performance. Do Remember, If you have Bad neighborhood, it also affects your Google Page Rank and visibility in some search engine listings.

Related Tools:
http://www.domaintools.com/reverse-ip/