I want to make a homepage and can be search by google and yahoo.I must make the robots file if I want to search engine can find my homepage,right???What is the content of the robots.txt.
Can anyone help me to solve this problem??
Thanks a lot.

I think the robots.txt is more for telling the robots NOT to look at or index some specific folders.  If you want the robot to be able to find your site, you have to put keywords in your meta tags.  And you might want to submit your url or site info to individual search engine site to be added to their index if you do not wish to wait for the robots to crawl to your site (they will eventually do).  For example, Google has a page for webmasters and you can find what you need there:

http://www.google.com/webmasters/

Hope that helps.

mickeyGoUp,

Thank you for your information.
I had already registered yahoo and google for about six weeks but cannot search my site when I type the keyword.If google,I type the url,then will go to my site directly.Of course I also write the keyword at the meta tag.
That mean robots.txt is no use for search engine,right??!

I see.  Are the keywords  "stand-out" enough?  If your keywords are too general, then your site will not be listed near the top of the list.  For example, if your keywords are "computer", "programming"..., there are tons of other sites that match those words, and it is harder to compete with them to get to the top of the search result.

And sometimes it take them longer than 6 weeks.  I have setup a site for a dental office and it took I think two months (or longer, I did not keep track) to show up on the search even I typed in the location and "dentist" as the search words.  

robot.txt only prevent bots to scan the site. So dont use it if you want to be indexed

Hi lhy,

Thank you so much.
I see.So that will be better as I delete the robots txt file.
And do you know any method that can be make the site be easier to search by search engine??

add your url to google, yahoo etc... and use good keyword. Also the content of the page is very important too. Try to make url that make sens. search engine also look at that.

http://www.webreference.com/dlab/books/html-pre/43-0.html

lhy,

That e-book is so great.I think I will be more understand as I read that.Thanks a lot.