Robots Text WordPress Sitemap

How to add a Robots.txt with a sitemap to Your WordPress Website

Here’s an example of the code for a robots text will that will allow all search engines to search the website. Using an FTP client and any text editor simply create a file named robots.txt and upload the file to main public directory of your server. If you don’t have an xml sitemap set up for your WordPress site, you can install Yoast WordPress SEO plugin and it will automatically generate one for you.

If you need a specific function in your robots.txt, like blocking Google Image search, here is a good  robots text generator http://www.mcanerin.com/EN/search-engine/robots-txt.asp.