There may be some pages and images on your website that you don’t want Search Engine Crawlers to index. Index means to show as results in Search Engine Result Pages (SERP). To do that you need to add a robots.txt file on your website. This file tells Crawlers what and what not to index. Robots.txt Generator creates that Robots.txt file.