Sitemap.xml - XML Sitemap like a blueprint for your house and each web page were a room.
XML Sitemaps are important for SEO because they make it easier for Google to find your site’s pages—this is important because Google ranks the web pages, not just websites.
Robots.txt - The most common usage of Robots.txt is to ban crawlers from visiting private folders or content that gives them no additional information. Robots.txt Allowing Access to Specific Crawlers.
Allow everything apart from certain patterns of URLs.





Reply With Quote

Bookmarks