Crawlability/indexability: Your sitemap and robots.txt collectively explain to Google That which you do and don’t want Google to crawl and index. These will make your site seem cleaner and truly feel much more professional, and also keep the loading time down. Areas of the URL could be exhibited in search https://www.youtube.com/watch?v=vd3_Ur7Wjks