As part of Brambl SEO, we auto-generate a robots.txt and sitemap.xml when a Brambl website goes live.
Your sitemap.xml and robots.txt are automatically generated when your site is live.
Whenever you publish a new page, the sitemap will be updated to include the new page.
You can find the sitemap and robots using the following link
Sitemap.xml: http://www.yourwebsite.com/sitemap.xml (eg http://www.aal-uk.co.uk/sitemap.xml)
Robots.txt: http://www.yourwebsite.com/robots.txt (eg http://sitemaps.uk.brambl.com/robots.txt)
These will only appear when your site is live, and will be updated when you publish your site.
Please note: old sites created before this update that are live will need to republish to get the new sitemap & robot files.
Sitemaps are used to provide search engines a map of all the pages for your website.
You can upload sitemaps via Google & Bing webmaster tools, see this guide for example on how to submit a sitemap to Google.
If a site goes live, it can take awhile for search engines to realise it exists. Uploading a sitemap is a great way of speeding up the time it takes a search engine to crawl your website.
Robots.txt can be used to tell search engines not to look at certain pages. For Brambl websites, all pages are generally viewable so we autoapply a rule of
User-Agent: *
This tells search engines all pages are visible