Currently, robots.txt is only published if the site owner configures a custom robots.txt value in site settings.

I think a robots.txt should always be published, even if a custom one is not specified. It should also include a reference to sitemap.xml.

e.g.

User-agent: *
Allow: /

Sitemap: http://www.example.com/sitemap.xml