GTWY-3294 Create robots.txt file for use in production gateways
GTWY-45 Verify that important content is indexable by search engines