Engine-Aware 'robots.txt' Policy with Google's Matcher Algorithm

---

Documentation for package ‘robotstxtr’ version 0.3.1

• DESCRIPTION file
• User guides, package vignettes and other documentation
• Package NEWS
• Package README

Help Pages

allowed_by_robots_text	Match URLs against a supplied robots.txt body
allowed_by_robots_url	Fetch robots.txt and match URLs under the deterministic fetch policy
as_legacy_robots_decisions_v1	Convert a v1 Google engine result to the legacy decision schema
print.robots_decisions	Print a 'robots_decisions' object
print.robots_fetches	Print a 'robots_fetches' object
print.robots_validations	Print a robots.txt validation result
robots_body	Preview or extract a stored robots.txt body
robots_engine_contract_v1	Inspect the versioned engine-aware robots contract
robots_engine_contract_v2	Inspect the engine-aware/v2 robots contract (Bing activation)
robots_evaluate_text_v1	Evaluate URLs against supplied robots.txt under an explicit engine profile
robots_evaluate_url_v1	Fetch and evaluate robots.txt under an explicit engine profile
robots_fetch	Fetch robots.txt documents for URLs under the deterministic fetch policy
robots_resolve_matcher_profile_v1	Resolve a matcher-profile selector before evaluating anything
robots_validate_text	Validate a supplied robots.txt document
robots_validate_url	Fetch and validate robots.txt documents for URLs