On This Page
v 1.0.0
Last updated: 16/03/2026
Robots.txt Manager
Manage robots.txt and crawler hints from the dashboard—no FTP.
Overview
Robots.txt Manager lets you edit and manage your site’s robots.txt file from the WordPress admin. No FTP or file editor needed. Add or edit directives for search engine crawlers. Reference your sitemap. Block or allow specific paths. The module typically serves a virtual robots.txt (via rewrite rules or a query) so you do not need a physical file. Changes take effect immediately after saving.
Who is this for?
Site owners and SEOs who need to control crawler access via robots.txt without editing files directly. Useful for blocking admin, wp-includes,
other paths from indexing.
Features
Edit robots.txt from Admin
Text area or builder to edit robots.txt content. Save and it is served at yourdomain.com/robots.txt.
Sitemap Reference
Add your sitemap URL (e.g. Sitemap: https://yoursite.com/sitemap.xml). Helps search engines find your sitemap.
Standard Directives
User-agent, Allow, Disallow. Follow robots.txt standards.
Virtual File
Typically served via WordPress (rewrite or query). No physical file required. Overrides default WordPress robots.txt if present.
How It Works
1
Enable the Module
Go to WP PowerSuite, find "Robots.txt Manager," and click the Enable button.
2
Edit Content
Open the module settings. Edit the robots.txt content. Add sitemap URL. Save.
3
Verify
Visit yourdomain.com/robots.txt. Confirm your content is displayed.
Performance Note
Minimal. robots.txt is requested infrequently by crawlers. Served via a lightweight endpoint. No impact on normal page loads.
Installation & Activation
1
Install WP PowerSuite
Make sure the WP PowerSuite plugin is installed and activated on your site.
2
Enable the Module
Go to the WP PowerSuite menu, find "Robots.txt Manager" in the module list, and click the Enable button.
3
Configure
Open the module settings. Edit robots.txt. Save.
Requirements
PHP 8.1 or higher
WordPress 6.0 or higher
Permission to manage options (Administrator)
Use Cases
- CrawlerBlockingBlock wp-admin from crawlers
- PathBlockingBlock wp-includes, uploads, or other paths
- SitemapReferenceAdd sitemap reference
- CrawlerControlCustomize crawler behavior
Frequently Asked Questions
Does robots.txt block indexing?
robots.txt tells crawlers what not to crawl. It does not guarantee non-indexing. For sensitive content, use noindex meta or .htaccess. robots.txt can be ignored by some crawlers.
What if I have a physical robots.txt?
The module typically overrides the default. Check how your host serves robots.txt. The module may take precedence when enabled.
Troubleshooting
- Changes not showingClear cache (browser, CDN, plugin). Visit robots.txt in incognito. Ensure the module is enabled and you saved. Check rewrite rules if using virtual robots.txt.
