Help Center
-
Knowledge Base
-
-
-
- Articles coming soon
-
- Home
- Install Settings
- Merchandising
- Personalize
- Category
- NLP
- Script-Hooks
- Analytics
- Manage Users
- Agentic Search (AI-Powered Conversational Search)
- AI Synonyms
- Semantic Search
- AI Shop Assistant (Chat Widget)
- Recommendations Widget
- Advanced Filtering & Filter Trees
- Troubleshooting Guide
- Search Performance Optimization
- BigCommerce Search Integration
- WooCommerce Search Integration
- Getting Started with eCommerce Search
- Show all articles (5) Collapse Articles
-
-
- Install Settings
- Analytics
- Search settings
- Merchandising
- Category
- NLP
- Search-API info
- Manage Users
- Script-Hooks
- Personalize
- Agentic Search (AI-Powered Conversational Search)
- AI Synonyms
- Semantic Search
- Recommendations Widget
- Advanced Filtering & Filter Trees
- Troubleshooting Guide
- Search Performance Optimization
- WooCommerce Search Integration
- Show all articles (3) Collapse Articles
-
-
- Instructions to install app
- Home
- Install Settings [Shopify]
- Category
- Merchandising
- Search settings
- NLP
- Analytics
- Search-API info
- Manage Users
- Script-Hooks
- Personalize
- Agentic Search (AI-Powered Conversational Search)
- AI Synonyms
- Semantic Search
- AI Shop Assistant (Chat Widget)
- Recommendations Widget
- Commerce MCP Server (Shopify AI Tools)
- Advanced Filtering & Filter Trees
- Troubleshooting Guide
- Search Performance Optimization
- Show all articles (6) Collapse Articles
-
- List of all supported fields
- [Solved] Crawl error with Cloudflare website, Cloudflare is blocking crawler bot
- Expertrec Crawler - Control your Index
- Take Screenshot
- Manual extraction
- Automatic extraction
- Boost Fields
- Filter fields
- Create search for e-commerce store
- Use Existing Search Box
- Wordpress plugin
- Code
- Troubleshooting Guide
-
- Articles coming soon
-
< Back
You are here:
Print
Crawl speed
Created On
Last Updated On
byExpertrec Marketing
Crawl speed-
Crawl speed sets the speed in which expertrec’s search crawler has to crawl your website. a higher value will crawl your website fast, but will put more load on your website. The recommended setting is value=2.

It has 4 values-
- Value=1-Slow crawl
- Value=2-Polite crawl
- Value=3-Fast Crawl
- Value=4- aggressive Crawl.

Page load delay time– This is the speed at which expertrec’s crawler loads consecutive pages. The lesser the value, the faster the crawl will happen. The recommended time delay is 30 seconds which is the default as well.
Table of Contents