Help Center
-
Knowledge Base
-
-
-
- Articles coming soon
-
-
-
-
-
- List of all supported fields
- [Solved] Crawl error with Cloudflare website, Cloudflare is blocking crawler bot
- Expertrec Crawler - Control your Index
- Take Screenshot
- Manual extraction
- Automatic extraction
- Boost Fields
- Filter fields
- Create search for e-commerce store
- Use Existing Search Box
- Wordpress plugin
- Code
-
< Back
You are here:
Print
Crawl speed
Created On
Last Updated On
bymuthali ganesh
Crawl speed-
Crawl speed sets the speed in which expertrec’s search crawler has to crawl your website. a higher value will crawl your website fast, but will put more load on your website. The recommended setting is value=2.
It has 4 values-
- Value=1-Slow crawl
- Value=2-Polite crawl
- Value=3-Fast Crawl
- Value=4- aggressive Crawl.
Page load delay time– This is the speed at which expertrec’s crawler loads consecutive pages. The lesser the value, the faster the crawl will happen. The recommended time delay is 30 seconds which is the default as well.
Table of Contents