Static WordPress Community

Fix for Crawling or Deployment Failure

Assuming your development site is which you’re trying to crawl and wanna publish at

There are few tips which can help avoiding crawl failure.

  • If you’re using Cloudflare, I would recommend bypass cache for ** for frontend.

  • Whitelist hosting server IP in Cloudflare, if you’ve static IP you can whitelist yourself as well.

  • For backend, you should point localhost IP to domain name using hosts file.

  • Make sure you have provided correct CDN/Storage login credential.

  • If you’re using HTTP Authentication, you should either provide correct details in plugin settings or consider whitelisting from HTTP web server configuration.

  • Sometime increasing fastcgi_read_timeout value can also help.

fastcgi_read_timeout 10800;
  • Last but not least, use faster Cloud Instance and keep server region same as your Storage region for faster deployment.
1 Like