Initial crawl not completed (500 error)

Hello,
After initial 404 errors, I’ve been trying to run wp2static but end up in a "500 error code returned from server.

*Please check your server's error logs or try increasing your max_execution_time limit in PHP if this consistently fails after the same duration.*"

I tried increasing limits for memory and timeout but still got the same error - Max time is now 600 seconds.
I then moved by site to a different hosting provider, even though still on CPanel

I then uploaded the “wp2staticfixmultisitemap” version mentionned in another thread by someone also facing a 500 error, but no luck, still that 500 error.

The error log showed that error which does not seem to be the fatal one ?

[26-Feb-2021 09:57:01 UTC] PHP Warning: Invalid argument supplied for foreach() in /home/mysite/public_html/wp-content/plugins/wp-seopress/inc/functions/sitemap/template-xml-sitemaps-single-term.php on line 70

The site indeed has had a sitemap from the start - generated by the WordPress plugin SEOPress at /sitemaps.xml, specifying this because it seems sites lacking one faced issues.

On my server however I see 2 folders “wp2static-crawled-site” and “wp2static-processed-site” containing files, which I am comparing to what should apparently be generated.

NB: Also realised the last line from the logs is
“2021-02-28 22:21:25: 404 for URL /wp-content/plugins/wp-seopress-pro/inc/functions/google-analytics/style/ruleset.xml” FWIW.

I’ll continue testing and hope I can help solve the problem for others too.
Thanks.

Added: also noticed that while in my Options page I set the output URL to https://docs.mysite.fr/testfolder/ I get files with example.com in all URLs, I guess this is because the process failed and a final replacement was not done ?

Hi @mondebot, deploying to a /subdir/ isn’t well supported, I should really block that until it is.

The wp2static-processed-site dir should have all URLs rewritten once process completes, so sounds like you’re right about it like that due to not having finished.

Here’s the latest development build, in case that resolves any of the issues for you (there’s a CHANGELOG.md file in GitHub, only issue resolved that may help you is around Sitemaps, but still in development mode/not covered with tests/trustable yet:

https://paste.c-net.org/CouponsAsbestos

MD5 (mondebot717dev.zip) = f056654350d10e2354b8e31d20021528

Noted, I had read too fast about subdirs and thought it was about source sites installed in subdirs, not deployments to. Moved to a dedicated subdomain.
I’ll try that version now, thanks for the reply.

FWIW, I could never complete a run so far. I l moved the site to a new host but keep getting that error.
I increased my server limit to

memory_limit 512 Mb
max_execution_time set to 900

than ran again, same error. I then deactivated ALL extensions (except for WP2Static, doh), still getting that 500 error, way sooner than the 900 seconds limit I now set.
Log files bear no trace of such a fatal error.

Diagnostics all good:
|PHP max_execution_time|Unlimited|
|PHP memory_limit|512M|
|Uploads directory writable|Writable|
|PHP version|7.4.12|
|cURL extension loaded|Yes|
|WordPress Permalinks Compatible|Yes|

Any idea what to try next ?
Thanks.

Hi @mondebot, very sorry for the late reply, I’ve been on the road this week.

This guide I wrote a while back may help in isolating the exact cause of failure for you.

Re the 500’s, this bug seems to be affecting quite a lot of users now: 500 error when certain sitemaps are non-existant · Issue #751 · leonstafford/wp2static · GitHub