@john-shaffer, @bookwyrm thanks again for your recent works on WP2Static.
I’ve been a bit absent of late, focusing on an update to V6 (renamed to Static HTML Output), plus a bit of illness to sideline me most recently. I’m back to finishing up the next release for V6, just a few more blocking issues to go.
For WP2Static (V7), I need to update all add-ons to gel with the new Add-ons view.
In working on SimplerStatic and Static HTML Output recently, it’s given me some perspectives to improve WP2Static, in these areas:
- URL detection and rewriting
- background jobs processing
The filesystem detection of assets approach was useful to cover gaps in V6, where the URL detection happening during crawling wasn’t good enough to catch everything. But it pulls in too much.
V6 had a 2-pass crawling approach, so if it wasn’t linked to from the initial crawl list pages, it wouldn’t get crawled in 2nd phase. I think SimplerStatic uses a similar approach to the old old versions of WP2Static (named Static HTML Output Plugin back in the day), where it will continue to crawl until no new links are discovered.
The initial crawl list of detected URLs should suffice to just contain any known WP URLs and any CSS files, possibly JSON/XML, too, from which to detect any related assets. This touches on what @bookwyrm suggested to me yesterday as a feature where we can just export 1 page and it’s linked assets (ie, landing page export).
For background job processing, that will be a bit of a re-architecture, but would help in V7, where there is no handling of execution time/memory limits. This is a big problem when cancelling a job, there can be a bunch of PHP processes still running and either require a server reboot or end up with overlapping results as the old ones continue to write to filesystem/logs. Faster solution is some kind of lockfile checking, but I think shifting to background tasks will save a lot of headaches and is worth the effort.
I don’t want to context switch just now, will wait til new V6 release goes out to wp.org, then focus on V7 again.
Testing
Some new integration tests have been helpful in V6, with input/output HTML and CSS files helping speed up bug fixes/avoiding regressions. Still a long way to go, but it was useful in doing a big refactor of V6. Coverage % reports should be browsable from CircleCI artifacts, ie https://83-260261139-gh.circle-artifacts.com/0/coverage/index.html
I’ll get these in place for WP2Static and SimplerStatic and try to get coverage up across all 3.
There’s still a need for end to end tests. I have some in a VPS I was using with WP2Static prior to really test the deployed site’s content, even 3rd party hosted ones. I’ll likely move these into being used within the https://lokl.dev containers, for a known, documented environment. It’s not my main passion, but I can see an ISO/AMI coming out of that area for quickly spinning up VPS/EC2 dev sites, but focus for that is really trying to get the simplest local WP env for Mac/Win/Linux.
Anyway, that’s where I’m at right now. I’m looking forward to diving back into WP2Static/V7 and the add-ons. Another big shift is to try and make things as simple as possible, UX wise, ie, with just API key for a deployer, have a wizard to setup the bucket/remote stuff from within plugin in as few steps as possible (while keeping CLI and other options available for programmatic control).
Thanks again for your efforts and time and sorry I’m not as advanced as I’d hope to be yet!