Subpages not caching for WP install in subdirectory

#1

Hi, pro customer here, I’ve tried support but it was a dead end, so hoping to get some help here.

I have a WP install in a subdirectory. Aside from this one issue, Comet Cache is working perfectly. My problem is that CC isn’t caching subpages correctly. It’s not writing subdirectories inside the cache folder, and I get 404s for subpages unless I implement a very janky manual workaround.

All top level pages (domain.com/toplevel) cache perfectly. I have a single top level parent page which is the parent for five subpages. So the desired URL for these five subpages is ‘domain.com/parent/subpage’. /parent/ caches fine. When the subpages are set, in WPs ‘Page Attributes’ section, to domain.com/parent/subpage/ I get endless 404s. Digging around in the cache folder reveals that when the subpages are set to subpages, CC wont create any cached file for them. If I remove the subpage from the parent page, so it’s a top level page (domain.com/subpage), CC creates the cached file.

If I leave the intended subpage in WP as a top level page, then manually create the ‘parent’ subdirectory inside the cache folder and move the top level cached file into the ‘parent’ subdirectory, then set the currently top level subpage in WP back to a subpage, CC moves the cached file from the cache folder into a ‘cachexxxxxxxxxxx-tmp’ folder inside the ‘comet-cache’ folder, obviously giving me more 404s.

I’m currently working around this by setting the intended subpage to a top level page in WP, allowing CC to create the cache file in the root ‘cache’ folder, then changing the intended subpage back to a subpage in WP. This doesn’t delete or move the cached file. Then I create the ‘parent’ subdirectory inside the cache folder, whereby I can move the original cached file, and no more 404s.

This is obviously not a permanent fix for the problem. I’m about to start removing htaccess rules to see if I can find something that’s causing the issue, but honestly I’m not a developer and am just poking around kind of blind. I figured this might be a better place to get more focused help.

Support asked to see my htaccess rules, but didn’t suggest anything that might be causing the issue. After that, they said: “You mentioned that you installed WordPress into a sub-directory on the server. This is the most likely cause of the problems you’re experiencing as this requires a more advanced configuration and for the URLs in your WordPress Dashboard to be entered exactly right by the site owner. Please see: https://premium.wpmudev.org/blog/install-wordpress-subdirectory/”

I worked through that checklist one by one, with no success. Everything else in regards to CC is working perfectly, it’s just not creating directories inside the cache folder.

#2

I just tried this out on a test site of mine. I did not experience any problems at all. The sub-page was cached and remained accessible. There was no 404. So that suggests that this is an issue specific to your site.

What are your Comet Cache settings? Is this happening when logged in or logged out? Are you running a security plugin, or something like Varnish, or Cloudflare?

#3

Thanks for the reply Tim, much appreciated. I have no doubt it’s either something I’ve overlooked or due to my own htaccess config/subdirectory setup, etc. Just to confirm, your test site was also installed in it’s own subdirectory?

Is there somewhere specific I should start with detailing my CC settings? It’s a pretty comprehensive menu in there. I did keep it mostly at default, html compression is on. Apache optimisations: I’ve got gzip compression and leverage browser caching enabled. Originally I had canonical links enforced as well, and wondered if that might have been an issue, so removed that rule from my htaccess. Both times, same problem: CC only writes a cached file for a WP page that has no parent.

Logged in and logged out. It just will not create a subdirectory for any subpages regardless of either.

Plugins I’m running: Visual Composer, The SEO Framework, Comet Cache and Wordfence. As far as I can tell, there’s no conflict going on with WF and CC. It doesn’t seem to be a permissions issue as it will happily create the cached file for these pages in the root cache folder when they have no parent.

I’m using Cloudflare but only as a DNS host, none of the Performance or CDN features are turned on.

#4

Yes, I actually tested it both in a main directory and a sub-directory, just in case it gave me any clues, but the cache worked exactly the same.

Hmm, that would have been my first guess too!

I was also wondering (a) whether you are caching pages for logged-in users and whether the nonces are then somehow getting messed up, or (b) whether it’s an html compression issue. I’d suggest testing again with both features disabled.

If the problem remains, we’d have to move on to your plugins. While I’m not a fan of Visual Composer, I don’t see how it can be causing this. And I use The SEO Framework myself, so that suggests it’s not the problem either.

So that leaves Wordfence and Cloudflare, both of which would have been high on my list of suspects after canonical links.

I suspect Wordfence because there are plenty of threads on here where it has apparently caused problems (though I don’t recall anything like this before). But I don’t use it myself (I tried it once and hated it) so I can’t give you any specifics. But it would presumably be easy for you to deactivate so you could test to see.

Cloudflare, on the other hand, is something I have personal recent experience of. I was using it as a CDN on a site where it was causing numerous javascript problems (no matter what settings I tried), so I reverted to using it just as a DNS. That made little difference, so I abandoned it altogether for a more traditional approach, and now everything works fine. One thing I found with Cloudflare was that it mangled javascript redirects, so that they always went to 404s.

#5

Thanks again. It’s frustrating, but I’m determined to find the cause of the issue, because CC is such a great plugin, however as it stands right now it’s just not functional.

So I’ve worked through each of these, no luck. I never had the ‘cache pages for logged in users’ setting enabled, so unless I’m missing something, I guess I can rule that out? Turned html compression off, same problem.

Disabled Wordfence, same problem.

Which I guess leaves me with Cloudflare? Could this really be a redirect issue though? Why is CC creating the cached file for these pages when they have no parent, but not creating anything at all when they’re set to a subpage? Has that kind of behaviour been seen before? I’ve dug around online as much as possible and not had any luck finding other examples. If I manually create the parent directory inside the cache folder and move the cached files into it, the site works fine, no redirects.

Additionally, I have to do that in a set order: I set the subpages to no parent, update the page in WP, load the page from a logged out browser, then CC creates the cached file. If, however, I create the parent directory inside the cache folder, then move the cached file into the folder, then go back to WP and change the page back to a subpage and update it in WP, CC moves the cached file into a cache-xxxxxx-tmp directory outside the cache folder.

In order to keep the cached file in the cache folder (so I can move it) I have to first change the page back to a subpage in WP and update it. CC does nothing to the original cached file, although now I get 404s, because the directory structure isn’t correct in the cache folder. That’s when I can create the parent directory inside the cache folder, then move the originally cached file into the folder, then subpages load fine.

#6

Essentially, that’s what caching is: redirecting to a static html page instead of a dynamically-generated one. But my problem was definitely with a javascript redirect, whereas yours doesn’t sound like it’s concerned with javascript.

I know leaving Cloudflare is not simple and it is supposed to be among the fastest DNS around. So do you have a testing environment where you can try this without Cloudflare?

#7

Sorry, I worded that redirect question poorly; my line of thought was better reflected in the last two paragraphs I edited in after the original reply. I don’t have redirect issues if I manually create the directory structure and jump through several hoops to create the cached pages.

I’m not opposed to leaving Cloudflare. I do have a current test environment, however it’s always been just for theme updates and isn’t running Comet Cache. Would that even be suitable to troubleshoot completely, as I only have a single site license for CC?

#8

OK, but I’d hate for you to leave and find that the problem still wasn’t resolved!

If your second site is just for testing, then using CC on it would comply with the license.

#9

Thanks for the heads up, I didn’t realise that about the license.

Something interesting: with this info in mind, I decided to setup a brand new local test site, as it’s a relatively small project. All I have installed right now is my WP theme, which also installs Visual Composer. So just those two from my live site. I intended to import the theme options, setup the top level and subpages, then add plugins one by one until I hit a problem. Thing is, I’ve already run into the problem: subpages are 404ing already, from a brand new setup. So it’s either my WP configuration (maybe?), my theme (possibly?), or Visual Composer (unlikely?). I’ve clearly overlooked something here.

Which I guess means any solution falls outside the boundaries of this forum, as it’s clearly not a CC issue. I don’t really feel any closer to solving it, but at least I’ve narrowed it down a little more.

#10

That’s very interesting. It sounds like a strange permalinks or rewrite issue then. Have you tried re-saving your permalinks?

If that doesn’t work, I’d suggest switching to the 2017 theme and seeing if it works then.

#11

Yeah, I’ve done that multiple times, no luck.

You beat me to it. Further progress: I feel terrible for admitting this, but I’ve never actually tested the subpages out on one of the default WP themes. Just activated the Twenty Seventeen theme, and of course subpages load fine. No 404s in site. Guess it’s my theme. Partial success!

#12

Well, at least you know where the problem lies. Does your theme use custom post types? (I know they wouldn’t be your pages, but I’m trying to think how a theme could affect permalinks.)

#13

For anyone stumbling across this in the future, I can now confirm the 404/not writing a cache file for subpages issue was 100% caused by my theme. CC/subpage caching is working in my test environment with a default WP theme, and I’ve been able to find success getting the theme to cache subpages in an indirect manner, by using a custom permalinks type plugin and setting the desired /parent/subpage/ permalink.

Thanks for your help Tim. I suspect the issue has something to do with a custom page type perhaps? I found a three year old reference on the theme support forum referencing a very similar issue with a portfolio type page, where domain.com/portfolio/ pages were working, but domain.com/portfolio/subpage wasn’t. It’s a fairly popular theme, so I’m surprised nobody has run into the same issue as me before, and ideally I’d like to have the native WP parent/subpage functionality working as it should, but the solution is above my pay grade, so to speak, so I’ll settle for my plugin solution.

#14

Quite possibly. I assume that this is not a free theme, so I can’t take a look. But if (a) that is the problem, (b) you can find the file where the custom page type is created, and © you could then post that code here, it should be possible to work out a fairly simple fix.

#15

It is paid. Support has ended, otherwise I definitely would have lodged a ticket. I honestly don’t have enough php knowledge (i.e., ‘none’) to even begin narrowing it down in any useful manner, as I’m not even certain it is a custom page issue.