Cloudflare Caching

[chartung@community.nodebb.org]

To minimize load on our servers, I would like to set Cloudflare to cache as much of the content as possible.
I'm assuming that any image files or anything with a css/js/json file extension is safe to cache.
Obviously we can't cache html content for logged-in users, since there is user-specific data included in the headers.
As far as I can tell, it should be safe to cache html content for guests as long as the TTL is short (guests cannot post on our site, if that's relevant). However, it's not clear how we can implement that. The express.sid cookie is set for guests, so we can't just cache when that cookie is absent. It should be possible to set a custom cookie on login and clear that cookie on logout, but I can't find anywhere to set that cookie. I'm using a fork of https://github.com/julianlam/nodebb-plugin-sso-oauth, and the only place I can find a res object is https://github.com/julianlam/nodebb-plugin-sso-oauth/blob/master/library.js#L124, but setting the cookie fails because res.cookie is undefined.

Is there any way I can set a custom cookie on login, or ensure that NodeBB does not set an express.sid cookie for guests?

[julian@community.nodebb.org]

@chartung can CloudFlare just cache by respecting the cache control headers? That'd be the easiest and I'd be surprised if it could not.

[julian@community.nodebb.org]

@chartung you can also check out the scaling tips here:

Scaling - NodeBB Documentation

Documentation portal for NodeBB Forum Software

(docs.nodebb.org)

[chartung@community.nodebb.org]

@julian No, Cloudflare can use the cache-control response header to determine how long to cache content, but not whether to cache content.
Even if it was possible to cache based on response headers, it wouldn't be useful in this case; we want to ensure both that we're not serving user-specific content to guests (or other users) and that we're not serving guest-specific content to users, and the cache-control headers can only help with the former, not the latter.

[djensen47@community.nodebb.org]

I had to enable Cloudflare to cut out a massive flood of Chinese bots/crawlers. I have to Pro plan so websockets should be supported but I get nothing but errors now:

nodebb.min.js?v=qg59f2a0n14:70  POST https://www.axisandallies.org/forums/socket.io/?_csrf=556...a5a&EIO=4&transport=polling&t=PiQDHrJ&sid=AyNfwAoKNwtqarQqAP63 400 (Bad Request)

I have a rule for everything under /forums to bypass the cache.

[djensen47@community.nodebb.org]

If I turn off Cloudflare proxy, the problem stops. Also, my setup is like this:

internet -> Cloudflare -> HAProxy —vpc-> NodeBB servers

I'm wondering if I need to configure something between Cloudflare and the proxy?

[julian@community.nodebb.org]

@djensen47 what CF SSL settings are you working with? I think when CF terminates your SSL WS could go wonky... We use CF and web sockets work okay.

[djensen47@community.nodebb.org]

> @chartung said:
>
> Is there any way I can set a custom cookie on login, or ensure that NodeBB does not set an express.sid cookie for guests?

I'd like to get back to this question. Bots are hammering my site and CF Super Bot Fight mode isn't picking up new bots for me. I don't want to enable "Under Attack Mode" forever — it's not recommended.

One solution is to serve up cached items to the bots from CF but I don't want to be overly aggressive caching for users.

[julian@community.nodebb.org]

@djensen47 have you tried using Anubis? It's the only thing that worked, and has carveouts for search index crawlers.