Feature req: 'afterHeader' event or the ability to specify an ETag prefix

[papandreou]

I'm writing a piece of connect middleware similar to Apache's mod_negotiation. Based on req.url, the Accept-Language and Accept headers, a custom locale cookie, and which files are found on disc, it figures out which file to actually serve.

For example:

GET / HTTP/1.1
Cookie: language=fr
Accept: text/html

could get mapped to /index.fr.html, and

GET /someDir/foo.html HTTP/1.1
Accept-Language: nl, da

could get mapped to /someDir/foo.da.html -- unless a Dutch version of the file was also available.

I'd like to use send for this, or maybe just require that my module sits right above connect.static in the middleware stack.

Here's my problem -- since any given incoming url might map to different files on disc, which could very well have the same modification times and byte lengths, send's current ETag scheme won't work. If a client does a conditional GET request for a page after switching to a different language (either changing the language cookie or Accept-Language), it could lead to a "false positive" 304 response.

I know I can set the ETag header in my own code, and that's my current solution -- I use send's algorithm and append -<localeId> to the ETag. Unfortunately that means my server is doing 2 stat calls for each static file request, which I'm not a fan of.

So what I'd like instead is for send's setHeader to support adding to existing "ETag fragments", or alternatively an event being emitted after the headers have been set, but before the freshness check. This would be a nice spot: https://github.com/visionmedia/send/blob/master/lib/send.js#L348

What do you think? Is this use case too convoluted to warrant added complexity in send?

[papandreou]

By the way, the module I was talking about was just open sourced: https://github.com/One-com/express-negotiator

[tj]

you shouldn't map the same url to multiple files anyway, at least redirect to the canonical resource

[papandreou]

That's a bit of a categorical statement. I'm guess you're not a fan of mod_negotiation with MultiViews then?

I find it very useful. My build system outputs one HTML file per language (index.en_US.html, index.fr.html, etc.), and this technique allows me to avoid HTTP redirects and have a pretty url for my app regardless of the active language. Switching to a different language is just a matter of setting a cookie and refreshing the page.

[tj]

a url is meant to point to a single resource, especially when you're dealing with statics, anything else just messes with caching up the chain, that's what redirects are for

[tj]

or alternatively Vary on Accept-Language

[papandreou]

If it's the same content in different languages, isn't it the same resource?

[papandreou]

My module also outputs Vary: Accept-Languge, of course, but on top of that it's still useful to allow the client to do a conditional GET.

We've got two single-page apps running it in production with Varnish in front and excellent caching performance. Varnish recognizes the Vary header and caches the language variants seperately.

[tj]

fair enough, could you tap into the "header" event that connect adds? You should be able to manipulate them from there

[papandreou]

Unfortunately header is emitted after send has decided whether to send a 304, so that's too late.

[tj]

hmm damn yeah i cant think of anything cleaner than .on('header for the send() call and pass the file or whatever else may be helpful

[dougwilson]

Not only are the etags marked a weak now, but the file path is included in the etag generation, so the two language files will have different etags now.

[papandreou]

@dougwilson Thanks a lot for letting me know. Do you happen to know which version of Express that change landed in?

[dougwilson]

The change just happened yesterday. They are not yet in a release of this module, though I intend to release send sometime today and bump the dep in the various modules I have access to (including express), but can't guarantee any release time frames for those respective modules.