Last modified: 2014-08-27 03:22:33 UTC
When loading a gagdget or custom script from eg. meta/mediawiki/enwiki with -mw.loader.load('//meta.wikimedia.org...&maxage=86400&smaxage=86400') the responded cache-control directive is - private, s-maxage=0, max-age=0, must-revalidate which prevents these scripts from being cached properly and causes a lag (40-500ms) on each call. (IMHO JS pages should have a reasonable default > 0) Example with requested cache: Remote Address:xx.xx.xx.xx:443 Request URL:https://meta.wikimedia.org/w/index.php?title=User:Hedonil/Test/XTools.js&action=raw&ctype=text/javascript&maxage=86400&smaxage=86400 Request Method:GET Status Code:304 Not Modified Request Headersview source Accept:*/* Accept-Encoding:gzip,deflate,sdch Accept-Language:en,no;q=0.8,nb;q=0.6,vi;q=0.4,nl;q=0.2,zh;q=0.2,ru;q=0.2,en-US;q=0.2,zh-CN;q=0.2,zh-TW;q=0.2 Connection:keep-alive centralnotice_bucket=0-4.2; uls-previous-languages=%5B%22en%22%5D; mediaWiki.user.sessionId=4n8HMszYkP6cFIWMpzFAXdt0yQt1KsLK; metawiki-campaign=loginCTA; metawikiUserID=4326783; metawikiUserName=Anthelix; metawikiSession=d8f8b1cff708d4d092ddc67b62df397d; centralauth_User=Anthelix; centralauth_Session=9057b1467cc6fa35e848cc313af5f71c; forceHTTPS=1 Host:meta.wikimedia.org If-Modified-Since:Wed, 13 Aug 2014 04:18:39 GMT Referer:https://en.wikipedia.org/wiki/Charles_Schild User-Agent:Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36 Query String Parametersview sourceview URL encoded title:User:Hedonil/Test/XTools.js action:raw ctype:text/javascript maxage:86400 smaxage:86400 Response Headersview source Accept-Ranges:bytes Age:0 Cache-Control:private, s-maxage=0, max-age=0, must-revalidate Connection:keep-alive Content-Encoding:gzip Content-Type:text/javascript; charset=UTF-8 Date:Wed, 13 Aug 2014 07:55:42 GMT Last-modified:Wed, 13 Aug 2014 04:18:39 GMT Server:nginx/1.1.19 Vary:Accept-Encoding Via:1.1 varnish, 1.1 varnish, 1.1 varnish X-Cache:cp1065 miss (0), amssq50 miss (0), amssq31 frontend miss (0) X-Content-Type-Options:nosniff X-Varnish:948365503, 2378009697, 4022640906 ConsoleSearchEmulationRendering Example without requested directive: Remote Address:xx.xx.xx.xx:443 Request URL:https://en.wikipedia.org/w/index.php?title=MediaWiki%3AGadget-HotCat.js%2Flocal_defaults&action=raw&ctype=text/javascript Request Method:GET Status Code:304 Not Modified Request Headersview source Accept:*/* Accept-Encoding:gzip,deflate,sdch Accept-Language:en,no;q=0.8,nb;q=0.6,vi;q=0.4,nl;q=0.2,zh;q=0.2,ru;q=0.2,en-US;q=0.2,zh-CN;q=0.2,zh-TW;q=0.2 Connection:keep-alive Cookie:centralnotice_bucket=1-4.2; uls-previous-languages=%5B%22en%22%5D; mediaWiki.user.sessionId=2QWfgI0PCzO7nqVHetUbLsTGj025kox1;enwikiSession=a2cd738feeaa6203a52e18e27710a479; centralauth_User=Anthelix; centralauth_Session=9057b1467cc6fa35e848cc313af5f71c; forceHTTPS=1 Host:en.wikipedia.org If-Modified-Since:Mon, 23 Jun 2014 19:15:43 GMT Referer:https://en.wikipedia.org/wiki/Charles_Schild User-Agent:Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36 Query String Parametersview sourceview URL encoded title:MediaWiki:Gadget-HotCat.js/local_defaults action:raw ctype:text/javascript Response Headersview source Accept-Ranges:bytes Age:0 Cache-Control:private, s-maxage=0, max-age=0, must-revalidate Connection:keep-alive Content-Encoding:gzip Content-Type:text/javascript; charset=UTF-8 Date:Wed, 13 Aug 2014 07:55:42 GMT Last-modified:Mon, 23 Jun 2014 19:15:43 GMT Server:nginx/1.1.19 Vary:Accept-Encoding Via:1.1 varnish, 1.1 varnish, 1.1 varnish X-Cache:cp1053 miss (0), amssq55 miss (0), amssq54 frontend miss (0) X-Content-Type-Options:nosniff X-Varnish:2605122819, 917135542, 2533692991
I can reproduce this on any WMF wiki (including *.beta.wmflabs.org, both logged in and logged out), but not on my private test wiki or on other public wikis like http://www.explainxkcd.com, where Cache-Control is set as requested by the parameters (or as the set default). So I guess it's Varnish that replaces the Cache-Control with "Cache-Control:private, s-maxage=0, max-age=0, must-revalidate"
I can confirm that appending &bcache=1 to the URL will set the Cache-Control as expected, so this definitely is caused by http://git.wikimedia.org/commitdiff/operations%2Fpuppet.git/ce1f2b638ab2a09097ffe207ec6dc35c3823bc1b
(In reply to Michael M. from comment #2) > I can confirm that appending &bcache=1 to the URL will set the Cache-Control > as expected, so this definitely is caused by > http://git.wikimedia.org/commitdiff/operations%2Fpuppet.git/ > ce1f2b638ab2a09097ffe207ec6dc35c3823bc1b Thanks for investigating. Mark: Any comments, as you worked on that change?
Mark: Any comments, as you worked on that change?
(In reply to Andre Klapper from comment #4) > Mark: Any comments, as you worked on that change? That's an old change (2013) so he might have forgotten ;) Krinkle/Roan/Trevor: Thoughts here on the right cache settings?
That change simply puts the same behaviour in place that was there with the Squid cluster before Varnish. The problem is that we don't have separate Cache-Control headers for _our caches_ vs _clients_ in MediaWiki, and never have had. So MediaWiki's Cache-Control header is targeted at our caches, and Varnish replaces that header with one that for most URLs doesn't allow caching (just like our Squid config did). Unfortunately, Varnish can only guess from the URL.
If this is caused by this lines (as Michael M. stated): sub vcl_deliver { /* Override the Cache-Control header for wiki content pages */ if (req.url ~ "(?i)^/w(iki)?/.*" && req.url !~ "^/wiki/Special\:Banner(Controller|ListLoader)" && req.url !~ "(?i)^/w/(extensions/.*|api\.php)" && req.url !~ "(?i)bcache=1") { set resp.http.Cache-Control = "private, s-maxage=0, max-age=0, must-revalidate"; } } Why not just simply add a regex to meet the directives of the request? Request-url and query-string are available at present; the "new" parameter bcache is respected, why not maxage/s-maxage? IMHO at least "private" should be removed for requested .js/.css pages or for action "raw". This would allow proxies/local browsers to intelligently set caching. "private" + "must revalidate" leaves no elbowroom for systems to guess the right things. https://meta.wikimedia.org/w/index.php?title=User:Hedonil/Test/XTools.js&action=raw&ctype=text/javascript&maxage=86400&smaxage=86400