[09:20:31] @posix_memalign merged [09:20:33] deploying now [09:22:49] should be deployed [09:23:50] [1/4] Thanks! I'll start testing robots.php changes on beta next then. [09:23:50] [2/4] I'm thinking of: [09:23:51] [3/4] 1. Use localized version of special page titles in robots.txt [09:23:51] [4/4] 2. Make sure that robots.txt works on bing. We need to either remove the bing entry (thereby losing the crawl-delay directive) or duplicate our rules for `*`, either by copy-pasting or programmatically. [09:28:13] Nice [09:40:22] [1/9] Ok I think I got it working on https://fr.mirabeta.org/w/robots.php [09:40:22] [2/9] ``` [09:40:23] [3/9] # Special pages in English [09:40:23] [4/9] Disallow: /wiki/Special: [09:40:23] [5/9] Disallow: /Special: [09:40:24] [6/9] # Localized special page URLs auto-generated by robots.php. [09:40:24] [7/9] Disallow: /wiki/Sp%C3%A9cial: [09:40:24] [8/9] Disallow: /Sp%C3%A9cial: [09:40:24] [9/9] ``` [09:56:47] [1/7] Some tools don't like encoded URLs, changing it to [09:56:47] [2/7] ``` [09:56:47] [3/7] # Localized special page URLs auto-generated by robots.php. [09:56:48] [4/7] Disallow: /wiki/Spécial: [09:56:48] [5/7] Disallow: /Spécial: [09:56:48] [6/7] ``` [09:56:48] [7/7] Instead [09:58:34] Oh ffs Bing's robots.txt tester wants the percent-encoded URLs. [10:03:55] Ok now it generates both versions. I think this is good enough. Will head to sleep now and let anyone else interested in it take a look. [10:04:13] https://github.com/miraheze/mw-config/pull/6381 [10:12:29] [1/6] Somehow it's empty for EN wikis, even though everything works fine in shell.php. I'll investigate tomorrow I guess. [10:12:29] [2/6] ``` [10:12:29] [3/6] # Localized special page URLs auto-generated by robots.php. [10:12:29] [4/6] Disallow: [10:12:30] [5/6] Disallow: /wiki [10:12:30] [6/6] ``` [17:12:17] [1/2] https://cdn.discordapp.com/attachments/1006789349498699827/1523013591824072958/image.jpg?ex=6a4a9070&is=6a493ef0&hm=b79467d250ddc4c919b8abb853219ce7f417e639f66127841e02e561467abcde& [17:12:17] [2/2] https://cdn.discordapp.com/attachments/1006789349498699827/1523013592234987560/image.jpg?ex=6a4a9070&is=6a493ef0&hm=680f59e0704662d51c50ce1505cb0e5e3e5037601dbd2af5b74305894d87c63a&