[02:35:02] Hello, I'm trying to connect moodle instance with MediaWiki OAuth, but I've figured with some problem: error/Could not discover end points for identity issuer: Wikimedia [URL: https://meta.wikimedia.org/.well-known/openid-configuration] [02:35:48] would be https://meta.wikimedia.org/wiki? -yup, I know, this is a mediawiki support but maybe I can get the response here- [02:45:32] Is there a way to anonymously log all searches that are not page titles? [08:00:28] Kokushibo: Wikunedia OAuth doesn't support autodiscovery [08:01:19] ... Wikimedia... [08:02:12] And it is not an OpenID provider at all [08:04:32] nvm the last one, that well-known is for OpenID Connect, not OpenID [08:06:26] MediaWiki OAuth does have a homegrown OIDC-like endpoint, not sure if it would work with a proper OIDC client though [19:57:51] I am writing software that requires me to scrape a few thousand wikipedia pages periodically. The weekly dumps are not suitable. https://en.wikipedia.org/wiki/Wikipedia:Database_download#Please_do_not_use_a_web_crawler indicates that I would need to scrape at most 1 page per second, however this will take 17 minutes per 1000 articles. The page indicates that fetching wikicode from Special:Export is less [19:57:57] taxing on the servers. At what rate could I scrape from these pages without getting my IP blocked? [19:59:02] Also the page asks that if I exceed 1 per second on HTML then I do so at low usage times, but the linked graphs do not display wikipedia usage by time of day [20:03:26] writir: if you use a User-Agent header that gives a way to contact you (e.g. email address), someone will contact you about it if your requests result in too much load [20:04:18] and any block would probably be applied to just the User-Agent value, so don't worry too much about getting blocked [20:05:49] writir: you might want to check out https://en.wikipedia.org/api/rest_v1/ too. [21:08:47] I'm trying to generate a sitemap of one of my Wikis and It doesn't seem to be working. I've got my wikis set up with a giant switch statement (like on the Wiki Farm Manual), and I'm passing "php update.php --conf ../WikiLocalSettings.php" first and then update.php and that does its thing but then when I use: [21:08:47] php /var/www/html/mediawiki/maintenance/generateSitemap.php --fspath=/var/www/html/mediawiki/sitemap/ --server=https://wikiname.domain.com --compress=no" I get an error saying "this wiki is not available" [21:08:54] Does anyone know how I can get around this? [21:12:16] Is the actual error "this wiki is not available"? [21:12:37] Can you run other maintenance scripts fine? [21:12:43] Do you not need to pass --wiki=dbname [21:13:43] This wiki is not available. Check configuration.root@[SERVERNAMEHERE]:/var/www/html/mediawiki/maintenance [21:13:48] is the error [21:15:02] "This wiki is not available." doesn't seem to be an error in mediawiki code [21:17:29] Really? That's the error I'm getting. [21:18:11] It's definitely not [21:18:14] Or at least, not in master [21:18:19] "Check configuration" similarly [21:18:44] What does that imply, then? [21:19:30] Some other custom code? [21:20:27] Pretty sure I haven't added anything so it's unlikely. [21:20:39] https://www.mediawiki.org/wiki/Manual:Wiki_family#Giant_switch_statement [21:20:47] it's clearly in that code there [21:20:47] echo 'This wiki is not available. Check configuration.'; [21:21:01] So yes it is custom code you added [21:21:24] presumably this means that --server doesn't result in $_SERVER['SERVER_NAME'] being set [21:22:27] But looking at the example switch vs the command you're trying to run [21:22:34] You probably don't want the https:// protocol [21:25:20] Okay that worked. It generated a list of sitemaps per namespace at this url https://wikiname.domain.com/sitemap/sitemap-index-wikiname.xml but I get an error when submitting that to Google? is there something further I need to do with it before submitting it? [21:26:40] What is the error? [21:27:12] HTP 404: We encountered an error while trying to access your Sitemap. Please ensure your Sitemap follows our guidelines and can be accessed at the location you provided and then resubmit. [21:27:38] Can you actually view/download the sitemap in your browser? [21:30:18] I can view this https://postimg.cc/tZGgqNLB [21:32:15] is it really wiki.domain.com? [21:32:44] No [21:32:59] and can you load those hz files? [21:33:00] *gz [21:33:08] noting they're not in /sitempa/ [21:33:23] Nope > The requested URL was not found on this server. [21:33:42] I guess that's part of your problem them... [21:33:44] I assume it's because it isn't pointing them to /sitemap/.. [21:34:39] You might want to use --urlpath to generateSitemap [21:34:57] >The URL path corresponding to --fspath, prepended to filenames in the index; defaults to an empty string [21:39:03] Ahh that works! Thank you! [21:41:01] Do I have to submit each namespace URL individually? I've submitted https://spicewars.whiki.online/sitemap/sitemap-index-mediawiki.xml but it didn't discover any URLs but when I submit it individually, it does? [21:45:02] https://postimg.cc/JHq0dHYP [21:46:23] No, the main URL points to the individual namespace files [21:48:23] Keeps throwing a couldn't fetch 404 error? [21:57:18] Nvm it works now!