[03:58:07] Something strange seems to have happened from my Wikipedia exports and imports. For example https://en.wikipedia.org/w/index.php?title=Template:Site_active&action=edit compared to https://moasspedia.org/w/index.php?title=Template:Site_active&action=edit [03:58:24] {| class="wikitable" vs <noinclude>{| class="wikitable" [04:16:25] I checked the source XML file for that "Template:Site active" and the XML file shows -> <noinclude>{| class="wikitable" [04:20:17] Maybe one of the runjobs will fix it? [04:29:17] hmmmmmmm, I just ran a test import into dev site with same template, and it worked without issues, so I'm not sure what happened with the production site having an issue. [04:41:11] Oh sweet! I just downloaded the blob data from the database using phpmyadmin and the blob data also shows --> {| class="wikitable" [04:41:24] so it's not that the MySQL data is wrong, but something is not parsing correctly [04:41:40] and I can't seem to use the {{Site active}} template in a page too [04:42:15] not properly anyway, it ends up including showing all the documentation for each usage [04:49:45] lol what? and now it (((almost))) fixed itself. Partially something seems fixed, and I didn't do anything! The edit page doesn't show < anymore, and correctly shows < [04:51:05] but still when I try to use the template and preview, it is still including the documentatin and not working properly (in the production environment), but it worked previewing in the development environment! [04:51:50] I'll just go to sleep and wake up and it'll probably be fixed, lol [05:16:49] Oh wait, I made a mistake? https://moasspedia.org/w/index.php?title=Template:Site%20active&action=edit is still showing "<" [14:26:39] Hi. Is there an automatic tool to login all projects? [14:35:15] Damn! Wake up to the same issue: The markup for some of the imported XML template pages appears/parses incorrectly, showing < instead of < and > instead of > [14:41:36] MySql -> SELECT * FROM `text` WHERE `old_text` LIKE '{| class=\"wikitable\"%|-%|style=\"background: #9EFF9E; vertical-align: middle; text-align: {{{align|center}}}; {{{style|}}}\" class=\"active table-siteactive\"|{{{1|Active}}}%|}%{{Documentation|Template:Table cell templates/doc}}%' [14:42:00] That returns 1 match in the production environment database, as well as the development environment database. [14:43:12] Ah, I suppose I'll ask for help at https://mediawiki.org/wiki/Project:Support_desk [14:49:28] ryzenda: < is equivalent to < https://www.codetable.net/hex/3c [14:51:41] Vulpix, That's what I thought, however, I first noticed an issue when I tried to use {{Site active}} in wiki markup, and it started including the documentation and other strange artifacts in each location. [14:54:53] When I edited https://moasspedia.org/wiki/List_of_ape-made_websites changing all the {{yes}} to {{site active}} and clicked "Show preview" the preview was wonky [15:01:59] mediawiki parses < and > characters, but not < nor < [15:14:30] Vulpix, Yep. I figured as much. I resummarized the issue again here https://mediawiki.org/w/index.php?title=Topic:Wt1rgdbnv57jls30&action=history [15:54:41] Hey hey [15:55:11] I've started working on an Ansible role to install mediawiki, if anyone wishes to test it out to find bugs or contribute to it let me know :) [15:55:11] https://git.laquadrature.net/lqdn-interne/piops-roles/ansible-role-mediawiki [15:55:19] hey, we want to write a !projectwish IRC plugin that can insert new items onto a bulletpoint list on a specific page of our wiki [15:55:31] is there a intended route towards that? some API? [15:55:53] our mediawiki's login is through ldap/saml/keycloak so maybe not very easy to use by that bot [16:00:07] buZz: Special:BotPasswords [16:01:03] you can generate a secondary password for a user account for the purposes of authing against the API, and that secondary password will have limited access according to whichever scopes you grant it [16:01:19] hmm, i wonder how that'll work with saml in place [16:01:35] it should, the bot password login is a completely seperate mechanism than regular client auth [16:02:01] (and only usable via API) [16:02:51] as for how to actually do it, it'll just be the regular edit API to edit pages; you'll grab the current wikitext of the page, add your bullet entry to wherever it needs to go, and save that new wikitext [16:04:28] https://www.mediawiki.org/wiki/API:Edit has more details, including some sample code (which walks through everything except grabbing current page text) [16:05:32] use https://www.mediawiki.org/wiki/API:Revisions for obtaining content [16:05:59] Special:ApiSandbox is helpful for discovery and building API queries as well [20:39:04] I want to debug and modify the MediaWiki PHP files to track down the cause for "" appearing as "<noinclude>" within the MediaWiki processing environment to figure out what is causing the data to be misrepresented for some templates, but not other templates. Which PHP file is a good starting point? [20:43:49] none of them [20:44:15] is your wiki public? can you link to the page showing the issue? [20:45:27] if not, I would try to isolate the issue using Special:TemplateSandbox first so you can see which constructions are causing it to appear. It'll save your sanity compared to attempting to debug the parser [20:52:45] moonmoon, It's public. For example, this shows the issue: https://moasspedia.org/w/index.php?title=Template:Site_active&action=edit [20:52:51] compared to https://moasspedia.org/w/index.php?title=Template:No&action=edit [20:53:52] in terms of using the {{site active}} template on a page, I only have previews with it for now, and nothing saved, but when I convert all the {{no}} to {{site inactive}} in https://moasspedia.org/w/index.php?title=List_of_ape-made_websites&action=submit and preview the page, the issue shows [20:54:08] same issue with {{site active}} and {{site inactive}} (and a few others) [20:54:24] well, your source literally says <noinclude> [20:54:28] so that's your problem [20:54:32] That's the thing [20:54:34] if you want the tag to work properly, it needs to be < [20:54:35] the database says otherwise [20:55:14] have you tried changing it to <> in the editor and saving? [20:55:37] I haven't because for the Template namespace I am not editing any of those, and only importing directly from XML exports from Wikipedia [20:56:16] well I'd try that first [20:56:38] and as I described in https://mediawiki.org/wiki/Topic:Wt1rgdbnv57jls30 I tried to track down the source data [20:58:30] I'm not too sure about the mapping of the MySQL database exactly in terms of writing an SQL SELECT statement to return the latest revision for a specific page_title but I manually glanced through the database to track it down and downloaded the blob data [21:01:38] all I can say is that the wiki believes that the db contains literal "<" strings in that page [21:01:52] so editing the page to change them into < and > should work [21:04:37] I'm gonna try to prepare a SELECT statement to reproduce what I manually identified when exploring the database, to confirm what I previously found [21:06:06] Otherwise previously I used: SELECT * FROM `text` WHERE `old_text` LIKE '{| class=\"wikitable\"%|-%|style=\"background: #9EFF9E; vertical-align: middle; text-align: {{{align|center}}}; {{{style|}}}\" class=\"active table-siteactive\"|{{{1|Active}}}%|}%{{Documentation|Template:Table cell templates/doc}}%' [21:06:23] well that wouldn't find it [21:06:55] It did find it! [21:07:04] maybe an older rev, but not the current rev [21:07:18] because you're explicitly excluding the badly-formatted version [21:08:21] https://i.imgur.com/42lOkV0.png [21:08:43] that may be a possibility, but I will check to make sure, after I figure out the correct SELECT statement [21:08:48] you didn't disprove anything of what I said [21:08:59] first you'll want to figure out what the correct old_id to use is [21:11:26] SELECT content_address FROM content JOIN slots ON slot_content_id = content_id JOIN revision ON rev_id = slot_rev_id JOIN page ON page_latest = rev_id WHERE page_title = 'Site_active' AND page_ns = 10 [21:11:36] will give you a string like 'tt:12345321' [21:11:46] the number is the old_id corresponding to the current revision [21:12:10] so you can then SELECT * FROM text WHERE old_id = 12345321; [21:13:07] #1054 - Unknown column 'page_ns' in 'where clause' [21:13:09] ryzenda: can you explain why that template has all revisions duplicated? One with the encoded tags and one without them https://moasspedia.org/w/index.php?title=Template:Site_active&action=history [21:13:29] er, page_namespace, sorry [21:14:43] Vulpix, Oh, good point, i didn't even pay attention to that yet. I don't know why they are duplicated. Looking into it [21:15:01] Each revision is encoded/unencoded in succession [21:15:51] (but tl;dr is that the current revision of the text in the db is wrong, which is why things aren't working. The fix is to edit the page to be not-wrong, it isn't a bug with the parser) [21:16:27] yeah, not a bug in the parser, the content was explicitly encoded there when it was imported [21:16:40] From bash command line I've been using this to import each page individually --> for file in ../../../downloads/xml_exports/Wikipedia.enwiki/Template/*;do status=-1;while [ $status -ne 0 ]; do echo $file; php importDump.php --conf ../LocalSettings.php --no-local-users --username-prefix Wikipedia "$file";status=$?;done;done [21:17:09] because my webhost Dreamhost sometimes kills processes citing using too many resources, so when an import fails, the import will restart [21:27:17] Aha, so glancing at https://moasspedia.org/w/index.php?title=Template:Site_active&action=history I see latest revision oldid is 745169 and old_text value for that is https://dpaste.org/gpGwVVe/raw which does not even look like the right content [21:28:03] but maybe that oldid # is not the same as the old_id value in the text table [21:30:04] In the page table, the value for page_latest for the "Site_active" page_title is 716908 [21:36:09] it is not [21:36:23] you have to go from revision id -> content id via the slots table [21:36:40] and the content table contains the content address which is the text table's old_id [21:36:53] aka the query I posted above [21:44:21] I get it. The query above had an error, and I didn't figure out how to fix it right away, but working on it I got --> SELECT content_address FROM content JOIN slots ON slot_content_id = content.content_id JOIN revision ON rev_id = slots.slot_revision_id JOIN page ON page_latest = revision.rev_id WHERE page_title = 'Site_active' AND page_namespace = 10 [21:44:26] value is tt:670176 [21:53:19] k, so the text.old_id value in question is 670176 [21:56:20] moonmoon, yep, and I confirmed, you were correct, old_text is https://dpaste.org/OnOsECK/raw [21:57:40] Oh crap! lol [21:58:06] SELECT * FROM `text` WHERE `old_text` LIKE '%<noinclude>%' ORDER BY `old_id` DESC; // 8,356 results [21:58:26] I would look into your import process as the likely culprit [21:59:18] also many of those 8.3k entries are likely old revisions so it doesn't matter too much if they're wrong [21:59:50] does confirm the fix however: edit the page to fix the text. And look into the files you're importing to see if you can figure out why it's importing bad text in the first place [22:01:08] I mentioned in the MediaWiki Discord server, `~/downloads/xml_exports/Wikipedia.enwiki/Template$ grep -r "<noinclude>" ./` shows 0 results [22:05:38] ryzenda: actually, they'll be double-encoded, since & is a special character for XML [22:05:54] maybe &#x3c;noinclude&#x3e [22:10:22] SELECT count(*) FROM text JOIN content ON old_id = Replace(content_address, 'tt:', '') JOIN slots ON slot_content_id = content.content_id JOIN revision ON rev_id = slots.slot_revision_id JOIN page ON page_latest = revision.rev_id WHERE old_text LIKE '%<noinclude>%' [22:10:27] That shows 327 [22:11:06] I'm not sure if the SQL statement is correct, but I tried to find all the matches in the text table for all pages that page_latest is listed in the page table (but I don't know how to include that part) [22:39:59] I'm trying to add a mustache template above the body tag in the vector skin, does anyone know which file I would find this in to add it to? I've tried skin-legacy.mustache but the highest this gets me is above the mw-page-base section, I want to go above the body tag really. [23:00:20] Anyone any ideas?