[11:08:14] hi, id like to ask about language parsing experiments here. wanna check my wiki page against its sources to identify bias, or info which is not supported. please advise [17:59:12] hi gry -- on the question of bias, what aspects are you interested in? as for checking whether info is supported, there was some research about this sort of tooling for Wikipedia (https://www.nature.com/articles/s42256-023-00726-1) but there is no official tool at this time for that task that I'm aware of. there are also some general benchmarks used within the language modeling literature for evaluating content for hallucinations [17:59:12] which could likely be applied to the Wikipedia use-case: https://arxiv.org/abs/2305.14251. one of the challenges when I was playing with this myself was that you also need pretty reasonable text pre-processing pipelines for extracting the specific claim from Wikipedia and cleaning external source HTML (removing boilerplate text etc.). [20:22:16] isaacj: two typical scenarios are, for example: when describing a protest: "a tremendous amount of people showed up" (opinionated words without specifics) [20:23:27] isaacj or describing why the protesters protested (our wages are too low, employer does not provide safe working conditions, etc) without stating responses or official comments from employer ('wages were increased by 5% in last 2 years, why the heck are they protesting') [20:24:22] isaacj would it be possible to know, how, from a sysadmin point of view, this was implemented ? [20:35:14] gry: ahh you might be interested in https://www.mediawiki.org/wiki/Edit_check/Tone_Check then as a project oriented towards that first sort of language bias. the second part that you describe (due weight) is a lot trickier to detect and I don't know of any models that really do it well. you can check out the Seeing like an AI paper in the June 2025 Research Showcase if you're curious to learn more: [20:35:14] https://www.mediawiki.org/wiki/Wikimedia_Research/Showcase#June_2025 [20:37:55] isaacj, this is pretty much what i'm looking for https://en.wikinews.org/wiki/User:Gryllida/sandbox [20:40:37] yes tone chekc looks like it can do one of these tasks [20:41:55] you might consider adding it to the Community Wishlist: https://meta.wikimedia.org/wiki/Community_Wishlist. the reference search piece could be relevant to Wikipedia and other projects as well and some academic researchers have explored assisted writing like this project called STORM: https://storm.genie.stanford.edu/ [21:11:58] isaacj, is there a kickstarter like place where i can add to community wish list and leave a bounty. so if it is coded successfully within a given time period, i provide that payment to the developer who makes it. [21:15:04] gry: I don't think so. interesting idea though