Wikivoyage is underrated and that was not helped by the acrimonious split with Wikitravel (which was acquired a predatory marketing company), but it finally seems to be pulling ahead.
> I wouldn't be surprised if some Wikipedia editors balk at their volunteer work being actively marketed and reformatted for ease of LLM training
As someone who avidly edited Wikipedia for 6-8 years, I am happy to see my volunteer work used for LLM training. I also agree some other editors likely aren't.
Given that all Wikipedia editors have explicitly consented to their content being released under the Creative Commons Attribution-ShareAlike 4.0 License, they don't get a choice about their content being used for any purpose.
Redistribution of content is an entirely different matter, and the legal status of copyrighted material in relation to LLM training is an open issue that is currently the subject of litigation.
> "it is important to note that Creative Commons licenses allow for free reproduction and reuse, so AI programs like ChatGPT might copy text from a Wikipedia article or an image from Wikimedia Commons. However, it is not clear yet whether massively copying content from these sources may result in a violation of the Creative Commons license if attribution is not granted. Overall, it is more likely than not if current precedent holds that training systems on copyrighted data will be covered by fair use in the United States, but there is significant uncertainty at time of writing."
The new Wikimedia Enterprise APIs facilitate attribution. For example, the "api.enterprise.wikimedia.com/v2/structured-contents/{name}" response [2] includes an "editor" object in a "version" object. So the Wikipedia editor who most recently edited the article seems quite feasible to attribute. ML apps could incorporate such attribution in their offering, and help satisfy the "BY" clause in the underlying CC-BY-SA 4.0 license for Wikipedia content.
> if you want to have really structured and semi-reliable information you will probably have to rely, at some point, on something like Wikipedia meta-information (DBpedia).
Wikidata is also worth considering for that task. It is:
* Directly linked from Wikipedia [1]
* The data source for many infoboxes [2]
* Seeded with data from Wikipedia
* More active and integrated in community
* Larger in total number of concepts
Wikidata also has initiatives in lexicographic data [3] and images [4, 5].
On the subject of Cyc: the CycL "generalization" (#$genls) predicate inspired Wikidata's "subclass of" property [6], which now links together Wikidata's tree of knowledge.
Gage's work is probably not prohibitively expensive in terms of money. A few plane tickets to Iowa and New Hampshire every four years, lodging in each for maybe a few days. He also attends Comic-Con every year. He has a good DSLR and lens kit, but that's a one time cost. If he had a paid summer internship in accounting, a side job during the academic year, and supportive parents this all seems doable for a middle class college student attending a public university with in-state tuition.
The vast reach of his photography is likely a sufficient incentive for Gage to invest all that time and significant-but-not-immense money. I hope Gage continues his excellent work.
>>Gage's work is probably not prohibitively expensive in terms of money.
"between states to more than 40 speaking engagements."
"I traveled to nearly every part of the country to cover his political events,”
"Skidmore is hot on the campaign trail again, toggling his time between New Hampshire, Iowa, and Arizona"
Prohibitively expensive is a relative term, but it is hard to imagine this not costing 10s of thousands of dollars and being out of the reach of most high school and college students.
Maybe not in this particular case, but you could definitely make a case that photographing political candidates as that pass within range of a day-trip (or maybe a two-day-trip) of where you live is just the cost of time, gas, (food,) and lodging.
Someone could do something similar using such a model.
> * it is hard to imagine this not costing 10s of thousands of dollars and being out of the reach of most high school and college students.*
I would be surprised if Gage and his parents have spent more than $10,000 of their personal money on this hobby. Again, cost near that range is certainly significant, but not monetarily immense for a middle class kid with a consuming hobby and supportive parents over the course of 7+ years.
Gage has been frugal in his choice of college, and gets funding from GoFundMe campaigns. He also seems to have had side jobs. Simply choosing to attend a community college and then an in-state public university as Gage has done -- rather than a private university for 4 years -- is probably enough to defray a huge portion of his hobby's cost.
I suspect Gage is also frugal in his means of travel and lodging. A sibling comment mentions the possibility of photographing candidates that come within a day or two trip of home. I imagine that accounts for most of Gage's photography.
Consider this note from [2]: "Skidmore is a 19-year-old student at Glendale Community College in Phoenix and a freelance graphic designer. A Ron Paul supporter, he began photographing politicians when he was living in Terre Haute, Indiana, attending events held by Rand Paul during his successful 2010 Senate run in Kentucky." The drive from Terre Haute, IN to Lexington, KY is about 4 hours. That's completely doable in a day trip. I've driven 4 hours each way in day trips for similar free culture pursuits. It costs about $80 for gas and food.
> "between states to more than 40 speaking engagements."
Travel among multiple US states to attend 40 speaking engagements over the course of 7 years is not necessarily a major financial burden, even for someone Gage's age.
> "I traveled to nearly every part of the country to cover his political events"
"Part" can be pretty general. One could have covered events in Arizona, Iowa, New Hampshire and, say, Virginia and say one has traveled to nearly every part of the country -- the American West, Midwest, Northeast and South.
> "Skidmore is hot on the campaign trail again, toggling his time between New Hampshire, Iowa, and Arizona"
I think it's much more likely that Gage has been to New Hampshire and Iowa each once or twice in the 2016 campaign season, rather than flying out every weekend or so like a high-level political operative or corporate executive from his Arizona State University dorm room.
New York will feature a talk about Wikidata, how to query it with SPARQL, and how we are integrating it with Wikipedia and pushing forward the Semantic Web. Other NYC talks include things like "Git-flow approach to collaborative editing", "Copyright and plot summaries", and "Automated prevention of spam, vandalism and abuse". We will be linking up with San Francisco and likely some other cities for a global teleconference at 4:00 - 5:00 PM ET (21:00 UTC).
The Wikidata taxonomy is basically the successor to Wikipedia's category tree. It not only irons out language-based differences (e.g. the category tree being different among Chinese, Spanish, English, etc. Wikipedias), but also captures the idea of generalization through a more semantically meaningful relation. This Wikidata concept tree is constructed with "subclass of" (P279) [1], a property that expresses the proposition "all instances of these items are also instances of those items". The goal is to have a subsumption hierarchy that classifies all human knowledge.
There's an RDF/OWL export of the Wikidata taxonomy available at [2] as wikidata-taxonomy.nt.gz, which can be explored with Semantic Web browsers like Protege [3].
Another fundamental relation -- "part of" (P361) [4] -- expresses mereological relationships. For (oversimplified) example: "iris part of eye", "eye part of head", "head part of body", etc. Both "subclass of" and "part of" are transitive.
A separate comment of mine in this discussion [5] describes how to traverse the "subclass of" tree in the Wikidata UI and a third-party tool called Wikidata Generic Tree. The same principle applies to the "part of" tree. The latter gets less attention, but is also quite interesting.
I am very excited about the potential knowledge engineering possibilities opened up by this large structured datasets.
I believe that at the very least we're going to have within a generation a machine-generated ontology to rival Kant and Aristotle. Then we'll have to figure out if this tells us more about how we've digitally organized the knowledge we have or whether it does in fact reveal something about reality and being.
Besides 'subclass of' and 'part of' are there any other taxonomic ways for concepts to relate to other concepts? There are parallels here of course with object-oriented-programming. It's funny, I only within the last year or so started reading up on mereology[0] but as soon as one starts thinking about concepts and there relationships one ends up there eventually. 'part of' is like encapsulation. 'subclass of' is like inheritance. Is there more?
'Instance of' and 'subclass of' provide Wikidata with a way to express the basic philosophical notion of type-token distinction [3]. For things that are a subclass of something like 'material entity', all instances are physical objects that have a unique location in space and time.
Not all instances are spatiotemporal particulars, though. For example, one might say "Homo sapiens instance of taxon", where taxon is a metaclass, i.e. a class in which the instances are classes. (Here 'taxon' would not be a subclass of 'material entity' -- i.e. taxa are information artifacts, not physical objects.) Support for this kind of "punning" via metamodeling is a major feature of OWL 2 DL [4].
If this sort of thing interests you, definitely take a look into Wikidata [5]. The project will be a sea change for several key features in Wikipedia (e.g. infoboxes), and will likely be a main hub of the Semantic Web.
Fantastic, I've read through your entire comment history :)
I'm familiar with OWL and RDF. I've been using Sparql and DBPedia, I'll switch to Sparql and Wikidata if you think that's the way to go. How do you see the overlap between DBPedia and Wikidata?
I'm concerned that there's going to be knowledge-grab by corporations and (perhaps) government entities. I fear that the knowledge graphs inside the big G and FB and Yandex and Apple and MS and so on to power their search engines and personal assistants will be orders of magnitude more sophisticated and complex and comprehensive that what will be available to open access research. Witness Freebase. Are my fears misplaced would you say?
I've read that SEP article, I've also read a good bit of Peirce's original journal article. As it says in SEP "It should be mentioned that for Peirce there is actually a trichotomy among types, tokens and tones,[...]" - I think it's amusing that basically everybody ignores the triadic distinction that Peirce claimed to be the case for a dualistic type/term distinction.
I'm looking forward to going through your tutorial quill in hand and pot of ink at the ready.
Regarding Wikidata and DBpedia: to my understanding the latter gets much of its content by scraping Wikipedia infoboxes. Wikidata will increasingly provide data for those infoboxes, and thus DBpedia.
Regarding your fears: I don't share them. Wikidata will greatly enhance the accessibility of knowledge for open access research.
Wikidata's new SPARQL service is probably the most useful topic in this tutorial for software developers and anyone interested in the Semantic Web. It allows one to query the vast, free knowledgebase that backs Wikipedia -- almost 15 million entities and over 70 million statements.
Yes. Kian and WikiBrain are two such projects. Kian is an artificial neural network designed to serve Wikidata, e.g. for classifying humans based on content in Wikipedia [1, 2]. WikiBrain uses Wikidata to recognize the type of relationships or connections between Wikipedia concepts [3, 4].
I suspect larger applications of Wikidata in AI will follow. For example, as of 2010, IBM Watson acquired at least some of its content from DBpedia and YAGO [5], which ultimately derive much of their content from scraping Wikipedia's infoboxes and category system. Now, come 2015, Wikidata is supplying data for some Wikidata infoboxes, and the proportion of infoboxes that pull from structured data in Wikidata will increase over time. And I expect Wikipedia's category system will gradually be supplanted by Wikidata's more expressive property system over time.
Thus, I imagine Wikidata will form a semantic backbone for Q&A systems like Watson in the future.
The Wikidata development team's work is funded through donations by the Allen Institute of Artificial Intelligence, Google, the Gordon and Betty Moore Foundation, and Yandex [6]. So organizations with an interest in AI see potential in Wikidata.
Is Wikidata only for notable data or any data? If only notable data is allowed, then how is that enforced? Who decides if some piece of data is notable?
Can data be permanently, unrecoverably deleted, or is it more like Wikipedia where you can usually go back to see text that was deleted (in older versions of an article)?
The way that units are being handled is troubling. Is the plan to assign a unique integer to every unit that's ever been used? That's a long list of integers.
Wikidata is only for notable data, but the notability threshold is much lower than that for Wikipedia. The criteria for notability are described at [1]. For example, we might add items for all known pathogenic genetic variants, but likely would not have an item for the fire hydrant on your street.
For things not notable enough for Wikidata, interested users could install a local instance of Wikibase [2], the software that runs Wikidata. Wikidata editors and administrators determine what is notable, and have places like [3] to discuss questionable cases.
> Can data be permanently, unrecoverably deleted, or is it more like Wikipedia
Wikidata works like Wikipedia in that regard. Previous versions of a given item or property are almost always viewable (and recoverable) through the History tab [e.g. 4]. In extraordinary cases, like a vandal posting sensitive information about a person, data can be hidden from normal view and/or actually deleted.
> Is the plan to assign a unique integer to every unit that's ever been used?
Basically yes, to my understanding. How many units do you think exist? We already have items for many units, e.g. meter, micrometer, nanometer, foot, yard, bit, byte, , gigabyte, etc. I can see how this implementation might seem naive; e.g. perhaps we could represent one standard metric (like meter or byte) and handle factor conversions of scale (kilometer, millimeter, etc.) through some mechanism we don't have in place right now. Consider also asking about this on the "Contact the Development Team" page [5].
It's just 1 order of magnitude, if we're comparing the same language. English Wikipedia has 19,339 articles on philosophy [1]. (Anyone know if there's a resource comparable to SEP in a language other than English?)
If we're talking all philosophy articles in all 291 Wikipedias [2], and (generously) assume that the average Wikipedia has 10% as many philosophy-related articles as English Wikipedia, that's 19,339 * 0.10 * 291 = 562,764 philosophy articles on all Wikipedias. That's 3 orders of magnitude more than SEP's 1,500 articles -- although that's not really comparing apples to apples as we're comparing many languages to one.
