Archive:SMWWriter


 * This page describes an extension developed in 2010, which was not released and appears to have been abandoned.

Semantic MediaWiki Writer (SMWWriter) is an extension for Semantic MediaWiki (SMW) that provides an API and tools to edit facts within the wiki. This documentation describes the why, how, and what of this extension.

Background
SMW extends the well-known MediaWiki wiki software, which powers a few thousands wikis all over the world. SMW allows users to add metadata to each page, describing the entity this page is about, and then to query the wiki knowledgebase that is collaboratively created this way. This improves the consistency of the wiki content, decreases the maintenance, and allows users to access the knowledge within the wiki from external applications using web-standards like RDF or JSON.

Adding, changing, or removing the metadata inside an SMW is not easily performed by a machine, due to the growing number of annotation schemes and practices. Whereas it is possible to use the existing MediaWiki-API to access the current text and change it, changing a single fact requires the user to understand how annotations are performed within SMW. SMWWriter offers an API to do exactly that, and offers an abstraction layer over the existing MediaWiki API to simply edit the wiki's knowledgebase.

Philosophy
SMWWriter offers a "best-effort" functionality. In general the given task -- change metadata -- is a challenge that requires human-level intelligence.

Instead of guaranteeing certain functionality, SMWWriter will try to achieve the task, if possible. Programmers are advised to program defensively, i.e. to check if a certain change has actually been done, etc., and to, in general, distrust the results of the API.

However, we expect that most situations will allow for the API to work without any problem, and we are always interested in improving the API (see development section below). The API is developed in such a way to offer a stable interface, and any improvements can happen behind the scenes.

Installation
SMWWriter installs like any other MediaWiki extension.

Requirements
We have tested the extension using the following system:
 * MediaWiki 1.16
 * Semantic MediaWiki 1.5
 * PageObjectModel SVN revision 63243

Note that the extension may work with older (or newer) versions, but it has not yet been tested with them. If you test SMWWriter with other settings, please update this documentation appropriately.

Instructions
svn checkout http://svn.wikimedia.org/svnroot/mediawiki/trunk/extensions/SMWWriter/ include_once("$IP/extensions/SMWWriter/SMWWriter.php");
 * If using a release:
 * Download the extension from XXX
 * Decompress it in your MediaWiki's extensions/ folder
 * If using the SVN version:
 * From within your MediaWiki's extensions/ folder call
 * In both cases:
 * Check the accompanying README file for any new information
 * add the following line to your MediaWiki's LocalSettings.php (after you included SMW):

Now SMWWriter should be installed. Check out Special:Version to see if SMWWriter is listed.

API documentation
There is actually not one, but two APIs: a web-service based API to be used by external tools and by AJAX-based extensions to the wiki, and the internal API that offers the same functionality.

Web-service based API
The web-service based API extends the existing MediaWiki API. It provides two new actions, smwwriter and smwwritable. The interfaces are described just as all other MediaWiki web-service based APIs are described.


 * smwwriter : requires a POST action (since you ask the wiki to change resources). It takes the following parameters:
 * title: Title of the page to modify
 * token: Edit token. You can get one of these through prop=info
 * add: Annotations to add. The format is like simple wiki annotations, i.e. property::value
 * remove: Annotations to remove. The format is like simple wiki annotations, i.e. property::value . If value is '*' then all values of that property will be removed. If remove is '*', then all annotations will be removed. If remove is empty, then content simply gets added and no metadata is removed
 * flags: Flags for the edit process. The following flags exist: ATOMIC_CHANGE, CHANGE_TEXT, EDIT_MINOR, IGNORE_CONSTANT. The flags are described below. The flags are connected with the | symbol
 * summary: Edit summary for the change history


 * smwwritable : offers a list of all metadata on a page that SMWWriter can change. smwwritable only has a single parameter:
 * title: Title of the page

Internal API
The internal API consists of one class with four public methods (including the constructor). The SMWWriter class has a constructor and three further methods, the update, getUpdateable, and getError methods.

An SMWWriter object needs to be created using the constructor with a single, required parameter, the Title object representing the page to be changed. An SMWWriter object can only be used for this one page. In order to change a different page, a new SMWWriter object needs to be created.

The update method offers no output and the following input parameters:
 * remove: an SMWSemanticData object that contains the metadata that should be removed. If the value of a property-value pair is created with false, all the property-value pairs with this property will be deleted. If there are no property-value pairs, all property-value pairs will be deleted. If the subject of SMWSemanticData has been created with false, no property-value pairs will be deleted and the facts in add will be added instead.
 * add: an SMWSemanticData object that contains the metadata that should be added. Note that if it requests to add metadata that is already there this will be recognized as superfluous and ignored.
 * editsummary: Edit summary for the change history. If empty, a text will be autogenerated. Advised not to leave empty.
 * flags: an integer bitfield using the following constants: ATOMIC_CHANGE, CHANGE_TEXT, EDIT_MINOR, IGNORE_CONSTANT.
 * ATOMIC_CHANGE: if the API thinks that it cannot perform the complete request, it will not perform any of it, e.g. if it is requested to remove a specific property-value pair that does not exist it will not perform the request but rather set an error
 * CHANGE_TEXT: if possible we will change the text. Usually the API is defensive with regards to changing the text, i.e. it rather removes annotated links from the text and adds a #set statement instead. If CHANGE_TEXT is set, it will instead change the text in the annotated links. This may lead to wrong grammar
 * IGNORE_CONSTANT: if an ATOMIC_CHANGE is requested, the request will fail if it asks to remove a property-value pair that the API thinks it can not remove. If IGNORE_CONSTANT is set, then such constants (i.e. not updateable parts of the metadata) will not be required to be changed for an ATOMIC_CHANGE

The getUpdateable method has no input parameters and returns a SMWSemanticData object that includes all the metadata that SMWWriter thinks it is able to change it.

The getError method can be called to get a string to describe what and if something has gone awry. In case the API thinks everything is OK, it will return an empty string. The method has no input parameters.

Examples
This section lists a few examples of what you can do with the new API, and how it is done with the internal API and the webservice API. Note: the webservice calls need to be using the POST method. You also have to be careful in doing the proper URL encoding.

Add a new book to the wishlist of Johnny. Do not remove anything from the page.

Using the internal API: $page     = SMWWikiPageValue::makePage("Johnny", NS_MAIN); $writer  = new SMWWriter( $page->getTitle ); $add     = new SMWSemanticData( $page ); $remove  = new SMWSemanticData(SMWWikiPageValue::makePage(false, NS_MAIN)); $property = SMWPropertyValue::makeUserProperty("wishes"); $value   = SMWDataValueFactory::newPropertyObjectValue($property, "The End of Mr Y"); $add->addPropertyObjectValue($property, $value); $writer->update( $remove, $add, "Adding Johnny's new wish" );

Using the webservice API: /api.php?action=smwwrite&title=Johnny&add=wishes::The End of Mr Y&summary=Adding Johnny's new wish&token=token

Replacing the prize for the book, whatever it was before:

Using the internal API: $writer  = new SMWWriter( Title::newFromText("The End of Mr Y") ); $add     = new SMWSemanticData(SMWWikiPageValue::makePage("The End of Mr Y", NS_MAIN)); $remove  = new SMWSemanticData(SMWWikiPageValue::makePage("The End of Mr Y", NS_MAIN)); $property = SMWPropertyValue::makeUserProperty("prize"); $value   = SMWDataValueFactory::newPropertyObjectValue($property, "14,80 €"); $add->addPropertyObjectValue($property, $value); $value   = SMWDataValueFactory::newPropertyObjectValue($property, false); $remove->addPropertyObjectValue($property, $value); $writer->update( $remove, $add, "Changing the prize" );

Using the webservice API: /api.php?action=smwwrite&title=The End of Mr Y&add=prize::14,80 €&remove=prize::*&summary=Changing the prize&token=token

And after you bought the book for Johnny, you may want to remove it:

Using the internal API: $writer  = new SMWWriter( Title::newFromText("Johnny") ); $add     = new SMWSemanticData(SMWWikiPageValue::makePage("Johnny", NS_MAIN)); $remove  = new SMWSemanticData(SMWWikiPageValue::makePage("Johnny", NS_MAIN)); $property = SMWPropertyValue::makeUserProperty("wishes"); $value   = SMWDataValueFactory::newPropertyObjectValue($property, "The End of Mr Y"); $remove->addPropertyObjectValue($property, $value); $writer->update( $remove, $add, "Removing Johnny's new wish" );

Using the webservice API: /api.php?action=smwwrite&title=Johnny&remove=wishes::The End of Mr Y&summary=Removing Johnny's new wish&token=token

And finally, if you want to remove all the current existing metadata from Johnny (for whatever reason), you would do it like the following:

Using the internal API: $writer  = new SMWWriter( Title::newFromText("Johnny") ); $add     = new SMWSemanticData(SMWWikiPageValue::makePage("Johnny", NS_MAIN)); $remove  = new SMWSemanticData(SMWWikiPageValue::makePage("Johnny", NS_MAIN)); $writer->update( $remove, $add, "Removing Johnny's metadata" );

Using the webservice API: /api.php?action=smwwrite&title=Johnny&remove=*&summary=Removing Johnny's metadata&token=token

I hope the examples are helpful in order to get started writing tools using the API. Please add more if you think something isn't clear.

Development
External developers are explicitly encouraged to contribute (actually, I wouldn't even mind if they took over the lead on the further development of this extension -- Denny). This section outlines some basic thoughts on the further development of SMWWriter and on current development decisions. The source code contains further spots that are marked with TODO and that would require further improvement.

Why not in the core?
Our current aim is to keep SMW a lean and small extension. Our first thought was to have SMWWriter be a proper part of SMW core, but as we started developing it we noticed that the code got bigger than expected. Also it was decided that the dependency on PageObjectModel was not desired for SMW core. If the dependency can be removed and the code turns out to be a manageable size, SMWWriter may be merged into the core, but this is not a priority currently.

Dependency on PageObjectModel
PageObjectModel (POM) is an extension to MediaWiki that was developed in order to enable a web-service based editing of templates. Unluckily the maturity of POM was first overestimated. In order to develop SMWWriter we had to extend POM considerably, but we still do not regard POM as a mature piece of software as it is now (sorry, Sergey).

There are three possible options to deal with this dependency in a medium timeframe. We list them in what we consider is an order of increasing desireability:
 * 1) re-implement the POM function within SMWWriter properly and thus remove the dependency. Basically increases redundancy, but makes SMWWriter a stand-alone extension.
 * 2) take ownership of POM and improve it considerably.
 * 3) actually replace the dependency of POM with calls to the internal MediaWiki parser model of PPNodes and PPFrames and PPWhatevers, understand the MediaWiki parser and use its model instead. This may require to extend the parser with a wiki-text serializer and methods to actually change the values. This also will remove the inherent redundancy of POM with regards to the parser's own DOM, and make SMWWriter solely depend on MediaWiki's core technology. This is the hardest, but also the most rewarding path.

We hope that someone will pick up on this challenge.

Smarter functions and best effort
Currently, the update methods themselves are not terrifically smart and could be vastly improved. This is especially true for the updatePropertyValues method that solely deletes the property values and then add new ones instead of replacing the values within the already existing annotations. Furthermore, text with annotations may be treated more cleverly. Imagine a list in an article on the European Union: * member::France * member::Germany * member::Italy etc. Requesting to add member::Romania for the most recent expansion could, instead of adding a #set call somewhere in the article, add a further bullet point with the new member. It should be obvious that this is not a trivial task. :)

SPARQL and SMWWriter
The current SMWWriter offers a proprietary MediaWiki-based web-service API for changing the wiki-based knowledgebase. It would be much, much nicer if additionally SMWWriter would accept SPARQL requests. Hereby we mean the new SPARQL standard that is currently being developed under the auspices of the W3C and that also allows updates. These updates can be parsed, checked if they fit into SMWWriter, rewritten and executed, which would make it easier to make SMW a component within the Semantic Web ecology.

This is especially true for using external URIs, i.e. it would be interesting to be able to add the following triple:    and meanwhile the wiki checks if it knows about the resources that are mapped to the given external URIs, translates them to the local entities, and then adds the given metadata on the appropriate page (i.e. it could be city::Stuttgart on the page Denny )

UI extensions on top of SMWWriter
Based on the API it should be easy to build new extensions that use the API to display, change, update, and synchronize the wiki knowledgebase. Here are some possible tools we expect to be developed: We expect that small such extensions can be maintained within the SMWWriter directory on the WikiMedia SVN, just like the different Semantic Result Formats are mostly gathered in one extension. These would be both example implementations as well as small useful tools in themselves.
 * PIM synchronizer for address books, birthdays in calendars, tasks
 * Synchronizer for data like stock exchange value, weather information, currency conversion
 * Knowledge apps that display information of a specific type and allow to quickly change them, e.g. an App that displays all your books and allow you to quickly skim through them and quickly change the metadata on them
 * External tools that communicate with the SMW and use it as a knowledge backend, e.g. a multimedia player that pulls and displays additional information about played tracks
 * A UI within the wiki similar to Semantic Forms or the Halo Annotation Sidebar, that allows to quickly add and change metadata