Topic.topicContent

There are two content fields: Topic.topicContent and TopicVersion.versionContent. Is Topic.topicContent containing the current version? Mike 24-Jul-2007 05:21 PDT

Usually. That field is there as a convenience so that pages like view-topic-include.jsp don't need both a Topic and a TopicVersion object in order to display topic content. Most of the time topicContent will contain the latest version content, although when viewing topic history it would be set to the content of the version being viewed. -- Ryan 24-Jul-2007 09:17 PDT

Alternative parsers

Archived from the Feedback page:

Prior to starting work to support Mediawiki templates I started looking into whether another technology might make the parser simpler. The problem I'm running into with JFlex is that parsing is very linear, so if the parser matches something like ~~~~ there is no easy way to replace that with (for example) [[User:wrh2|Ryan]] 03-Oct-2006 14:57 PDT and then re-parse. This is an especially import issue for templates, where the code may need to perform multiple levels of parsing when faced with something like {{template1|foo={{template2|bar={{template3}}}}}}

Thus far I've looked at:

• JavaCC. The input file format is fairly difficult to follow, but this seems to offer the required functionality. Anyone have any experience? Are the input files really as difficult to work with as they appear to be?
• Axel's parser. This might be a good option since it already includes a lot of useful functionality. The downside is that Axel would need to give an OK to integrate it into JAMWiki, the Coding Style is somewhat different from current JAMWiki code, and we'd need to figure out who would be the "official" maintainer of the code. If JAMWiki uses a parser that can't evolve rapidly (ie either Axel or myself can change it) then that would be a problem.
• Antlr. I didn't look into this one too closely. Anyone have any experience with it?
• Keep JFlex. JFlex can be made to work, I'm just wondering if other options might be preferable.
• Something else?

If anyone has experience with these sorts of technology, feedback would be appreciated. -- Ryan 03-Oct-2006 16:31 PDT

Having looked into things a bit more, I think I'd lean towards adopting an approach similar to Axel's parser, or even to incorporate Axel's parser outright. The current JAMWiki parsing code is too complex and too inflexible. Axel's approach allows custom tags to be incorporated fairly easily, and is fairly easy to understand. I'm not sure if Axel wants to continue maintaining his parser as a separate project or not, so I'll wait to hear his thoughts. -- Ryan 04-Oct-2006 00:54 PDT

How does AnsiData/QueryHandler write Topic.content?

Moved from the Feedback page:

When saving new Topic content from an external resource TopicSpaces using the internal database, topic content got saved and the page was created. When I moved to MySQL, the Topic itself gets saved, but not the content. Clicking on a recent changes link opens an edit page. Something changed.

I use this code

WikiBase.getDataHandler().writeTopic(topic, topicVersion, parserDocument, true, null);

which is driven by this code

ParserDocument parserDocument = Utilities.parseSave(parserInput, contents);

I do not see any reference to Topic.getTopicContent() except to drive Category setup. I do not see ParserDocument.getContent() anywhere. What am I missing? -- Jack 17-April-2007 13:44 PDST

I think you're looking for AnsiQueryHandler.insertTopicVersion() and topicVersion.getVersionContent(). The content is stored with the version, and the topic table then contains a pointer to the current version. And yes, there is probably a lot of room for code cleanups, documentation, and simplifications :) -- Ryan 17-Apr-2007 20:38 PDT

Thanks! What, however, is different from MySQL to the HSQLDB way of handling save? It sounds like I need to explicitly call saveVersion or something. -- Jack 18-Apr-2007 0729 PDT

I should point out that the code I use to save a topic also includes the following, all of which was adapted from org.jamwiki.servlets.EditServlet:

TopicVersion topicVersion = new TopicVersion(user, "TopicSpaces", "New TopicSpaces topic", contents);

topicVersion.setEditType(TopicVersion.EDIT_NORMAL); -- Jack

Provided writeTopic() is used to write to the database then there shouldn't be any differences between databases - I test with Postgres, MySQL and HSQL on a regular basis and haven't encountered any problems. Do you have any error messages in the logs, or have any other files been modified? -- Ryan 18-Apr-2007 09:08 PDT

What I now know is that JAMWiki is properly writing the data. That I am not apparently properly retrieving the data was first thought to be associated with the toLower aspect of AnsiQueryHandler's topic fetching. I am now busy instrumenting code to see why I cannot retrieve that which appears to be properly saved. -- Jack

Provided lookupTopic() or lookupTopicVersion() are used to retrieve the topic then you should probably have all of the data you need available in the Topic and TopicVersion objects that are returned. If anything fails to get initialized please let me know and I can investigate, or alternatively if you find a bug feel free to fix it directly in Subversion (you have access, right?). I may have a little bit of time to investigate any issues tonight, but that will depend on how many active brain cells remain after getting off work. -- Ryan 18-Apr-2007 12:49 PDT

At this time, I am able to determine that TopicSpaces can see the saved data, but JAMWiki cannot, at least in terms of generating a URL to display it. That is to say, if I perform a search on the saved data, JAMWiki will find the proper Topic(s) that contain(s) that saved data, but it continues to think that the page itself needs editing; it will not open the page directly. A typical page it will not see carries the topic name: AirSubjectPropertyTypeDescription_0, which leads me to wonder if the trailing _0 is somehow changed by the wiki parser when defining the URL to use. Trailing dash numbers identify version numbers of certain content in TopicSpaces. Side note: I started writing this comment at the same time Ryan was just saving his comment above, so this final edit is added during content resolution. Yes, I got one of those big red notices that says the content needs resolution! As far as my instrumentation tells me, everything seems to be initialized properly. RecentChanges properly lists the topic names, but they display in red, which suggests that the parser (somehow) failed to find the content when setting up the RecentChanges page. As a final check, pasting the topic name into the browser bar confirms the "topic does not exist" even though search can find its content. More instrumentation coming up... -- Jack

Note that writeTopic() does more than just insert or update a record - for performance reasons it also updates cached values and the search index. If a topic is cached as "non-existent", it will show up as a red link until the cache record is updated. All of that code is consolidated in writeTopic() to make it easier to integrate, but if writeTopic() isn't used there will be problems. If records are created or inserted from another source then that source would need to perform the additional work that writeTopic() performs, but be warned that using anything other than writeTopic() may break with future JAMWiki versions. -- Ryan 18-Apr-2007 14:04 PDT

Interesting observation -- loading this page tagSlotTypeDescription_0 gets this trace from AnsiDataHandler.lookupTopic: AnsiQueryHandler.lookupTopic topicName tagSlotTypeDescription 0

That is, something in the JAMWiki code is stripping out the underscore from the topicName. I'm tempted to think that's not playing nice, but maybe there's a reason for that? I'm wondering if I went to a dash rather than an underscore...Just tried that with tagSlotTypeDescription-0 and it appears to have left the dash in! This suggests a theoretical bug fix: change to dash from underscore. Believe I'll go try that... -- Jack

A conundrum: the JCR appears to replace '-' with '_' for a variety of internal reasons, meaning, I'm stuck with underscores. JAMWiki's Utilities.decodeFromURL replaces '_' with ' ', meaning, JAMWiki will not accept underscores. Open question: what necessitates removal of underscores in Topic names? -- Jack

The underscore removal is a Mediawiki-compatibility thing since they automatically convert underscores to spaces. -- Ryan 18-Apr-2007 19:11 PDT

It is not clear to me what you are saying here. Consider this valid Wikipedia URL: http://en.wikipedia.org/wiki/Bohm_Dialogue -- Jack 19-Apr 14:07 PDT

The topic name for that article is "Bohm Dialogue" - there is no way in Mediawiki to create a topic named "Bohm_Dialogue". The URL http://en.wikipedia.org/wiki/Bohm%20Dialogue will also go to that same topic. -- Ryan 19-Apr-2007 18:22 PDT

I have rewritten my code to avoid dashes or underscores. The problem has left the building! Plus which, I have a much better understanding of this aspect of the bowels of JAMWiki. Thanks. -- Jack 19-Apr-2007 PDT