Tech:Parser isolation
Anonymous editing has been temporarily disabled. Please create an account in order to leave feedback or test wiki functionality.

Description

It would be an excellent idea to isolate the parser from the jamwiki core.

  • It would force a clean design.
  • It would allow others to use the parser as an independent tool, likely inviting additional development.
  • It would allow others to implement a different parser
    • Just a note, but a different parser can be implemented pretty easily using the current code and extending AbstractParser. JAMWiki ships with two parsers: the standard parser and the bliki parser. -- Ryan 26-Jul-2007 12:13 PDT

Original situation

This is the metrics view from Jamwiki

As we can see, there is a strong coupling (all nodes reachable from each other, called "tangle" from now on are in Red) between:

  • parser
  • model
  • org.jamwiki
  • utils
  • db
  • servlets
  • search

org.jamwiki.parser

Afferent Couplings Efferent Couplings Abstractness Instability Distance
7 8 33.0% 52.999996% 13.0%
Abstract Classes Concrete Classes Used by Packages Uses Packages
org.jamwiki.parser.AbstractParser
org.jamwiki.parser.ParserTag
 org.jamwiki.parser.ParserDocument
org.jamwiki.parser.ParserInput
org.jamwiki.parser.TableOfContents
org.jamwiki.parser.TableOfContents$TableOfContentsEntry
 org.jamwiki
org.jamwiki.db
org.jamwiki.parser.bliki
org.jamwiki.parser.jflex
org.jamwiki.search
org.jamwiki.servlets
org.jamwiki.utils
 java.io
java.lang
java.util
java.util.regex
org.jamwiki
org.jamwiki.model
org.jamwiki.utils
org.springframework.web.util

org.jamwiki.parser.bliki

Afferent Couplings Efferent Couplings Abstractness Instability Distance
0 8 0.0% 100.0% 0.0%
Abstract Classes Concrete Classes Used by Packages Uses Packages
None org.jamwiki.parser.bliki.BlikiParser
org.jamwiki.parser.bliki.JAMWikiModel
 None info.bliki.wiki.filter
java.lang
org.jamwiki
org.jamwiki.model
org.jamwiki.parser
org.jamwiki.parser.jflex
org.jamwiki.utils
org.springframework.util

org.jamwiki.parser.jflex

Afferent Couplings Efferent Couplings Abstractness Instability Distance
1 11 5.0% 92.0% 4.0%
Abstract Classes Concrete Classes Used by Packages Uses Packages
org.jamwiki.parser.jflex.AbstractLexer
 org.jamwiki.parser.jflex.CharacterTag
org.jamwiki.parser.jflex.HtmlCommentTag
org.jamwiki.parser.jflex.HtmlLinkTag
org.jamwiki.parser.jflex.HtmlPreTag
org.jamwiki.parser.jflex.HtmlTag
org.jamwiki.parser.jflex.IncludeOnlyTag
org.jamwiki.parser.jflex.JAMWikiPostProcessor
org.jamwiki.parser.jflex.JAMWikiPreProcessor
org.jamwiki.parser.jflex.JAMWikiProcessor
org.jamwiki.parser.jflex.JAMWikiSpliceProcessor
org.jamwiki.parser.jflex.JFlexParser
org.jamwiki.parser.jflex.NoIncludeTag
org.jamwiki.parser.jflex.ParserUtil
org.jamwiki.parser.jflex.TemplateTag
org.jamwiki.parser.jflex.WikiHeadingTag
org.jamwiki.parser.jflex.WikiLinkTag
org.jamwiki.parser.jflex.WikiListTag
org.jamwiki.parser.jflex.WikiNowikiTag
org.jamwiki.parser.jflex.WikiReferenceTag
org.jamwiki.parser.jflex.WikiReferencesTag
org.jamwiki.parser.jflex.WikiSignatureTag
 org.jamwiki.parser.bliki
 java.io
java.lang
java.text
java.util
java.util.regex
org.jamwiki
org.jamwiki.model
org.jamwiki.parser
org.jamwiki.utils
org.springframework.util
org.springframework.web.util

Proposal

At this point, there is

  • jamwiki-parent which is only a Maven project
  • jamwiki-core which is takes most code from the oldt jamwiki
  • jamwiki-parser-api which is empty at the moment but should contain the API of the parser TODO isolate the API from jamwiki-core
  • jamwiki-parser-jflex which contains the JFlex implementation of the parser
  • jamwiki-parser-bliki is another parser. At the moment it does not work, because its implementations extends JFlexParser and I want them to be independant. TODO make it work
  • jamwiki which is the final webapp. It has dependencies on core and parser, and adds servlets and configuration (such as web.xml).

TODO update diagramm

Current situation

Author(s)

Status

As of revision 2027 it is now possible to use the JAMWiki parser in a standalone mode. There are further changes that will occur to cleanup the interfaces, but currently the jamwiki-core Maven project will generate a JAR file containing everything required to parse Mediawiki syntax into HTML. See the org.jamwiki.parser.ParserTest.java file within the jamwiki-core project's unit test directory for an example of how to use the parser in a standalone setting, noting that several resource files and a DataHandler instance will still be needed to successfully run the parser.

Update

The original work on this task was abandoned due to the inter-dependencies between all of the JAMWiki packages. Over the past few releases I've been making an effort to reduce these dependencies up as much as possible, with an eventual goal of modularizing the JAMWiki project to look something like the following (NOT final - will definitely change):

jamwiki-core
org.jamwiki.*
org.jamwiki.utils.*
org.jamwiki.model.*
jamwiki-db
org.jamwiki.db.*
jamwiki-parser
org.jamwiki.parser.*
jamwiki-web
org.jamwiki.authentication.*
org.jamwiki.servlets.*
org.jamwiki.taglib.*
jamwiki-???
org.jamwiki.ldap.* (this package may go away with the next Acegi release)
org.jamwiki.mail.* (currently unused)
org.jamwiki.search.*

This re-organization is a work-in-progress and not a tremendously high priority, but I'm posting here for those who watch the Subversion commits and wonder what the goal of the occasional cleanup re-organization commit might be. -- Ryan 19-Jan-2008 17:40 PST

The re-organization work has been continuing lately, and I've been forced to learn a lot more about Maven. As a result, the Maven project structure is currently:
  • jamwiki-core
  • jamwiki-war
  • javadiff
Additionally, there is a Super POM that ties these sub-projects together. mvn site is currently a bit broken, but a working jamwiki WAR file can be generated. The next step is to split out a jamwiki-parser project, and to potentially split out additionally modules if it makes sense to do so. -- Ryan 27-Jan-2008 22:35 PST
There will probably be further cleanups, but at the moment the jamwiki-core project is sufficient to run the parser in a standalone mode and I am using it for unit testing. Contributions and cleanups are welcome, and in particular if someone is looking for a project it would be great to see some documentation on how to use the parser code from a standalone app. If no one else gets to it first I'll eventually try to put such a wiki topic together. -- Ryan 17-Feb-2008 22:15 PST
Is it possible to maintain my bliki parser for jamwiki in this JUnit standalone version. I think I don't have the time to always test it with a locally installed full jamwiki version? If yes, what steps are necessary? Axel Kramer 26-Feb-2008 10:54 PST
I'll take a look tonight - it should be a one-line change to run the JFlex unit tests against the Bliki parser (basically, just changing the "parser" environment variable from JFlex to Bliki). The only problem would be differences in parser output due to whitespace, but I'm hoping to try and make the JFlex output exactly match Mediawiki over the next few release cycles so with luck both parsers will eventually produce exactly the same output. Also, should you ever want to move more of the Bliki parser code into the JAMWiki project repository I'll then make sure any updates to JAMWiki are compatible with your implementation. -- Ryan 26-Feb-2008 12:01 PST