Developer's Guide

From DOC

Overview

This article is intended for those making a change to Tolven that goes beyond configuration. Development may involve web pages, resource bundles, Java code, rules, etc. If you have already configured Tolven, the concept of plugins should not be new to you. Now it is necessary to understand them in a bit more depth.

Almost all of Tolven is made up of plugins: Only a small bootstrap configuration controls which plugins are loaded and executed. If you are building a new Tolven application, the product of the development process will be a plugin.

What is in a plugin

Your project will most likely consist of a single (additional) plugin added to the list of plugins already configured. In essence, a plugin contains all of the little files needed to add some capability to the Tolven environment - all in one place. For example, your plugin might add a wizard page, a drilldown page, perhaps a new validator, maybe some new rules, additions to one or more locale resource bundles. Even a new Java API and implementation.

In most cases, your plugin will make additions to the underlying configuration provided. Here is how this plugin feature works: You will describe an "extension" to an existing plugin. For example, the tolvenweb plugin has a login page within it that was designed by Tolven. Your plugin manifest can specify that a login page that you define should override the login page provided by Tolven. Likewise, you may add additional pages to the existing tolvenweb application. Technically, these substitutions are made prior to deployment of the "ear" file to the application server which means that if you were to look at the running application, it would not contain any page that was overridden by you.

Plugin deployment can seem complex at first. Ant scripts hide most of this complexity, but you still need to understand the basics:

All plugins are zip files and they must contain at least one file called the manifest, which is always named tolven-plugin.xml. This tells TPF how to process your plugin. The manifest will refer to other contents of your plugin typically by directory or file names that are relative to the root of the zip file.

Plugin names

The name of the plugin zip file is significant. <plugin-id>-<version>.zip. The name of the zip file must match the id in the manifest. Likewise, the version scheme is significant. TPF must be able to compare version numbers to determine which is newer. Therefore, the version number must only contain periods (dots) and decimal digits. Version 1.10.1 is greater than, more recent than, version 1.2.0.

Plugin repositories

Plugins reside in plugin repositories. While any number of repositories can be defined, there are three types that you should know about:

A library respository generally contains plugins that may be used at a particular site. The primary Tolven on-line library is http://tolven.org/download which contains plugins published by Tolven. Depending on the configuration specified in your tolven-config/plugins.xml file, which is used by tpf -getPlugins, plugins from this library will be added to the tolven-config/repositoryRuntime repository. You might also think of this as a catalog of plugins.

The runtime repository, located at tolven-config/repositoryRuntime is the collection of plugins that will be used to assemble and configure your system. In other words, all of the deployment-type commands (configPhase1, configPhase2, etc) and any other explicit plugin command such as tpf -plugin plugin name operates against this repository. So, in this context, runtime means TPF runtime, not necessarily the application server.

The local plugin repository starts empty in most installations but will be the one that you will use often as a plugin developer: tolven-config/repositoryLocal. This repository can be located anywhere, but we will leave it in the tolven-config directory for this article. The repositoryLocal is scanned at the same time as the main library repository. In other words, it extends the main library with additional plugins. However, repositoryLocal is not an override mechanism. If exactly the same version of the same plugin is found in both repositories, TPF will throw an exception. Therefore, it is important that you increase the version number of a plugin you are updating so that it takes precedence over older plugins with the same id. repositoryLocal is intended to be temporary: A plugin remains in repositoryLocal while it is being tested and is then moved to a library/catalog during the publish process.

Each plugin repository has the same structure: A single file at the top-level named plugin.xml describes the contents of a nested folder named plugins. The plugins folder contains each of the zipped plugins. The top-level plugins.xml file, referred to in this context as repository metadata, must match the contents of the plugins directory. A checksum is computed to detect changes in the plugins directory. this file also contains plugin dependencies.

Two lifecycles to consider

The lifecycle of a plugin project is different from the lifecycle of a plugin zip, the final packaged version of a plugin. The following are some guidelines you should consider:

While a new plugin is initially being developed, the plugin is unpublished. That is, the plugin is not available for download outside of your development environment. It's existence is limited to tolven-config/repositoryLocal, which is sufficient for testing.

At the same time, the plugin project may be committed to CVS/SVN etc. for safe keeping. This generally does not include the plugin zip and does not signify that the official plugin has changed yet, only that someone is working on a new version of the plugin.

Plugin structure (Sections)

A plugin will often contain source, web page, and configuration elements that comprise a functional area. Take allergies for example: client code to load allergies, web pages to display and enter allergies, rules that apply to allergies, etc. So, to make Allergies pluggable, we need to put all (or at least most) of those components related to allergies into a single plugin. Tolven calls its version of this plugin org.tolven.deploy.allergies and maintains it in a project named pluginAllergies.

To keep all of the possible parts of a complete plugin organized, and to facilitate refactoring, the plugin manager organizes plugins into several sections. All sections are optional. And you may add sections. In general, a section contains the components that will be directed to a certain system component as described in the table below. Each section in the plugin can contain classes and configuration files appropriate to that section. The project contains a number of section folders. There are two differences between a section folder in your plugin project and the section folder in the plugin:

The project holds class files in the temporary build folder whereas the plugin zip has the classes stored in the section.
Sources in the plugin project are stored in the section while in the plugin zip, sources are stored in an embedded devLib.jar.

The following table shows what the section folders of a complete plugin project might look like.

Section	Components	Description
manifest	tolven-plugin.xml, tolven-plugin-fragment.xml, license file	Defines the plugin as a whole. This directory is copied to the top-level in the finished plugin zip file.
app	applications, trim, etc	Application configuration files. These include application metatdata (xxx.application.xml), core trims, etc.
tpf	source, voc, etc	Components that apply to and run at configuration time. In other words, these classes may be loaded and used when the tpf command is run.
ejb	source, orm	Components added to the tolvenEJB environment that share an entityManager with Tolven. At assembly time, the class files associated with this section will be loaded into appropriate places in the tolvenEJB.jar file, which will then be added to the tolven.ear. This code does not run in the tpf itself.
web	source, web/ajax, web/five, web/drilldown, web/wizard, etc.	Components added to the tolvenWEB war file which is included in the tolven.ear file. At assembly time, the class files associated with this section will be loaded into appropriate places in the tolvenWEB.war file, which will then be added to the tolven.ear. This code does not run in the tpf itself.

Reminder: You don't have to use this structure. It is just a guideline supported by the build scripts in the plugin manager plugin project.

The section structure partitions the build process rather thoroughly - source code in one section cannot refer to classes built in another section of the plugin. In some cases, you many need one section to refer to the compiled classes in another section. A common example would be the web section depending on the ejb section. The ejb section needs to be built first and then the web section must include those ejb classes in its classpath. To do this, include the lib param as shown:

   <antcall target="buildSection">
      <param name="section" value="web"/>
      <param name="lib" value="${buildFolder}/ejb/classes"/>
   </antcall>

The contents of the plugin manifest, manifest/tolven-plugin.xml, must correspond to your plugin structure. In fact, the template includes all of these sections, but most are commented.

Setup

This section covers one-time setup needed for a plugin development environment.

Prerequisites

Prior to tolven-RC1, the developer install was completely different from the "binary install". Now, the installation procedure is the same in either case. Therefore, if you have not done so already, you should install and configure Tolven.

Creating a Tolven plugin requires nothing more than a text editor, possibly a Java compiler, and a zip program. Nevertheless, this guide will describe the process using some Ant targets that make the job easier. Further, it will describe these tasks assuming that you are using an Integrated Development Environment (IDE) such as NetBeans or Eclipse though these Ant scripts can be used from the command line just as well.

Install the Developer Plugins

If not done already, you will need to modify the root plugins file tolven-config/plugins.xml file to contain two plugins (your configuration may already contain one or both of the following, so please check).

For installation kit versions 0.0.3 and greater:

<plugin id="org.tolven.developmentmgr">
    <root />
</plugin>

For installation kit versions prior to 0.0.3:

<plugin id="org.tolven.deploy.source">
    <root />
</plugin>
<plugin id="org.tolven.config.deploy.source">
    <root />
    <property name="deploy.source.active" value="true" />
</plugin>
<plugin id="org.tolven.source.ant.plugin">
    <root />
</plugin>

If any of these plugins are already specified in your plugins.xml file, do not add them again but rather, edit them to look like the above. In particular, the deploy.source.active property must be set to true in order to activate that plugin.

Then at the command line in the <install-dir>/bin directory, make sure these plugins have been downloaded by executing:

tpf -getPlugins

In general, when you modify tolven-config/plugins.xml, you will almost always need to run tpf -getPlugins. Running this command when not needed is harmless and fast. However, if new versions of plugins are available, they will be downloaded.

Once this command-line process is done the first time, the Ant scripts, described below, will call getPlugins automatically.

Installing devLib

Running the org.tolven.deploy.source plugin causes the creation of a directory containing one jar for each plugin in your repository runtime. Each jar file contains the source code and class files for that plugin. The jars are normally downloaded to tolven-config/devLib.

After verifying that tolven-config/plugins.xml has the correct configuration, as described above, use the following command to generate tolven-config/devLib:

For installation kit versions 0.0.3 and greater:

tpf -plugin org.tolven.developmentmgr -devLib

For installation kit versions prior to 0.0.3:

tpf -plugin org.tolven.deploy.source

The Ant scripts in this plugin project will use the jars in this directory for all compilation and Tolven-related execution.

If you are using an Eclipse or other IDE, you can also create a "User library" containing the full contents of this directory. This will prepare your environment for development against Tolven APIs and for debugging the running Tolven application server. Do this in Eclipse:

Window > Preferences > Java > Build Path > User Libraries

Add a new User Library called DevLib (or something you will remember) and add all of the jars in the tolven-config/devLib directory to that User Library.

Referencing devLib from Eclipse projects

Your plugin project in Eclipse should reference the devLib user Library to resolve references to Tolven and third-party libraries. Do this by opening the project properties wizard > Java Build path > Libraries > Add Library > User Library > and select the devLib library.

You must also be clear about the order in which jars are searched during debugging sessions. While still in the Build Path dialog, go to the Order and Export tab and make sure that devLib is after your source directory. This means the source code in your project folder will be used in preference to a potentially older version of the same code in devLib (which you are in the process of updating).

Plugin Manager

The org.tolven.source.ant.plugin plugin contains a project that contains Ant build files that will be useful to you while developing, testing, and deploying a plugin. Find the org.tolven.source.ant.plugin zip file in your tolven-config/repositoryRuntime/plugins directory and unzip it to a new directory named plugin in a location you will use for development.

If using an IDE, you should create a new project named plugin (the name plugin is required). Otherwise, just create a directory named plugin and then upzip the org.tolven.source.ant.plugin plugin to that project directory. The plugin manager project only contains build files and some templates so it does not need to be a Java project.

Build.properties

You may need to customize certain file locations used by the build scripts. Do this by creating a file called build.properties in the plugin project. If not overridden in this file, the properties specified in buildCommon.xml will be used. For example, if your tolven-config is located in, say, c:\test\tolven-config rather than the default c:\tolven-config as specified in buildCommon.xml, add an override to build.properties.

tolvenConfig.location=c:/test/tolven-config

Notice the forward slashes, as required by Ant.

Use this same style of override if you are installing on a Linux file system. For example:

tolvenConfig.location=/usr/local/tolven-config

This style of property override is used so that you can uptake new versions of Plugin Manager files without destroying your local settings.

Setting up Eclipse Ant View

You should include the managePlugins.xml build script file in your Ant view.

See: Example in Eclipse

Your development directory structure should now look something like this:

  tolvenDev (your workspace)
    plugin (the plugin manager project)
    tolven... (For previous developer installations, these Tolven projects should now be removed)

Viewing your Runtime Repository

Plugin Manager contains an optional task called repositoryRuntimeList which creates an html page containing a sorted, formatted list of the selected plugins in your repositoryRuntime. The processor requires a non-default XSLT processor (the default Trax processor does not support functions). Install, say, Saxon to some location and add an entry in your build.properties file that looks something like this:

xslt.processor.classpath=c:/saxon/saxon9he.jar

Again, note the use of forward slashes.

Updating devLib

To keep the sources (and classes) in DevLib up to date, an ant task called updateDevLib is provided. This task will first make sure that the plugins in your repositoryRuntime are up to date.

You can run updateDevLib periodically as your configuration changes, including when a new version of an existing plugin is downloaded.

For Eclipse users: Eclipse does not dynamically update the user library as the contents of DevLib changes so you may need to recreate the Tolven devLib User Library in Eclipse when a jar is added or removed. Go to Window > Preferences > Java > Build Path > User Libraries and delete devLib, then add it again as described above. In any case, after updating devLib, you will need to do a Project > Clean all for the changes to be seen by Eclipse.

Creating a plugin

The activities in this section are repeated for each new plugin you create. In most cases, you should be able to put all of your work into a single plugin.

As you prepare to create a new plugin project, there are a few things to consider about plugin project names and plugin Ids:

Project names will typically be private to your organization. Unlike the plugin id, there is no need to include your domain name in the project name. For example, a project named myPlugin might have a pluginId of com.myorg.myplugin.
At Tolven, all plugin project names are prefixed with plugin such as pluginCCR and pluginRxNorm. The only restriction is that you not name your project plain plugin. You might use a project name such as FancyPlugin - to signify it's purpose or value.
Popular source code management systems require a simple project name. For example, periods are not allowed in CVS project names. If that is not a restriction for you, then you could name your project the same as the plugin ID.
A project could host several plugins, though this article will not discuss how to do that: So, one project equals one plugin.
A published plugin represents a point-in-time whereas a project is timeless. Therefore, one project is likely to yield several versions of a plugin over time. This may be a source of confusion: In general, CVS/SVN should carry the source for a plugin (the contents of your plugin project) and the resulting plugin zip file is a target for publication but is usually not checked into CVS/SVN.

Separate from the project name is the plugin id. There are a few things to consider:

Unless you are doing work for Tolven, your plugin will have a prefix different from org.tolven.
Your plugin id will ultimately be combined with plugins from other sources. Therefore, it is important to include a sufficiently unique domain name prefix. For example, com.bigorg.division....
The name itself should not include a plugin version number. A version number will be appended automatically, when needed. TPF will ensure that only one version of a given plugin will be chosen. Internally, TPF keeps track of each plugin and then further keeps track of each version of each plugin. Therefore, when it looks for a plugin, it first looks for the pluginId, and then determines the latest version available for that plugin.

The "official" location for the name of a plugin is in the root element of the tolven-plugin.xml manifest. The procedure below will set the plugin id in that file for you. If you later decide to change the id, change it in that file. But be sure to "clean" your build directory at the same time.

Creating your plugin project

If not done already, you should create a new project to hold your new plugin. Some considerations:

If using Eclispe, you should create it as a Java project.
Because your project will contain Ant build scripts, it is important to keep separate folders for class files compiled by the IDE vs classes compiled by the build scripts. For example, Eclipse will target classes in the bin folder by default (which is fine). The plugin scripts, by default, will put compiled classes in the <section>/classes folders.
The step below will add files to the project. Be sure to refresh if needed after adding these files.

You can share your project with a code management system. Tolven's build targets will put all transient files in a folder named build so you can add this folder to the list of items not committed to the code management system (eg .cvsignore).

The resulting plugin for this project, a zip file, is not intended to be committed to code management and therefore is stored in the build folder. The build folder is deleted when the Ant clean target is activated.

Adding key files to your plugin project

The Ant target addPluginStructure in the plugins project will add a number of files to your new or exiting plugin project (it will not overwrite these files if they already exist):

The plugin manifest, manifest/tolven-plugin.xml, which completely defines how the plugin affects the Tolven environment.
An Ant script, build.xml, that will control the construction of your plugin. At it's core, build.xml will constructs the <plugin-id>-<version>.zip file. build.xml will depend on other build scripts in the plugin project to carry out many routine tasks.

Activate the addPluginStructure Ant target and answer the questions about the name of your project and the plugin id being created.

Then, if using Eclipse:

Refresh your Package Explorer view so that you can see the new files
Drag the generated build.xml file to your Ant view window
Make the source folder in each section you are going to use a Java build path source folder

This build file imports another build file from the plugin manager project which does a lot of the detailed work in building a plugin zip. However, you can also create the zip yourself. There is very little required structure. However, in this article, we will provide a suggested structure and support it with Ant scripts from the plugin project.

The Hello World application in your new plugin

The project files added above contains a small Java file which generates a Hello World message. In reality, Tolven would have a number of Hello World programs, corresponding to various sections (one for ejb, one for the web application, etc, etc). However, the only one supplied by default is targeted to run directly in TPF. In other words, the hello world code executes with the TPF command, not within the application server. To see the Hello World code run, you'll have to activate the plugin using the tpf command. For example, if you named your plugin id is com.myorg.myplugin:

  tpf -plugin com.myorg.myplugin

If TPF is not recognizing your plugin, it could be for one of the following reasons:

You must declare your plugin as <root/> in <config-dir>/plugins.xml. Without this, repositoryInit (getPlugins) would not know to include your plugin.
Have you run getPlugins? If not, then either the previous version or no version of your plugin will be available.
Are you sure that the plugin id in the tolven-plugin.xml exactly matches the pluginIds used in <config-dir>/plugins.xml.

Once you've seen this code work, you will most likely want to remove it. If you are not intending to add any code that runs in tpf, that is, in the configuration environment, then it can be removed completely:

Edit the manifest/tolven.plugin.xml file:
Remove class="HelloWorld"
Comment or remove the whole <runtime> section
In the tpf section, delete the HelloWorld.java file.

If you do need code that runs in the tpf configuration client, such as a loader, then you can just refactor HelloWorld to another name, adding a package name prefix as well. Also, be sure to change the full class name in the manifest/tolven.plugin.xml file.

Acquiring plugin contents from a prototype

The Ant target addPrototypeToProject can be used to easily grab the contents of a plugin already in a repository library. The prototype does not need to be and usually is not in your tolven-config/plugins.xml file.

This target will prompt you for the plugin id of the prototype plugin, the plugin id of the target plguin, and the name of the plugin project you want to add this prototype to. For example,

Prototype plugin id: org.tolven.prototype.simple
plugin id: com.myorg.myplugin
Plugin project name: myplugin

The plugin project should already exist. This script will simply expand the plugin to the prototype folder in your project. It does not attempt to place the contents of the prototype into the various folders in your project.

If using Eclipse, refresh the project in order to see the added contents.

At this point, you may need to clean up some of the added files depending on your circumstances.

The library from which the plugin is selected is assumed to be http://tolven.org/download but you can change the default by setting an override in build.properties, a file that Tolven will not override.

srcRepos=http://tolven.org/download # Change this location if needed

When you are done harvesting contents from the prototype, you can delete the prototype directory.

Adding Extensions to your plugin manifest

Most plugins are built up using extensions and/or sometimes extension-points. Each extension contributes some bit of information to the final application. For example, one extension might offer a list of web pages while another extension might contribute an EJB to the application. A few common sets of extensions:

Customize tolvenWEB Plugin - Adds pages to appropriate sections of the Tolven web application
Customize tolvenEJB Plugin - Adds EJBs to the Tolven EJB layer.
Customize tolvenEAR Plugin - Adds jars to the tolven EAR file.
Locale Resource Bundles - Add localized strings.

Packaging your plugin

In your project, the deploy-to-local-repository Ant target (imported from importBuild.xml in the plugin project) will perform several steps:

Compile source code if present in your build.xml script.
zip up the contents of your plugin according to the build-plugin target.
Copy the plugin zip file to your repositoryLocal directory.
Update the metadata for repositoryLocal.

This essentially puts your plugin on a peer with plugins at the central plugin site(s) except that your updated plugin is likely to have a higher version number than the one on the central site. (Remember that keeping the same version number will not work: Your plugin must have a higher version number than one that it is updating.)

Packaging and deploying your plugin to the local repository does not update your repositoryRuntime. That will occur as part of the next step. Of course the getPlugins target can be run to perform this operation explicitly.

Update Root Plugins

If your new plugin is not requested to be used, then it won't actually be deployed or otherwise used. If you have not done so already, you will almost always have to add a new root entry in tolven-config/plugin.xml to ensure that your plugin is included in the resulting application. You will add something like this:

<plugin id="com.myorg.myplugin">
    <root/>
</plugin>

ConfigPhaseX

You will have to determine which of the final configurations steps are needed to complete the cycle. configPhase1 is typically needed when server-side java, resource bundles, and/or web pages are changed. ConfigPhase2 is needed when database, indexes or LDAP schemas are changed (very rare) and configPhase3 is needed when accountType metadata, rules, and reports are deployed to the server. You may even need to use more than one of these phases depending on the scope of your plugin.

The build file in the project named plugin (not your plugin project) contains targets that will launch these functions. They perform exactly the same task as the command-line functions that you've used during the configuration process. However, note that these Ant targets automatically call the getPlugins function whereas at the command line, you must do this explicitly.

Debugging the running application server

If you have added server code (ejb/source or web/source) that you want to debug or you simply want to set some breakpoints in Tolven:

Edit tolven-jboss-4.2.2.GA/bin/run.bat. Toward the bottom of the file is a line that sets up for remote debugging. Uncomment the line and change suspend=y to suspend=n so that the line looks like this:

set JAVA_OPTS=-Xdebug -Xrunjdwp:transport=dt_socket,address=8787,server=y,suspend=n %JAVA_OPTS%

Restart the application server.
In Eclipse, run > Open Debug Dialog > create a new Remote Java application.
Set the connection properties to be locallhost and port 8787 (same as the line above).
In the source tab, select Add > Java Library
Select User Library
Select DevLib (the library you created above)
Save that.

Now, to debug the running application server, simply Debug the remote application you setup.

Publishing your plugin(s)

When your plugin is ready to leave the nest, there are several steps you should consider, some of which are automated. In general, this process can mean different things to different organizations but in any case, the mechanics are the same. For Tolven, publication means making the plugin available on the download site. For a developer creating a plugin for use in a single operation, publication may mean making the plugin available to operations for use in testing or production. In any case, the plugin is ready to leave the development environment and thus the <cofig-dir>/repositoryLocal folder.

While developing a plugin is a long-running process, publication should be considered a point-in-time operation, at least as close as possible to atomic. Publication consists of the following steps:

Verify that the version number is correct relative to any previous version in the target library. For example, if the current version is 0.0.1, then the version about to be published, should be 0.0.2 or greater. The version number is in the tolven-plugin.xml manifest.
Commit the plugin project to CVS/SVN.
Upload the new/updated plugin(s) to a public repository (part of the publish Ant target)
GenerateMetadata (merge) new/updated plugin(s) to the plugins metadata file at that public repository (part of the publish Ant target)
Upload the new metadata plugin.xml to the public repository (part of the publish Ant target)
Delete the plugin(s) from repositoryLocal (part of the publish Ant target)
GenMetadata on repositoryLocal to reflect the removed plugins in the metadata plugin (part of the publish Ant target)
Tag your project in CVS/SVN so that it corresponds to the just-published plugin. The tag could contain the plugin id and version like this com_myorg_myplugin_0_0_1.

You may then make additional changes to the source code in advance of the next version of the plugin to be published.

See Plugin Publication for more details.

Each plugin published by Tolven will also have an entry in this wiki in the form Plugin:<pluginId>. Each version of a plugin will have a Version Notes section describing that version of the plugin. The version notes will be in a section in the form Version <version>.

Deleting plugins from RepositoryLocal

As described earlier, RepositoryLocal is for temporary storage of plugins you are working on. Further, the publish process only works with the contents or repositoryLocal - it knows nothing of the individual plugin projects.

Say you are working on two plugins at once and you are about to publish plugin1. Plugin2 is still a work in process. However, both plugins are currently in repositoryLocal. If you leave both plugins in repositoryLocal then they will both be published. You can select the deleteAllFromLocal ant target in the plugin manager to completely clear out repositoryLocal. Then, select deploy-to-local-repository on the plugin1 project. Now, only one plugin is in repositoryLocal and the publish process will only "see" plugin1.

deleteAllFromLocal is executed automatically by the publish target. In other words, at the conclusion of a publish, repositoryLocal will always be empty.

Where to go from here

Creating an effective plugin depends in part on knowledge of other plugins, especially those specifically designed to allow you to customize the finished application server. Each plugin provided by Tolven is, or should be, described in this wiki. You can also view all versions of all published plugins at http://tolven.org/download. This site has links back to this wiki for each plugin and to version notes for the plugin.

Facts about Developer's GuideRDF feed