From Emergent

TODO: Update this doc for the new source layout, and pdp->emergent

Contents 1 TACSS Plugin system 1.1 Potential applications 1.2 Plugin Basics 1.3 Application view of Plugins 1.3.1 Plugin properties 1.3.1.1 unique_id 1.3.1.2 version 1.3.1.2.1 Upgrades 1.3.1.3 url 1.3.2 Viewing Plugin Status 1.3.3 Enabling and Disabling Plugins 1.3.4 Plugin Load Log 1.4 Plugin Issues 1.5 Folder Layout 1.5.1 System Folder Layout 1.5.2 Windows 1.5.2.1 Unix and Mac OS 1.5.3 User Folder Layout 1.6 tacss/pdp plugin startup sequence 1.7 Using plugins in projects 1.7.1 Determining Plugin Dependency of Classes 1.8 Creating a Plugin 1.9 Building Plugins 1.9.1 System Plugins 1.9.2 User Plugins 1.9.3 Misc 1.10 Qt resources

TACSS Plugin system

Emergent/TEMT can be extended via user-written libraries called plugins. Here are the main points about plugins:

• a plugin can be shared by all users ("system plugin") or be particular to one user ("user plugin")
• plugins can be used by any program that uses the tacss library, including generic css as well as emergent
• plugins must be written in C++
• plugin technology is based on Qt (v4.2) and so the development version of Qt 4.2 must be on the system
• Linux and Mac users can use the open-source (free) version of Qt
• Windows users must have the commercial version of Qt, since that is the only version support by tacss

Potential applications

• New network algos, e.g., cs, so, etc.
• Wizards - help w/ build/optimize proj
• Data generation
• Input patterns
• Analyzing output
• Novel in/out
• new network objects (ex. specialized Layer or Unit types)

Plugin Basics

A plugin is basically a set of one or more C++ classes that derive from one of the major root classes in tacss. Although you can have other non-taBase or flat functions in your plugin, those classes or functions will only be able to be used in code internal to your program, not by the emergent/temt system. Here are the main root classes:

• taBase -- root of all classes that can have editors, change members in code, call methods in programs, etc.
• taiType -- provides a new description for a new gui editing widget for types
• taiData -- object that mediates between tacss and a widget or editor

Most plugins will derive objects from taBase classes, for functional purposes.

The classes you create in your plugin can be instantiated in a tacss project in any context that enables user objects to be created. These typically pop up a dialog that lets you choose the type of object to create (ex., a network Layer type, etc.) -- once a plugin is enabled and loaded, its classes will automatically get added to these lists.

A project that uses plugin objects will require those respective plugins to be available whenever it is loaded. For this reason, projects contain a plugin dependency list that is examined as the project is being loaded -- if all the plugins required are not available, the user will be prompted to cancel the load.

Two important terms are used with plugins: "probed" and "loaded".

• probed -- the plugin library is available, has been compiled with a correct version of Qt, and can be tentatively loaded by Qt
• loaded -- the plugin has been probed, and has been fully loaded by the tacss system, and its type information has been integrated

A plugin that cannot be probed cannot be loaded. But a plugin that has been probed may not be loaded if it is not functioning, the wrong application version, or has not been enabled. These topics are covered further below.

Application view of Plugins

Plugins are managed via two collections in the root object:

• .plugins -- list of currently available and previously available pluginsBR
• .plugin_deps -- list of plugin dependencies for last project saved or loaded

Plugin properties

The .plugins collection lists all the available or previously available plugins. Each entry provides the following information about each plugin:

• name -- the plugin's "friendly" name, ex "HelloPlugin" or "so"
• description -- a description of the plugin
• unique_id -- this is an url-like string that uniquely identifies the plugin (see below)
• version -- this string is in the form: major.minor.revision.build (see below)
• url -- a url (web site) that can be visited to get documenation, the code, new versions, etc.

unique_id

This id is used to match plugin requirements in projects with available and loaded plugins at runtime. Each distinct plugin should be given its own id. Although any unique string can be used, it is suggested to use an url-like format, to help differentiate plugins, based on the user's company or educational affiliation. It is also acceptable to use the string form of GUIDs (globally unique id's), a predominantly Microsoft-centric construct.

version

The version is used to distinguish compatible vs. incompatible versions of a plugin. The convention is as follows:

• build -- complete compatability, just bug fix or such
• revision -- binary compatability, but might have minor behavioral change; forward and backwards compatible
• minor -- backwards compatible, but not forward compatible
• major -- probably not backwards or forwards compatible

TBA -- examples

For a major change, the unique_id should also be changed, since the plugin is no longer compatible.

Upgrades

TBD -- need some kind of mechanism to enable older projects using earlier plugin versions to be upgraded to newer versions of that plugin.

url

If a project is opened on a system without a required plugin, the plugin information is available in the .plugin_deps folder (until saving or loading another project.) The url in this information enables the user to visit the website and obtain the plugin. It also enables users to check for newer versions of plugins.

(At some point, an automated system may be added to help fetch the plugin automatically on user request.)

Viewing Plugin Status

There are three key parameters associated with a plugin:

1. . whether it is available
2. . whether it is enabled
3. . whether it is loaded

• An "available" plugin is one that exists and is functioning, ie., it has been probed.
• An "enabled" plugin has been set by the user to be enabled -- note though that it can only be actually loaded if it available.
• A "loaded" plugin is available, enabled, and has been successfully integrated in the tacss type system and application.

To view a plugin's status, do the following:

• navigate to the plugin entry, to make its properties available in the editor
• view the properties in the editor

Enabling and Disabling Plugins

When a plugin is first encountered, it is listed in the .plugins collection, but is not enabled by default. This is to provide user control, security, and to help avoid adding unexpected plugin dependencies to projects. In addition, some plugins could have additional system dependencies not established until they are loaded, which could cause problems if the dependencies are not available.

To enable a plugin, do the following:

• navigate to the plugin entry, to make its properties available in the editor
• check the Enabled checkbox
• click Apply
• (repeat for as many plugins as desired)
• close and reopen the application

To disable a currently enabled plugin:

• navigate to the plugin entry, to make its properties available in the editor
• uncheck the Enabled checkbox
• click Apply
• (repeat for as many plugins as desired)
• close and reopen the application

Plugin Load Log

The plugin loading process creates a log each time the application starts. It saves the content of this file after making each entry, so if the application is crashing, it is likely caused by the most recent action. The log contains lines of the forms "about to..." and "status of..." for each action, so if there is no status message after an "about to" message, it is likely that plugin causing the crash.

The plugin log can be found, depending on the platform:

Linux and Mac:

~/.<appname>/plugins.log

Windows: C:\Applications and Settings\<username>\Application Data\<appname>\plugins.log

Plugin Issues

Plugins are a blessing and a curse. They provide an elegant and relatively easy to use mechanism for extending applications. But because they are compiled in C++ and dynamically loaded, many issues can arise, particularly in development environments that are also building new or custom versions of the underlying application.

The first thing to do if a plugin issue arises, is to insure that the plugins have been compiled with the latest version of the program libraries. Go the system and (if applicable) user plugin folders, and issue:

> cd [system plugin folder]
> make clean
> make
# following if applicable:
> cd [user plugin folder]
> make clean
> make

There should be no compile or link errors, otherwise there is a problem.

If this does not solve the problem, it may be necessary to eliminate an offending plugin. Because plugins must be probed before being loaded there is a chance this probing process could cause crashes or other issues.

The easiest way to eliminate a plugin is to delete its file in the lib/ plugin subfolder. Note that this file can always be recreated by rebuilding the plugin. The most obvious plugin to delete is the one which has a "about to" but not a "status" line in the plugin log.

See the [Building Plugins] section below for instructions on preventing a plugin from being built.

Folder Layout

This is the folder layout from the point of view of plugins. We show the layout for the debug and non-debug versions of: css, emergent, and another (fictional) tacss-based program called sim. A number following an item indicates it is merely one of an unlimited number.

The plugin folder layout is designed to easily enable user plugins to be turned into system plugins without need to edit the plugin files.

System Folder Layout

(Note: on Windows, plugins are placed in the /bin folder, not the /lib folder)

Windows

<Program Files>\emergent
  \bin
    css
    css_debug
    emergent
    pdp++_debug
    tacss, pdp dlls (Windows only)
    tacss libs
    pdp libs
    \plugins_tacss
      tacssplugin1 -- release gui nompi binary of tacssplugin1
      \debug
        tacssplugin1_debug -- debug version of tacssplugin1
      \debug_mpi
        tacssplugin1_debug_mpi -- debug mpi version of tacssplugin1
      {etc.}
    \plugins_pdp
      {similar substructure as tacss}
  \plugins
    \tacss -- subfolder for tacss plugins
      /tacssplugin1 -- folder for src of tacsssplugin1
    \pdp -- subfolder for pdp plugins
      \pdpplugin1 -- folder for src of system plugin pdpplugin1

Unix and Mac OS

The Unix install uses the standard distributed install paradigm:

(pdp4.0 bin install folder, def: /usr/local/bin)
    css
    css_debug
    pdp++
    pdp++_debug

(pdp4.0 lib install folder, def: /usr/local/lib)
 
  /lib
    tacss libs
    pdp libs

(pdp4.0 system folder, def: /usr/local/share/pdp++-[version])
  /lib
    /plugins_tacss
      tacssplugin1 -- release gui nompi binary of tacssplugin1
      /debug
        tacssplugin1_debug -- debug version of tacssplugin1
      /debug_mpi
        tacssplugin1_debug_mpi -- debug mpi version of tacssplugin1
      {etc.}
    /plugins_pdp
      {similar substructure as tacss}

(pdp4.0 system folder, def: /usr/share/pdp4.0)
  /lib
    /plugins_tacss
      tacssplugin1 -- release gui nompi binary of tacssplugin1
      /debug
        tacssplugin1_debug -- debug version of tacssplugin1
      /debug_mpi
        tacssplugin1_debug_mpi -- debug mpi version of tacssplugin1
      {etc.}
    /plugins_pdp
      {similar substructure as tacss}
  /plugins
    /tacss -- subfolder for tacss plugins
      /tacssplugin1 -- folder for src of tacsssplugin1
    /pdp -- subfolder for pdp plugins
      /pdpplugin1 -- folder for src of system plugin pdpplugin1

User Folder Layout

The following gives an example layout for the pdp++ application. A similar layout would exist for css or sim.

/home/[username]/
pdp++_user {or whatever is the user's pdp4 user folder}
  /lib
    /plugins_pdp
      mypdpplugin1 -- release binary of mypdpplugin1
      /debug
        mycssplugin1_debug -- debug version of mycssplugin1
      {etc. for other configs}
  /plugins
    /pdp
      /mypdpplugin1 -- folder for src of user plugin mypdpplugin1

tacss/pdp plugin startup sequence

The startup sequence is as follows (see startup shutdown design:

1. . the previously saved list of plugins is restored to the .plugins collection
2. . the system and user plugin folders are added to the list of plugin folders to search
3. . the plugin folders are searched -- every file is assumed to be a plugin, and an attempt is made to probe it
4. . the list of successfully probed plugins is reconciled with the previously stored list -- newly discovered plugins are added, but old ones that aren't found are not deleted (these must be deleted manually)
5. . all available plugins marked "enabled" are then loaded -- this means they are are integrated into the tacss type system, and their initialization routine is called
6. . unused plugins are released from memory (whether they can be unloaded at a system level or not is implementation and platform-dependent)

Using plugins in projects

To use a plugin's objects in a project, simply create the desired object types as usual. The plugin dependency will be added automatically to the project.

When the project is saved, the list of plugin dependencies for that project will be created in the .plugin_deps collection -- you can browse this list to see what plugins your project depends on. (Note that this collection is managed automatically -- adding or deleting objects from this collection has no effect.)

TBD: maybe should have a "warn on plugin deps" option that warns user before saving project, or maybe when adding plugin objs

Determining Plugin Dependency of Classes

To find out what plugin a class is defined in, use the Class Browser ("Tools/Class Browser"). Find the class in question (either browsing, or with Find) -- the plugin (if any) will be indicated. If no plugin is indicated, the class is a built-in class.

TBD: given a plugin, what classes does it offer???

Creating a Plugin

The easiest way to create a plugin is to copy the sample plugin HelloPlugin and customize it. Assume you want to create the plugin "myplugin"

1. . create a new subfolder for the plugin
2. . copy the files: hw.pro, hw.h, hw.cpp to the new folder
3. . rename the files myplugin.pro etc.
4. . edit the file myplugin.pro and replace "hw" with "myplugin"
5. . follow the instructions for building a plugin

Building Plugins

System Plugins

The plugin build system is integrated with the main pdp build system -- when you rerun configure in the main system, it automatically reconfigures the plugins. When you run 'make' (or commands like 'make install') from the main pdp folder, it automatically builds the plugins.

The very first time a plugin is created, you need to go to the plugins folder and run the 'make qmake' command before you can run any make commands in the plugin itself. Note that for linux users, you may have Qt 3.x on the system, and so may need to issue the full path to the Qt4.2 qmake, ex. /usr/local/Trolltech/[version]/bin/qmake.

After running qmake the first time, you can run make thereafter.

There is a Makefile in the plugins folder -- you can also run make in that folder to rebuild all plugins

Makefile commands:

from /plugins folder:

• make -- runs qmake then make in each plugin folder -- this insures that the local Makefile in each plugin folder is up to date, and that the plugin itself is built
• make qmake -- runs qmake in each plugin folder -- you can do this when you add a new plugin
• make clean -- runs 'make clean' in each plugin folder
• make install -- copies the local lib/plugin_* folders to the pdp4 system folder (you must have administrator privileges for this)

from an individual plugin folder:

• qmake -- builds or rebuilds the Makefile
• make -- builds the plugin (debug and release versions
• make clean -- deletes object files, and the plugin itself; also deletes the Makefile

User Plugins

Building user plugins is similiar to system plugins, but generally simpler. Your plugins folder is automatically created when you first run pdp, and the build files are kept up to date. You must run 'make' in the plugins folder to build your plugins. You never run 'make install' for user plugins.

A user plugin can be moved to the system plugins area by an administrator. No changes are needed to the source files in your plugin. But you do need to make sure that the user plugin is moved out of the user area, since you cannot have the same plugin loaded twice.

Misc

If you want to disable a plugin from being built, you can place an empty file called NOBUILD into that plugin folder -- it will be skipped for all operations except 'clean'.

Qt resources

• http://doc.trolltech.com/4.2/plugins-howto.html
• http://doc.trolltech.com/4.2/qpluginloader.html
• http://doc.trolltech.com/4.2/tools-plugandpaint.html