<?xml version="1.0" encoding="UTF-8"?>
<!--
  ~ ModeShape (http://www.modeshape.org)
  ~
  ~ See the COPYRIGHT.txt file distributed with this work for information
  ~ regarding copyright ownership.  Some portions may be licensed
  ~ to Red Hat, Inc. under one or more contributor license agreements.
  ~ See the AUTHORS.txt file in the distribution for a full listing of 
  ~ individual contributors.
  ~
  ~ ModeShape is free software. Unless otherwise indicated, all code in ModeShape
  ~ is licensed to you under the terms of the GNU Lesser General Public License as
  ~ published by the Free Software Foundation; either version 2.1 of
  ~ the License, or (at your option) any later version.
  ~
  ~ ModeShape is distributed in the hope that it will be useful,
  ~ but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
  ~ or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU Lesser General Public License
  ~ for more details.
  ~
  ~ You should have received a copy of the GNU Lesser General Public License
  ~ along with this distribution; if not, write to:
  ~ Free Software Foundation, Inc.
  ~ 51 Franklin Street, Fifth Floor
  ~ Boston, MA  02110-1301  USA
  -->
<!DOCTYPE preface PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN" "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd"	[
<!ENTITY % CustomDTD SYSTEM "../custom.dtd">
%CustomDTD;
]>
<chapter id="introduction">
  <title>Introduction to ModeShape</title>
	<para>
		ModeShape is a <ulink url="&JSR283;">JCR</ulink> implementation that provides access to content stored in many different kinds of systems. 
		A ModeShape repository isn't yet another silo of isolated information, but rather it's a JCR view of the information 
		you already have in your environment: files systems, databases, other repositories, services, applications, etc.
	</para>
	<para>
		To your applications, ModeShape looks and behaves like a regular JCR repository.  Using the standard JCR 2.0 API (a.k.a. <ulink url="&JSR283;">JSR-283</ulink>),
		applications can search, navigate, version, and listen for changes in the content.  But under the covers, ModeShape
		gets its content by federating multiple back-end systems (like databases, services, other repositories, etc.),
		allowing those systems to continue "owning" the information while ensuring the unified repository stays up-to-date
		and in sync.
	</para>
	<para>
		Of course when you start providing a unified view of all this information, you start recognizing the need to store 
		more information, including metadata about and relationships between the existing content. ModeShape lets you do this, too. 
		And ModeShape even tries to help you discover more about the information you already have, especially the information 
		wrapped up in the kinds of files often found in enterprise systems: service definitions, policy files, images, media, 
		documents, presentations, application components, reusable libraries, configuration files, application installations, 
		databases schemas, management scripts, and so on. As files are loaded into the repository, you can make ModeShape
		automatically sequence these files to extract from their content meaningful information that can be stored in the repository, 
		where it can then be searched, accessed, and analyzed using the JCR API.
	</para>
	<para>
		This document goes into detail about how ModeShape works to provide these capabilities.  It also talks in detail
		about many of the parts within ModeShape - what they do, how they work, and how you can extend or customize the
		behavior.  In particular, you'll learn about ModeShape <link linkend='connector-framework'>connectors</link>
		and <link linkend='sequencing-framework'>sequencers</link>, how you can use the implementations included in ModeShape,
		and how you can write your own to tailor ModeShape for your needs.
	</para>
	<para>
		So whether you are a developer on the project, or you're trying to learn the intricate details of
		how ModeShape works, this document hopefully serves a good reference for developers on the project.
	</para>
	<sect1 id="use_cases">
		<title>Use cases for ModeShape</title>
		<para>
			ModeShape repositories can be used in a variety of applications.  One of the more obvious use cases for a metadata repository
			is in provisioning and management, where it's critical to understand and keep track of the metadata for models, database, services, 
			components, applications, clusters, machines, and other systems used in an enterprise.  Governance takes that a step 
			farther, by also tracking the policies and expectations against which performance of the systems described by the repository can be verified.
			In these cases, a repository is an excellent mechanism for managing this complex and highly-varied information.
		</para>
		<para>
			But these large and complex use cases aren't the only way to use a ModeShape repository.  You could use an embedded ModeShape repository
			to manage configuration information for an application, or you could use ModeShape just to provide a JCR interface on top of a few non-JCR systems.
		</para>
		<para>
			The point is that ModeShape can be used in many different ways, ranging from the very tiny embedded repository to a large and distributed
			enterprise-grade repository.  The choice is yours.
		</para>
	</sect1>
	<sect1 id="what_is_metadata">
		<title>What is metadata?</title>
		<para>
		Before we dive into more detail about ModeShape and metadata repositories, it's probably useful to explain what we
		mean by the term "metadata."  Simply put, <emphasis>metadata</emphasis> is the information you need to manage something.  
		For example, it's the information needed to configure an operating system, or the description of the information in an LDAP tree, 
		or the topology of your network. It's the configuration of an application server or enterprise service bus.
		It's the steps involved in validating an application before it can go into production. It's the description of your 
		database schemas, or of your services, or of the messages going in and coming out of a service. ModeShape is 
		designed to be a repository for all this (and more).
		</para>
		<para>
		There are a couple of important things to understand about metadata.  First, many systems manage (and frequently change) their own metadata and information.
		Databases, applications, file systems, source code management systems, services, content management systems, and even other repositories 
		are just a few types of systems that do this. We can't pull the information out and duplicate it, because
		then we risk having multiple copies that are out-of-sync.  Ideally, we could access all of this information through a homogenous API
		that also provides navigation, caching, versioning, search, and notification of changes.  That would make our lives significantly easier.
		</para>
		<para>
		What we want is <emphasis><link linkend="modeshape-connector-federation">federation</link></emphasis>.
		We can connect to these back-end systems to dynamically access the content and project it into a single, unified
		repository.  We can cache it for faster access, as long as the cache can be invalidated based upon time or event.
		But we also need to maintain a clear picture of where all the bits come from, so users can be sure they're looking
		at the right information.  And we need to make it as easy as possible to write new connectors, since there are
		a lot of systems out there that have information we want to federate.
		</para>
		<para>
		The second important characteristic of the metadata is that a lot of it is represented as files, and there are
		a lot of different file formats.  These include source code, configuration files, web pages, database schemas,
		XML schemas, service definitions, policies, documents, spreadsheets, presentations, images, audio files, workflow
		definitions, business rules, and on and on.  And logically if files contain metadata, we want to add those files
		to our metadata repository.  The problem is, all that metadata is tied up as blobs in the repository.
		Ideally, our repository would automatically extract from those files the content that's most useful to us,
		and place that content inside the repository where it can be much more easily used, searched, related, and analyzed.
		ModeShape does exactly this via a process we call <emphasis><link linkend="sequencing">sequencing</link></emphasis>,
		and it's an important part of a metadata repository.
		</para>
		<para>
		The third important characteristic of metadata is that it rarely stays the same.  Different consumers of the
		information need to see different views of it.  Metadata about two similar systems is not always the same.
		The metadata often needs to be tagged or annotated with additional information.  And the things being
		described often change over time, meaning the metadata has to change, too.  As a result, the way in which
		we store and manage the metadata has to be flexible and able to adapt to our ever-changing needs, and the object model
		we use to interact with the repository must accommodate these needs.  The graph-based nature of the JCR API provides this 
		flexibility while also giving us the ability to constrain information when it needs to be constrained.
		</para>
	</sect1>
  <sect1 id="jcr_intro">
    <title>What is JCR?</title>
	  <para>There are a lot of choices for how applications can store information persistently so that it can be accessed at a
	    later time and by other processes. The challenge developers face is how to use an approach that most closely matches the
	    needs of their application. This choice becomes more important as developers choose to focus their efforts on
	    application-specific logic, delegating much of the responsibilities for persistence to libraries and frameworks.</para>
	  <para>
	    Perhaps one of the easiest techniques is to simply store information in
	    <emphasis>files</emphasis>
	    . The Java language makes working with files relatively easy, but Java really doesn't provide many bells and whistles. So
	    using files is an easy choice when the information is either not complicated (for example property files), or when users may
	    need to read or change the information outside of the application (for example log files or configuration files). But using
	    files to persist information becomes more difficult as the information becomes more complex, as the volume of it increases,
	    or if it needs to be accessed by multiple processes. For these situations, other techniques often have more benefits.
	  </para>
	  <para>
	    Another technique built into the Java language is
	    <emphasis>Java serialization</emphasis>
	    , which is capable of persisting the state of an object graph so that it can be read back in at a later time. However, Java
	    serialization can quickly become tricky if the classes are changed, and so it's beneficial usually when the information is
	    persisted for a very short period of time. For example, serialization is sometimes used to send an object graph from one
	    process to another.  Using serialization for longer-term storage of information is far less useful.
	  </para>
	  <para>
	    One of the more popular and widely-used persistence technologies is the <emphasis>relational database</emphasis>. 
			Relational database management systems have been around for decades and are very capable. The Java Database Connectivity
	    (JDBC) API provides a standard interface for connecting to and interacting with relational databases. However, it is a
	    low-level API that requires a lot of code to use correctly, and it still doesn't abstract away the DBMS-specific SQL
	    grammar. Also, working with relational data in an object-oriented language can feel somewhat unnatural, so many developers
	    map this data to classes that fit much more cleanly into their application. The problem is that manually creating this
	    mapping layer requires a lot of repetitive and non-trivial JDBC code.
	  </para>
	  <para>
	    <emphasis>Object-relational mapping</emphasis>
	    libraries automate the creation of this mapping layer and result in far less code that is much more maintainable with
	    performance that is often as good as (if not better than) handwritten JDBC code. The
	    <ulink url="http://java.sun.com/developer/technicalArticles/J2EE/jpa/">Java Persistence API (JPA)</ulink>
	    provide a standard mechanism for defining the mappings (through annotations) and working with these entity objects. Several
	    commercial and open-source libraries implement JPA, and some even offer additional capabilities and features that go beyond
	    JPA. For example, <ulink url="http://www.hibernate.org">Hibernate</ulink> is one of the most feature-rich JPA implementations
			and offers object caching, statement caching, extra association
	    mappings, and other features that help to improve performance and usefulness.  Plus, Hibernate is open-source (with support
			offered by <ulink url="http://www.jboss.com">JBoss</ulink>).
	  </para>
	  <para>
	    While relational databases and JPA are solutions that work well for many applications, they are more limited in cases when the
	    information structure is highly flexible, the structure is not known <emphasis>a priori</emphasis>, or that structure is
	 		subject to frequent change and customization. In these situations, <emphasis>content repositories</emphasis>
	    may offer a better choice for persistence. Content repositories offer the storage capabilities of
	 		relational databases with the flexibility offered by other systems, such as using files.  Content repositories also
	    typically provide other capabilities as well, including hierarchical organization, versioning, indexing, search, access control,
	    transactions, and observation. Content repositories are often used by content management systems (CMS), document
	    management systems (DMS), and other applications that manage electronic files (e.g., documents, images, multi-media, web
	    content, etc.) and metadata associated with them (e.g., author, date, status, security information, etc.). The
	    <ulink url="&JSR283;">Content Repository for Java technology API</ulink>
	    provides a standard Java API for working with content repositories. Abbreviated "JCR", this API was developed through the
	    Java Community Process originally under <ulink url="&JSR170;">JSR-170</ulink> (as "JCR 1.0"), but has
	    since been revised and improved as "JCR 2.0" under <ulink url="&JSR283;">JSR-283</ulink>.
	  </para>
	  <para>
		  The JCR 2.0 API provides a number of information services that are needed by many applications,
		  including: read and write access to information; the ability to structure information in a hierarchical and flexible manner that can adapt
		  and evolve over time; ability to work with structured, semi-structured, and unstructured content; ability to (transparently) handle large strings; 
		  notifications of changes in the information; search and query; versioning of information; access control; integrity constraints;
		  participation within distributed transactions; explicit locking of content; and of course persistence.
		</para>
		<para>
			ModeShape implements the JCR 2.0 API, including many of the optional features.
		</para>
		<figure id="jcr-features">
			<title>JCR API features</title>
			<graphic align="center" scale="100" fileref="jcr-features.png"/>
		</figure>
	</sect1>
  <sect1 id="roadmap">
    <title>Project roadmap</title>
    <para>
	    The ModeShape open source project uses its 
      <ulink url="&JIRA;">JIRA instance</ulink>
      to track issues for tasks, requirements, bugs, and other activities. The
      <ulink url="&Roadmap;">roadmap report</ulink>
      shows how each of these issues are targeted to the upcoming releases, while the <ulink url="&ChangeLog;">change log report</ulink>
      shows all of the issues that were fixed in each of the past releases.
    </para>
    <para>
      By convention, the ModeShape project team periodically review JIRA issues that aren't targeted to a release, and then schedule
 			them based upon current workload, severity, and the roadmap.  And if we review an issue and don't know how to target it,
			we target it to the
      <ulink url="&FutureReleases;">Future Releases</ulink>
      bucket.
		</para>
		<para>
			At the start of a release, the project team reviews the roadmap, identifies the goals for the release, and targets (or retargets)
			the issues appropriately.
    </para>
  </sect1>
  <sect1 id="modules">
    <title>ModeShape modules</title>
    <para>
      ModeShape consists of quite a few separate modules. Just a few of these make up the essential core components of the system:
      <itemizedlist>
        <listitem>
          <para>
            <emphasis role="strong">modeshape-jcr</emphasis>
            contains ModeShape's implementation of the JCR 2.0 API. If you're using ModeShape as a JCR repository, this is the
 						top-level dependency that you'll want to use.  The module defines all required dependencies, except for
 						the repository connector(s) and any sequencer implementations needed by your configuration.
						As we'll see later on, using ModeShape as a JCR repository is as easy as defining a <link linkend="configuration">configuration</link>,
						obtaining the JCR &Repository; object for your repository using the &RepositoryFactory;, and then using the standard JCR API.
						This module also uses the JCR unit tests from the reference implementation to verify the behavior of the ModeShape implementation.
          </para>
        </listitem>
        <listitem>
          <para>
            <emphasis role="strong">modeshape-jcr-api</emphasis>
            defines a number of interfaces that extend several of the JCR API interfaces. 
            For example, this module defines a &Repositories; interface that defines a way to look up <code>javax.jcr.Repository</code> 
            instances by name, and that is implemented by the ModeShape <code>JcrEngine</code>. It also defines several new
            interfaces that extend the JCR 2.0 API's Query Object Model with additional behavior, including more criteria options
            (such as &Between;, the &NodeDepth; and &NodePath; pseudo-columns, and the &ReferenceValue; function), formal &Limit; and &Offset; clauses, and a &SetQuery; for unions, intersects,
            and difference queries.
            This module is very small, only depends upon the JCR API, and was designed so that client applications can depend
            only upon this module without having to depend on the <code>modeshape-jcr</code> interfaces or its dependencies.
            For example, this module defines a &Repositories; interface that defines a way to look up <code>javax.jcr.Repository</code> 
            instances by name, and that is implemented by the ModeShape <code>JcrEngine</code>, allowing client applications to
            dependency on this module to look up repositories by name <emphasis>without</emphasis> having to depend on the 
          </para>
        </listitem>
      </itemizedlist>
			Several other modules are also essential, but for the most part are hidden to client applications as they provide components used
			within the JCR implementation:
      <itemizedlist>
        <listitem>
          <para>
            <emphasis role="strong">modeshape-repository</emphasis>
            provides the core ModeShape graph engine and services for managing repository connections, sequencers, MIME type detectors,
						and observation. If you're using ModeShape repositories via our graph API rather than JCR, then this is where you'd start.
          </para>
        </listitem>
        <listitem>
          <para>
            <emphasis role="strong">modeshape-cnd</emphasis>
            provides a self-contained utility for parsing CND (Compact Node Definition) files and transforming
						the node definitions into a graph notation compatible with ModeShape's JCR implementation.
          </para>
        </listitem>        
        <listitem>
          <para>
            <emphasis role="strong">modeshape-graph</emphasis>
            defines the Application Programming Interface (API) for ModeShape's low-level graph model,
						including a fluent-style API for working with graph content.  This module also defines the
 						APIs necessary to implement custom connectors, sequencers, and MIME type detectors.
          </para>
        </listitem>
        <listitem>
          <para>
            <emphasis role="strong">modeshape-common</emphasis>
            is a small low-level library of common utilities and frameworks, including logging, progress monitoring,
            internationalization/localization, text translators, component management, and class loader factories.
          </para>
        </listitem>
      </itemizedlist>
      Most of the ModeShape modules, however, are optional extensions. Many of these depend on third party libraries, so you 
      will probably want to include only those modules that provide functionality you'll use in your repository.
      These modules are located in the source under the <code>extensions/</code> directory.
      <itemizedlist>
        <listitem>
          <para>
            <emphasis role="strong">modeshape-clustering</emphasis>
            contains ModeShape's clustering components and are needed only when two or more ModeShape engines are
            to be clustered together (so listeners in one session get notifications made from within any of the engines).
            ModeShape clustering uses the powerful, flexible and mature <ulink url="&JGroups;">JGroups</ulink> reliable multicast 
            communication library. Simply <link linkend="clustering">enable clustering</link> in ModeShape's configuration,
            include this library, and start your cluster. Engines can be dynamically added and removed from the cluster.
          </para>
        </listitem>
        <listitem>
          <para>
            <emphasis role="strong">modeshape-connector-infinispan</emphasis>
            is the preferred ModeShape repository connector for persistently storing content.
            <ulink url="http://infinispan.org">Infinispan</ulink> is an extremely scalable, highly available data grid platform
 						that distributes the data across the nodes in the grid.
            This connector makes it possible for repository content to be stored in a very efficient, fast,
            highly-concurrent (essentially lock- and synchronization-free), and reliable manner, 
            even when the content size grows to massive sizes. This connector is capable of storing any kind of content, and
						dictates how the content is stored on the data grid.  Therefore, this connector cannot be used to access the content
						of existing data grids created by/for other applications.
          </para>
        </listitem>
        <listitem>
          <para>
            <emphasis role="strong">modeshape-connector-jbosscache</emphasis>
            is a ModeShape repository connector that stores content within a
            <ulink url="http://www.jboss.org/jbosscache/">JBoss Cache</ulink>
            instance. JBoss Cache is a powerful cache implementation that can serve as a distributed cache and that can persist
            information. The cache instance can be found via JNDI or created and managed by the connector.
						This connector is capable of storing any kind of content, and dictates how the content is stored in the cache.
						Therefore, this connector cannot be used to access the content
						of existing cache instances created by/for other applications.
          </para>
        </listitem>
        <listitem>
          <para>
            <emphasis role="strong">modeshape-connector-jdbc-metadata</emphasis>
            is a ModeShape repository connector that provides read-only access to metadata and schema information from relational databases 
						through a JDBC connection.  This connector provides an optional and configurable caching facility to prevent frequent
						requests to the database.
          </para>
        </listitem>        
        <listitem>
          <para>
            <emphasis role="strong">modeshape-connector-store-jpa</emphasis>
            is a ModeShape repository connector that stores content in a JDBC database, using the Java Persistence API (JPA) and the
						very highly-regarded and widely-used <ulink url="http://www.hibernate.org">Hibernate</ulink> implementation.
						This connector is capable of storing any kind of content, and dictates the schema in which it stores the content.
						Therefore, this connector cannot be used to access the data in existing created by/for other applications.
          </para>
        </listitem>
        <listitem>
          <para>
            <emphasis role="strong">modeshape-connector-jcr</emphasis>
            is a ModeShape repository connector that accesses and stores content in an external JCR 2.0 repository. This allows
            ModeShape to integrate with other JCR implementations and even federate multiple JCR repositories into 
            a single unified repository. Any differences in namespaces are automatically handled, although node types used by
            the content in the external JCR repository must also be registered into the ModeShape repository using the connector.
            <emphasis>Note that this connector is currently a technical preview, and we're seeking feedback and assistance in
	          identifying the required functionality.</emphasis>
          </para>
        </listitem>
        <listitem>
          <para>
            <emphasis role="strong">modeshape-connector-filesystem</emphasis>
            is a ModeShape repository connector that accesses the files and folders on (a part of) the local file system, providing that
  					content in the form of <code>nt:file</code> and <code>nt:folder</code> nodes. This connector <emphasis>does</emphasis>
						support updating the file system when changes are made to the <code>nt:file</code> and <code>nt:folder</code> nodes.
						However, this connector does not support storing other kinds of nodes.
          </para>
        </listitem>
        <listitem>
          <para>
            <emphasis role="strong">modeshape-connector-svn</emphasis>
            is a ModeShape repository connector that accesses the content of an existing Subversion repository, providing that content in
						the form of <code>nt:file</code> and <code>nt:folder</code> nodes. This connector <emphasis>does</emphasis>
						support updating the SVN repository when changes are made to the <code>nt:file</code> and <code>nt:folder</code> nodes.
						However, this connector does not support storing other kinds of nodes.
          </para>
        </listitem>
        <listitem>
          <para>
            <emphasis role="strong">modeshape-sequencer-cnd</emphasis>
            is a ModeShape sequencer that extracts JCR node definitions from JCR Compact Node Definition (CND) files.
          </para>
        </listitem>
        <listitem>
          <para>
            <emphasis role="strong">modeshape-sequencer-ddl</emphasis>
            is a ModeShape sequencer that extracts the structure and content from DDL files.
            <emphasis>This is still under development and includes support for the basic DDL statements in
							in the Oracle, PostgreSQL, Derby, and standard DDL dialects.</emphasis>
          </para>
        </listitem>
        <listitem>
          <para>
            <emphasis role="strong">modeshape-sequencer-zip</emphasis>
            is a ModeShape sequencer that extracts the files (with content) and directories from ZIP archives.
          </para>
        </listitem>
        <listitem>
          <para>
            <emphasis role="strong">modeshape-sequencer-xml</emphasis>
            is a ModeShape sequencer that extracts the structure and content from XML files.
          </para>
        </listitem>
        <listitem>
          <para>
            <emphasis role="strong">modeshape-sequencer-xsd</emphasis>
            is a ModeShape sequencer that extracts the structure and content from XML Schema Definition (XSD) files.
          </para>
        </listitem>
        <listitem>
          <para>
            <emphasis role="strong">modeshape-sequencer-wsdl</emphasis>
            is a ModeShape sequencer that extracts the structure and content from Web Service Definition Language (WSDL) 1.1 files.
          </para>
        </listitem>
        <listitem>
          <para>
            <emphasis role="strong">modeshape-sequencer-sramp</emphasis>
            is a library with reusable node types patterned after the core model of S-RAMP, and used by other ModeShape sequencers.
          </para>
        </listitem>
        <listitem>
          <para>
            <emphasis role="strong">modeshape-sequencer-classfile</emphasis>
            is a ModeShape sequencer that extracts the package, class/type, member, documentation, annotations, and other information
            from Java class files.
          </para>
        </listitem>
        <listitem>
          <para>
            <emphasis role="strong">modeshape-sequencer-java</emphasis>
            is a ModeShape sequencer that extracts the package, class/type, member, documentation, annotations, and other information
            from Java source files.
          </para>
        </listitem>
        <listitem>
          <para>
            <emphasis role="strong">modeshape-sequencer-jbpm-jpdl</emphasis>
            is a prototype ModeShape sequencer that extracts process definition metadata from jBPM process definition language (jPDL) files.
						<emphasis>This is still under development.</emphasis>
          </para>
        </listitem>
        <listitem>
          <para>
            <emphasis role="strong">modeshape-sequencer-msoffice</emphasis>
            is a ModeShape sequencer that extracts metadata and summary information from
            <ulink url="http://office.microsoft.com/en-us/">Microsoft Office</ulink>
            documents. For example, the sequencer extracts from a PowerPoint presentation the outline as well as thumbnails of
            each slide. Microsoft Word and Excel files are also supported.
          </para>
        </listitem>
        <listitem>
          <para>
            <emphasis role="strong">modeshape-sequencer-images</emphasis>
            is a ModeShape sequencer that extracts the image metadata (e.g., size, date, etc.) from PNG, JPEG, GIF, BMP, PCS, IFF,
            RAS, PBM, PGM, and PPM image files.
          </para>
        </listitem>
        <listitem>
          <para>
            <emphasis role="strong">modeshape-sequencer-mp3</emphasis>
            is a ModeShape sequencer that extracts metadata (e.g., author, album name, etc.) from MP3 audio files.
          </para>
        </listitem>
        <listitem>
          <para>
            <emphasis role="strong">modeshape-sequencer-teiid</emphasis>
            contains two sequencers. ModelSequencer extracts the structured data model contained with a Teiid relational XMI model, 
						including the catalogs, schemas, tables, views, columns, primary keys, foreign keys, indexes, procedures, 
						procedure parameters, procedure results, logical relationships, and the JDBC source from which the model was imported.
						Teiid VDB files contain several models, so the VdbSequencer extracts the virtual database metadata and
						the structured data model from each of the models contained within the VDB.
          </para>
        </listitem>  
        <listitem>
          <para>
            <emphasis role="strong">modeshape-sequencer-text</emphasis>
            is a ModeShape sequencer that extracts data from text streams. There are separate sequencers for character-delimited sequencing 
						and fixed width sequencing, but both treat the incoming text stream as a series of rows separated by line-terminators
						with each row consisting of one or more columns. 
          </para>
        </listitem>  
        <listitem>
          <para>
            <emphasis role="strong">modeshape-search-lucene</emphasis> is an implementation of the &SearchEngine; interface that
						uses the <ulink url="http://lucene.apache.org/java/">Lucene</ulink> library. This module is one of the few
						extensions that is used directly by the <code>modeshape-jcr</code> module.
          </para>
        </listitem>
        <listitem>
          <para>
            <emphasis role="strong">modeshape-mimetype-detector-aperture</emphasis>
            is a &MimeTypeDetector; implementation that uses the
            <ulink url="http://aperture.sourceforge.net/">Aperture</ulink>
            library to determine the best MIME type given the name and contents of a file.
          </para>
        </listitem>
        <listitem>
          <para>
            <emphasis role="strong">modeshape-extractor-tika</emphasis>
            is a &TextExtractor; implementation that uses the
            <ulink url="http://aperture.sourceforge.net/">Apache Tika</ulink>
            parsing library to extract from binary content text that can be used for indexing the content.
          </para>
        </listitem>
        <listitem>
          <para>
            <emphasis role="strong">modeshape-classloader-maven</emphasis> is a small library that provides a
            &ClassLoaderFactory; implementation that can create &ClassLoader; instances capable of loading classes given 
						a Maven Repository and a list of Maven coordinates. The Maven Repository can be managed within a JCR repository.
          </para>
        </listitem>
      </itemizedlist>
      The following modules make up the various web application projects (and are located in the source
      under the
      <code>web/</code>
      directory). You may be able to use these artifacts "out of the box", but more likely the configuration defined in the WAR files
      will not be exactly what you want for your environment. In this case, you can replicate one of our "-war" modules and customize
      the configuration settings to easily assembly a custom WAR.
      <itemizedlist>
        <listitem>
          <para>
            <emphasis role="strong">modeshape-web-jcr-webdav</emphasis>
            provides a WebDAV server for Java Content Repositories.  This project provides integration with ModeShape's 
						JCR implementation (of course) but also contains a service provider interface (SPI) that can be 
            used to integrate other JCR implementations with these WebDAV services in the future.  
						For ease of packaging, these classes are provided as a JAR that can be placed in the WEB-INF/lib of a deployed WebDAV server WAR.
          </para>
        </listitem>
        <listitem>
          <para>
            <emphasis role="strong">modeshape-web-jcr-webdav-war</emphasis>
            wraps the WebDAV services from the modeshape-web-jcr-webdav JAR into a WAR and provides in-container integration tests.  This project
            can be consulted as a template for how to deploy the WebDAV services in a custom implementation. 
          </para>
        </listitem>
        <listitem>
          <para>
            <emphasis role="strong">modeshape-web-jcr-rest</emphasis>
            provides a set of JSR-311 (JAX-RS) objects that form the basis of a RESTful server for Java Content Repositories.  This project 
            provides integration with ModeShape's JCR implementation (of course) but also contains a service provider interface (SPI) that can be 
            used to integrate other JCR implementations with these RESTful services in the future.  For ease of packaging, these classes are
            provided as a JAR that can be placed in the WEB-INF/lib of a deployed RESTful server WAR.
          </para>
        </listitem>
        <listitem>
          <para>
            <emphasis role="strong">modeshape-web-jcr-rest-war</emphasis>
            wraps the RESTful services from the modeshape-web-jcr-rest JAR into a WAR and provides in-container integration tests.  This project
            can be consulted as a template for how to deploy the RESTful services in a custom implementation. 
          </para>
        </listitem>
        <listitem>
          <para>
            <emphasis role="strong">modeshape-web-jcr-rest-client</emphasis>
            is a library that uses POJOs to access the REST web service. This module eliminates the need for applications to know how
            to create HTTP request URLs and payloads, and how to parse the JSON responses. It can be used to publish (upload) 
            and unpublish (delete) files from ModeShape repositories. 
          </para>
        </listitem>
        <listitem>
          <para>
            <emphasis role="strong">modeshape-web-jcr</emphasis>
            provides a reusable library for web applications using JCR, and is used by the <code>modeshape-web-jcr-rest</code>
						and <code>modeshape-web-jcr-webdav</code> modules.
          </para>
        </listitem>
      </itemizedlist>
      ModeShape recently added several modules that make it very easy to
			<link linkend="deloying_modeshape_to_jbossas">deploy ModeShape in JBoss AS or EAP</link> as a
  		full-fledged, central, shared service that can be monitored and administered using the embedded console
			and used directly by web applications deployed to the application server. Our Maven build produces
			a "kit" ZIP file that can be unzipped into a JBoss AS profile. When your server restarts, ModeShape will be
			running with a very simple configuration (although that can be easily changed).
		</para>
		<para>
			The modules that make up the JBoss AS deployment kit are located in the source under the "<code>deploy/jbossas</code> directory":
      <itemizedlist>
        <listitem>
          <para>
            <emphasis role="strong">modeshape-jbossas-service</emphasis>
						provides several components that are deployed through the microcontainer in JBoss AS, registered
						in JNDI, and exposed through the Profile Service for monitoring and management. This service leverages
						the JAAS support within the application server.
          </para>
        </listitem>
        <listitem>
          <para>
            <emphasis role="strong">modeshape-jbossas-console</emphasis>
						defines the plugin for <ulink url="http://support.rhq-project.org/display/RHQ/Home">RHQ</ulink>
						that enables administration, monitoring, alerting, operational
						control and configuration. All of the major components within a ModeShape engine are exposed as RHQ
						resources, and the plugin provides a number of metrics and administrative operations as well as
						exposing most configuration properties.
						<emphasis>(We plan to add more metrics and operations over the next few releases, as we gain more
						experience using the ModeShape RHQ plugin.)</emphasis>
          </para>
        </listitem>
        <listitem>
          <para>
            <emphasis role="strong">modeshape-jbossas-web-rest-war</emphasis>
						defines a variant of the more general <code>modeshape-web-rest-war</code> that is tailored for deployment on JBoss AS,
						since it reuses the same ModeShape service deployed into the application server.
          </para>
        </listitem>
        <listitem>
          <para>
            <emphasis role="strong">modeshape-jbossas-web-webdav-war</emphasis>
						defines a variant of the more general <code>modeshape-web-webdav-war</code> that is tailored for deployment on JBoss AS,
						since it reuses the same ModeShape service deployed into the application server.
          </para>
        </listitem>
      </itemizedlist>
      There are also modules for ModeShape's documentation (located in the source under the <code>docs/</code> directory):
      <itemizedlist>
        <listitem>
          <para>
            <emphasis role="strong">docs-getting-started</emphasis>
            is the project with the
            <ulink url="http://www.docbook.org/">DocBook</ulink>
            source for the ModeShape Getting Started document.
          </para>
        </listitem>
        <listitem>
          <para>
            <emphasis role="strong">docs-getting-started-examples</emphasis>
            is the project with the Java source for the example application used in the ModeShape Getting Started document.
          </para>
        </listitem>
        <listitem>
          <para>
            <emphasis role="strong">docs-reference-guide</emphasis>
            is the project with the
            <ulink url="http://www.docbook.org/">DocBook</ulink>
            source for this document, the ModeShape Reference Guide document.
          </para>
        </listitem>
      </itemizedlist>
      There are several utility modules:
      <itemizedlist>
        <listitem>
          <para>
            <emphasis role="strong">modeshape-jpa-ddl-gen</emphasis> provides a standalone utility that can generate the DDL
						for the database schema used by the JPA connector. Because it uses Hibernate, it can generate DDL for any
						of the databases that the connector can use.  This is also useful for users who prefer not to give DDL privileges 
						to the ModeShape database user.
          </para>
        </listitem>
        <listitem>
          <para>
            <emphasis role="strong">modeshape-jdbc-local</emphasis> provides a JDBC driver implementation that allows JDBC clients 
						to query the contents of a <emphasis>local</emphasis> JCR repository using <link linkend="jcr-sql2-query-language">JCR-SQL2</link>. 
						The driver even supports JDBC metadata, making it possible to dynamically discover the tables and columns available for querying
						(which are determined from the node types). It can be configured as a data source in JBoss AS, and can even
						leverage the ModeShape service, allowing JDBC-based access by clients deployed to that JBoss AS instance
						to query the repository content. This library is very lightweight and fast, since it directly accesses the repository
						using the JCR API.
					</para>
        </listitem>
        <listitem>
          <para>
            <emphasis role="strong">modeshape-jdbc</emphasis> provides a JDBC driver implementation that allows JDBC clients 
						to query the contents of a <emphasis>local or remote</emphasis> JCR repository using <link linkend="jcr-sql2-query-language">JCR-SQL2</link>. 
						The driver even supports JDBC metadata, making it possible to dynamically discover the tables and columns available for querying
						(which are determined from the node types). It can be configured as a data source in JBoss AS, and can even
						leverage the ModeShape service, allowing JDBC-based access to the same repository content available
						via the JCR API, RESTful service, or WebDAV.
					</para>
        </listitem>
      </itemizedlist>
      There is another module that runs the full suite of JCR TCK tests, and which at the moment still contains a few failures.
      This module is never needed in client applications.
      <itemizedlist>
        <listitem>
          <para>
            <emphasis role="strong">modeshape-jcr-tck</emphasis> provides a separate testing project that executes all reference implementation's
 						JCR TCK tests on a nightly basis to track implementation progress against the JCR 1.0 specification. 
						This module will likely be retired when the ModeShape JCR implementation is complete, since <code>modeshape-jcr</code> and
						<code>modeshape-integration-tests</code> will be running the full suite of JCR TCK unit tests.
          </para>
        </listitem>
      </itemizedlist>
		  Another module provides system- and integration-level tests and is never needed in client applications:
      <itemizedlist>
        <listitem>
          <para>
            <emphasis role="strong">modeshape-integration-tests</emphasis>
            provides a home for all of the integration tests that involve more components that just unit tests. Integration
            tests are often more complicated, take longer, and involve testing the integration and functionality of multiple
            components (whereas unit tests focus on testing a single class or component and may use stubs or mock objects
            to isolate the code being tested from other related components).
          </para>
        </listitem>
      </itemizedlist>
      Finally, there is a Maven parent <code>pom.xml</code> file that aggregates all of the other projects, provides common
      defaults for Maven plugins and dependency versions used throughout the modules, and definition of various asset files
      to help build the necessary Maven artifacts during a build.
    </para>
  	<para>
      Each of these modules is a Maven project with a group ID of
      <code>org.modeshape</code>
      . All of these projects correspond to <ulink url="&ModeShapeMaven;">artifacts</ulink> in the
      <ulink url="&JBossMaven;">JBoss Maven 2 Repository</ulink>, the settings for which are described on the
      <ulink url="http://community.jboss.org/wiki/MavenGettingStarted-Users">JBoss.org wiki</ulink>.
    </para>
  </sect1>
  <sect1 id="development-tools">
    <title>Compiling and building</title>
    <para>
	    The ModeShape source code is freely available and easy to build. For more information about this, please see our
	    documentation that describes the <ulink url="http://community.jboss.org/wiki/ModeShapeDevelopmentTools">tools</ulink>, 
	    <ulink url="http://community.jboss.org/wiki/ModeShapeandMaven">build commands</ulink>, and even 
			<ulink url="http://community.jboss.org/wiki/ModeShapeDevelopmentWorkflow">how to contribute using Git</ulink>.
		</para>
	</sect1>
  <sect1 id="whats_new">
    <title>What's new?</title>
	  <para>
			ModeShape &versionNumber; includes several improvements and minor features, and numerous fixes for issues reported 
			against the earlier 2.x releases. For details,
			see the <ulink url="&ReleaseNotes;">release notes</ulink>.
		</para>
		<para><emphasis role="strong">ModeShape implements all of the required JCR 2.0 features</emphasis>: 
			repository acquisition, authentication, reading/navigating, query, export, node type discovery, and permissions and 
			capability checking. 
			<emphasis role="strong">ModeShape also implements most of the optional JCR 2.0 features</emphasis>:
			writing, import, observation, workspace management, versioning, locking, node type management, same-name siblings, 
			orderable child nodes, and shareable nodes.
			The remaining optional features (access control management, lifecycle management, 
			retention and hold, and transactions) may be introduced in future versions.
		</para>
	  <note>
		  <para>
			  ModeShape &versionNumber; currently passes 1372 of the 1391 JCR TCK tests, where 17 of these 19 failures appear
			  to be bugs in the TCK tests (see 
			  <ulink url="https://issues.apache.org/jira/browse/JCR-2648">JCR-2648</ulink>, 
			  <ulink url="https://issues.apache.org/jira/browse/JCR-2661">JCR-2661</ulink>, 
			  <ulink url="https://issues.apache.org/jira/browse/JCR-2662">JCR-2662</ulink>, and
			  <ulink url="https://issues.apache.org/jira/browse/JCR-2663">JCR-2663</ulink>). The remaining 2 failures are
			  due to a known issue (see 
			  <ulink url="&MODE;-760">MODE-760</ulink>).
  		</para>
	  </note>
  </sect1>
</chapter>
