<?xml version="1.0" encoding="UTF-8"?>
<!--
  ~ ModeShape (http://www.modeshape.org)
  ~
  ~ See the COPYRIGHT.txt file distributed with this work for information
  ~ regarding copyright ownership.  Some portions may be licensed
  ~ to Red Hat, Inc. under one or more contributor license agreements.
  ~ See the AUTHORS.txt file in the distribution for a full listing of 
  ~ individual contributors.
  ~
  ~ ModeShape is free software. Unless otherwise indicated, all code in ModeShape
  ~ is licensed to you under the terms of the GNU Lesser General Public License as
  ~ published by the Free Software Foundation; either version 2.1 of
  ~ the License, or (at your option) any later version.
  ~
  ~ ModeShape is distributed in the hope that it will be useful,
  ~ but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
  ~ or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU Lesser General Public License
  ~ for more details.
  ~
  ~ You should have received a copy of the GNU Lesser General Public License
  ~ along with this distribution; if not, write to:
  ~ Free Software Foundation, Inc.
  ~ 51 Franklin Street, Fifth Floor
  ~ Boston, MA  02110-1301  USA
  -->
<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN" "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [
<!ENTITY % CustomDTD SYSTEM "../custom.dtd">
%CustomDTD;
]>
<chapter id="sequencer_example">
	<title>The Sequencer Example</title>
	<para>
		The previous chapter walked through the process of downloading and building the examples.
		This chapter will focus on the sequencer example, showing how to run the example and then walking through the code
		to describe what it's doing.
	</para>
	<sect1 id="running_sequencer_example">
		<title>Running the sequencing example</title>
		<para>The sequencing example consists of a client application that sets up an in-memory JCR repository and that allows a user to
      upload files into that repository. The client also sets up the ModeShape services with seven sequencers:
	 		<itemizedlist>
				<listitem>
					<para>an image metadata sequencer that processes PNG, JPEG, GIF, BMP or other image filetypes to extract
						 the image's metadata (e.g., image format, physical size, pixel density, etc.)</para>
				</listitem>
				<listitem>
					<para>an MP3 sequencer that extracts the ID3 metadata (e.g., the author, title, album, year and comment)</para>
				</listitem>
				<listitem>
					<para>a Java source code sequencer that extracts the structure of Java classes by parsing the source code</para>
				</listitem>
				<listitem>
					<para>a Java class file sequencer that extracts the structure of Java classes by analyzing the class files</para>
				</listitem>
				<listitem>
					<para>a text file sequencer that extracts the comma-separated structured information contained in CSV files</para>
				</listitem>
				<listitem>
					<para>a text file sequencer that extracts the structured information contained in fixed-width files</para>
				</listitem>
				<listitem>
					<para>a ZIP archive sequencer that extracts the files and directory structure contained in ZIP and JAR files</para>
				</listitem>
			</itemizedlist>
      These sequencers automatically extract content from the files and store that content in the repository.
    </para>
		<note>
			<para>ModeShape includes several other sequencers, including sequencers for DDL files, Microsoft Office files, 
				JCR Compact Node Definition (CND) files, and XML files. Feel free to experiment with the example and add these
				or even sequencers you write.</para>
		</note>
		<para>
      To run the client application, go to the <code>examples/sequencers/target/modeshape-example-sequencers-basic.dir/</code>
      directory and type <code>./run.sh</code>. You should see the command-line client and its menus in your terminal:
      <figure id="example-sequencer-cli-client">
				<title>Example client</title>
				<graphic align="center" scale="60" fileref="example-sequencer-client.png"/>
			</figure>
      From this menu, you can upload a file into the repository, search for media in the repository, print sequencing statistics,
      or quit the application.</para>
		<para>
      The first step is to upload one of the example images. If you type 'u' and press return, you'll be prompted to supply the
      path to the file you want to upload. Since the application is running from within the
      <code>examples/sequencers/target/modeshape-example-sequencers-basic.dir/</code> directory, you can specify any of the files 
			in that directory without specifying the path:
      <figure id="example-sequencer-upload">
				<title>Uploading an image using the example client</title>
				<graphic align="center" scale="60" fileref="example-sequencer-upload.png"/>
			</figure>
      You can specify any fully-qualified or relative path. The application will notify you if it cannot find the file you
      specified. The example client configures ModeShape to sequence MP3 audio files, Java source files, Java class files,
      ZIP files, text files (<code>.txt</code>), CSV files (<code>.csv</code>), or image files with one of
      the following extensions (technically, nodes that have names ending in the following):
      <code>jpg</code>, <code>jpeg</code>, <code>gif</code>, <code>bmp</code>, <code>pcx</code>, <code>png</code>,
      <code>iff</code>, <code>ras</code>, <code>pbm</code>, <code>pgm</code>, <code>ppm</code>, and <code>psd</code>.
 			Files with other extensions in the repository path will be ignored. For your convenience, the example provides several
      files that will be sequenced (<code>caution.png</code>, <code>caution.jpg</code>, <code>caution.gif</code>, and
      <code>sample1.mp3</code>) and one image that will not be sequenced (<code>caution.pict</code>). Feel free to try other files.
    </para>
		<para>
      After you have specified the file you want to upload, the example application asks you where in the repository you'd like to
      place the file. (If you want to use the suggested location, just press <code>return</code>.) The client application
      uses the JCR API to upload the file to that location in the repository, creating any nodes (of type <code>nt:folder</code>)
      for any directories that don't exist, and creating a node (of type <code>nt:file</code>) for the file. And, per the JCR specification, 
			the application creates a <code>jcr:content</code> node (of type <code>nt:resource</code>) under the file node. 
			The file contents are placed on this <code>jcr:content</code> node in the <code>jcr:data</code> property. For example, if you specify
      <code>/a/b/caution.png</code>, the following structure will be created in the repository:</para>
		<programlisting><![CDATA[
/a   (nt:folder)
  /b   (nt:folder)
    /caution.png   (nt:file)
                /jcr:content    (nt:resource)
                            @jcr:data = {contents of the file}
                            @jcr:mimeType = {mime type of the file}
                            @jcr:lastModified = {now}
]]></programlisting>
    <para>Other kinds of files are treated in a similar way.</para>
		<para>When the client uploads the file using the JCR API, ModeShape gets notified of the changes, consults the sequencers to see whether
    	any of them are interested in the new or updated content, and if so runs those sequencers. The image sequencer processes image
    	files for metadata, and any metadata found is stored under the <code>/images</code> branch of the repository. The MP3 sequencer 
			processes MP3 audio files for metadata, and any metadata found is stored under the <code>/mp3s</code>
    	branch of the repository.  And metadata about Java classes are stored under the <code>/java</code> area of the repository.
      All of this happens asynchronously, so any ModeShape activity doesn't impede or slow down the client activities.
    </para>
		<para>So, after the file is uploaded, you can search the repository for the image metadata using the "s" menu option:
    	<figure id="example-sequencer-search">
				<title>Searching for media using the example client</title>
				<graphic align="center" scale="60" fileref="example-sequencer-search.png"/>
			</figure>
    	Here are the search results after the <code>sample1.mp3</code> audio file has been uploaded (to the <code>/a/b/sample1.mp3</code> location):
    	<figure id="example-sequencer-search-with-mp3">
				<title>Searching for media using the example client</title>
				<graphic align="center" scale="60" fileref="example-sequencer-search-with-mp3.png"/>
			</figure>
    	You can also display the sequencing statistics using the "d" menu option:
    	<figure id="example-sequencer-statistics">
				<title>Sequencing statistics using the example client</title>
				<graphic align="center" scale="60" fileref="example-sequencer-statistics.png"/>
			</figure>
    	These stats show how many nodes were sequenced, and how many nodes were skipped because they didn't apply to the sequencer's
    	criteria.
  	</para>
		<note>
			<para>There will probably be more nodes skipped than sequenced, since there are more <code>nt:folder</code> and <code>nt:resource</code>
      	nodes than there are <code>nt:file</code> nodes with acceptable names.</para>
		</note>
		<para>You can repeat this process with other files. Any file that isn't an image or MP3 files (as recognized by the sequencing configurations
    	that we'll describe later) will not be sequenced.</para>
	</sect1>
	<sect1 id="example_application_review">
		<title>Reviewing the example application</title>
		<para>Recall that the example application consists of a client application that sets up an in-memory JCR repository and
      that allows a user to upload files into that repository. The client also sets up the ModeShape services with an image sequencer so
      that if any of the uploaded files are PNG, JPEG, GIF, BMP or other images, ModeShape will automatically extract the image's
      metadata (e.g., image format, physical size, pixel density, etc.) and store that in the repository.  Or, if the client uploads
      MP3 audio files, the title, author, album, year, and comment are extracted from the audio file and stored in the repository.</para>
		<para>
      The example is comprised of 5 classes and 1 interface, located in the <code>src/main/java</code> directory:</para>
    <programlisting><![CDATA[
  org/modeshape/example/sequencers/ConsoleInput.java
                                  /ContentInfo.java
                                  /JavaInfo.java
                                  /MediaInfo.java
                                  /SequencingClient.java
                                  /UserInterface.java
  ]]></programlisting>
		<para><code>SequencingClient</code> is the class that contains the main application.  <code>ContentInfo</code> is a simple class
		  that encapsulate metadata generated by the sequencers and accessed by this example application, and there are two subclasses: 
		  <code>MediaInfo</code> encapsulates metadata about media (image and MP3) files, while <code>JavaInfo</code> is a subclass
		  encapsulating information about a Java class.  The client accesses the content from the repository and represents the
		  information using instances of <code>ContentInfo</code> (and its subclasses) and then passes them to the <code>UserInterface</code>.
      <code>UserInterface</code> is an interface with methods that will be called at runtime to 
			request data from the user.  <code>ConsoleInput</code> is an implementation of this that creates a text user interface, 
			allowing the user to operate the client from the command-line.  We can easily create a graphical implementation of
      <code>UserInterface</code> at a later date. We can also create a mock implementation for testing purposes that simulates 
			a user entering data. This allows us to check the behavior of the client automatically using conventional JUnit test cases, 
			as demonstrated by the code in the <code>src/test/java</code> directory:</para>
    <programlisting><![CDATA[
  org/modeshape/example/sequencers/SequencingClientTest.java
                                  /MockUserInterface.java
  ]]></programlisting>
		<para>If we look at the <code>SequencingClient</code> code, there are a handful of methods that encapsulate the various activities.</para>
		<note>
			<para>Some of the code samples included in this book have had some of the error handling and comments removed so that
				the code is more readable and concise.</para>
		</note>
		<para>The <code>main(String[] argv)</code> method is of course the method that is executed when the application is run.  This code
		  creates the ModeShape configuration using the programmatic style.
		</para>
		<programlisting language="JAVA" role="JAVA"><![CDATA[// Create the configuration. 
String repositoryId = "content";
String workspaceName = "default";
JcrConfiguration config = new JcrConfiguration();
// Set up the in-memory source where we'll upload the content and where the sequenced output will 
// be stored ...
config.repositorySource("store")
      .usingClass(InMemoryRepositorySource.class)
      .setDescription("The repository for our content")
      .setProperty("defaultWorkspaceName", workspaceName);
// Set up the JCR repository to use the source ...
config.repository(repositoryId)
      .addNodeTypes("sequencing.cnd")
      .setSource("store");
// Set up the image sequencer ...
config.sequencer("Image Sequencer")
      .usingClass("org.modeshape.sequencer.image.ImageMetadataSequencer")
      .loadedFromClasspath()
      .setDescription("Sequences image files to extract the characteristics of the image")
      .sequencingFrom(
          "//(*.(jpg|jpeg|gif|bmp|pcx|png|iff|ras|pbm|pgm|ppm|psd)[*])/jcr:content[@jcr:data]")
      .andOutputtingTo("/images/$1");
// Set up the MP3 sequencer ...
config.sequencer("MP3 Sequencer")
      .usingClass("org.modeshape.sequencer.mp3.Mp3MetadataSequencer")
      .loadedFromClasspath()
      .setDescription("Sequences mp3 files to extract the id3 tags of the audio file")
      .sequencingFrom("//(*.mp3[*])/jcr:content[@jcr:data]")
      .andOutputtingTo("/mp3s/$1");
// Set up the Java class file sequencer ...
config.sequencer("Java Class Sequencer")
      .usingClass(ClassFileSequencer.class)
      .setDescription("Sequences Java class files to extract the structure of the classes")
      .sequencingFrom("//*.class[*]/jcr:content[@jcr:data]")
      .andOutputtingTo("/classes");
// Set up the Java source file sequencer ...
config.sequencer("Java Sequencer")
      .usingClass("org.modeshape.sequencer.java.JavaMetadataSequencer")
      .loadedFromClasspath()
      .setDescription("Sequences Java files to extract the AST structure of the Java source code")
      .sequencingFrom("//(*.java[*])/jcr:content[@jcr:data]")
      .andOutputtingTo("/java/$1");
// Set up the CSV file sequencer ...
config.sequencer("CSV Sequencer")
      .usingClass("org.modeshape.sequencer.text.DelimitedTextSequencer")
      .loadedFromClasspath()
      .setDescription("Sequences CSV files to extract the contents")
      .sequencingFrom("//(*.csv[*])/jcr:content[@jcr:data]")
      .andOutputtingTo("/csv/$1");
// Set up the fixed width file sequencer ...
config.sequencer("Fixed Width Sequencer")
      .usingClass("org.modeshape.sequencer.text.FixedWidthTextSequencer")
      .loadedFromClasspath()
      .setDescription("Sequences fixed width files to extract the contents")
      .setProperty("commentMarker", "#")
      .setProperty("columnStartPositions", new int[] { 10, 20, 30, 40})
      .sequencingFrom("//(*.txt[*])/jcr:content[@jcr:data]")
      .andOutputtingTo("/txt/$1");

// Now start the client and tell it which repository and workspace to use ...
SequencingClient client = new SequencingClient(config, repositoryId, workspaceName);
client.setUserInterface(new ConsoleInput(client));
 ]]></programlisting>
		<para>
			The first block of code configures the <code>JcrConfiguration</code> and sets up the "store" source, the "content" repository,
			and three sequencers.  Again, this is done via the programmatic style. An alternative would be to load the entire configuration
			from a configuration file or from an existing configuration repository.  (The repository example shown in the 
			<link linkend="repository_example">next chapter</link> shows how to load the configuration from a file.)
		</para>
		<para>
			The second block simply instantiates the <code>SequencingClient</code> class, passing the configuration and the name of the repository
			and workspace, and finally sets the user interface (which then executes its behavior, which we'll see below).
		</para>
		<para>
			The <code>startRepository()</code> method builds the <code>JcrEngine</code> component from the configuration, starts the engine,
			and obtains the JCR <code>javax.jcr.Repository</code> instance that the client will use.  Note that the client has not yet
			obtained a <code>javax.jcr.Session</code> instance, since this will be done each time the client needs to access content from
			the repository.  (This is actually a common practice according to the JCR specification, since Sessions are intended to be
			very lightweight.)
    </para>
		<programlisting language="JAVA" role="JAVA"><![CDATA[
public void startRepository() throws Exception {
		if (this.repository == null) {
		    try {
		        // Start the ModeShape engine ...
		        this.engine = this.configuration.build();
		        this.engine.start();

		        // Now get the JCR repository instance ...
		        this.repository = this.engine.getRepository(repositoryName);
		    } catch (Exception e) {
		        this.repository = null;
		        throw e;
		    }
		}
}
  ]]></programlisting>
		<para>The <code>shutdownRepository()</code> method requests the <code>JcrEngine</code> instance shuts down and, since that may take
	    a few moments (if there are any ongoing operations or enqueued activities) awaits for it to complete the shutdown.
   </para>
    <programlisting language="JAVA" role="JAVA"><![CDATA[
public void shutdownRepository() throws Exception {
		if (this.repository != null) {
		    try {
		        this.engine.shutdown();
		        this.engine.awaitTermination(4, TimeUnit.SECONDS);
		    } finally {
		        this.repository = null;
		    }
		}
}
  ]]></programlisting>
		<para>None of the other methods really do anything with ModeShape <emphasis>per se</emphasis>. Instead, they merely work with the repository
      using the JCR API.</para>
		<para>If we look at the <code>ConsoleInput</code> constructor, it starts the repository and a thread 
			for the user interface. At this point, the constructor returns, but the main application continues under the user interface thread.  
			When the user requests to quit, the user interface thread also shuts down the JCR repository.</para>
    <programlisting language="JAVA" role="JAVA"><![CDATA[
public ConsoleInput( SequencerClient client ) {
  try {
      client.startRepository();

      System.out.println(getMenu());
      Thread eventThread = new Thread(new Runnable() {
          private boolean quit = false;
          public void run() {
              try {
                  while (!quit) {
                      // Display the prompt and process the requested operation ...
                  }
              } finally {
                  try {
                      // Terminate ...
                      client.shutdownRepository();
                  } catch (Exception err) {
                      System.out.println("Error shutting down sequencing service and repository: " 
                                         + err.getLocalizedMessage());
                      err.printStackTrace(System.err);
                  }
              }
          }
      });
      eventThread.start();
  } catch (Exception err) {
      System.out.println("Error: " + err.getLocalizedMessage());
      err.printStackTrace(System.err);
  }
}
  ]]></programlisting>
		<para>
			There is one more aspect of this example that is worth discussing.  While the repository example in the
			<link linkend="repository_example">next chapter</link> does show how to use JAAS, this example intentionally shows how
			you might integrate a different security system into ModeShape.  In the <code>createSession()</code> method,
			the <code>RepositoryClient</code> creates a <code>SecurityContextCredentials</code> wrapper around a custom
			<code>SecurityContext</code> implementation, then passes that credentials into the <code>login(Credentials,String)</code>
			method:
		</para>
    <programlisting language="JAVA" role="JAVA"><![CDATA[
protected Session createSession() throws RepositoryException {
    SecurityContext securityContext = new MyCustomSecurityContext();
    SecurityContextCredentials credentials = new SecurityContextCredentials(securityContext);
    return this.repository.login(credentials, workspaceName);
}
  ]]></programlisting>
		<para>
			where the custom <code>SecurityContext</code> implementation is as follows:
		</para>
   <programlisting language="JAVA" role="JAVA"><![CDATA[
protected class MyCustomSecurityContext implements SecurityContext {
    /**
     * @see org.modeshape.graph.SecurityContext#getUserName()
     */
    public String getUserName() {
        return "Fred";
    }

    /**
     * @see org.modeshape.graph.SecurityContext#hasRole(java.lang.String)
     */
    public boolean hasRole( String roleName ) {
        return true;
    }

    /**
     * @see org.modeshape.graph.SecurityContext#logout()
     */
    public void logout() {
        // do something
    }
}
 ]]></programlisting>
		<para>
			Obviously you would want to implement this correctly.  If you're using ModeShape in a web application, your <code>SecurityContext</code>
			implementation would likely delegate to the <code>HttpServletRequest</code>.
		</para>
		<para>
			But if you're using JAAS, then you could just pass in a <code>javax.jcr.SimpleCredentials</code> with the username and password,
			as long as your <code>JcrConfiguration</code>'s repository definitions are set up to use the correct JAAS login context name
			(see the repository example in the <link linkend="repository_example">next chapter</link>).  Or, you could use the approach listed
			above and supply an instance of the <code>JaasSecurityContext</code> to the <code>SecurityContextCredentials</code>.
		</para>
		<para>At this point, we've reviewed all of the interesting code in the example application related to ModeShape.  However, feel free
    to play with the application, trying different things.</para>
	</sect1>
	<sect1>
		<title>What's next</title>
		<para>
			This chapter walked through running the sequencer example and looked at the example code.  With the sequencer client, you could upload files into a
	    	JCR repository, while ModeShape automatically sequenced the source files you uploaded, extracted the metadata from the
	    	files, and stored that metadata inside the repository.
		</para>
		<para>
			In the <link linkend="repository_example">next chapter </link> we'll do the same for the repository example.
		</para>
	</sect1>
</chapter>
