DevOps: Workspace Structuring with Maven

"As any designer will tell you, it is the first steps in a design process which count for most. The first few strokes, which create the form, carry within them the destiny of the rest." - Christopher Alexander

A workspace is a collection of Java projects
I believe in naming my projects by feature, and my packages by layer

Somewhat contrary to this advice.

Workspace Design Overview

Java Projects

The core projects will be named according to their Maven groupId

org.swtk.core.feedback
org.swtk.core.freebase
org.swtk.core.lucene
org.swtk.core.<name>

Each of these projects will produce a JAR file, controllable by Maven.

Sample POM

 <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"  
      xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">  

      <modelVersion>4.0.0</modelVersion>  
      <groupId>org.swtk.core.lucene</groupId>  
      <artifactId>org.swtk.core.lucene</artifactId>  
      <version>0.0.1-SNAPSHOT</version>  

      <build>  

           <sourceDirectory>src</sourceDirectory>  
           <testSourceDirectory>test</testSourceDirectory>  

           <plugins>  

                <plugin>  
                     <artifactId>maven-compiler-plugin</artifactId>  
                     <version>3.1</version>  
                     <configuration>  
                          <source>1.7</source>  
                          <target>1.7</target>  
                     </configuration>  
                </plugin>  

           </plugins>  
      </build>  

      <properties>  

           <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>  
           <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>  

           <!-- Internal Versions -->  
           <swtk-commons.version>1.0.0</swtk-commons.version>  
           <swtk-core.ngrams-db.version>0.0.1-SNAPSHOT</swtk-core.ngrams-db.version>  

           <!-- External Versions (COMMON) -->  
           <junit.version>4.12-beta-1</junit.version>  
           <log4j.version>1.2.17</log4j.version>  
           <commons-io.version>2.4</commons-io.version>  
           <commons-lang.version>3.3.2</commons-lang.version>  
           <gson.version>2.3</gson.version>  

           <!-- External Dependencies (LUCENE) -->  
           <lucene-core.version>4.10.1</lucene-core.version>  
           <lucene-analyzers.version>4.10.1</lucene-analyzers.version>  
           <lucene-queryparser.version>4.10.1</lucene-queryparser.version>  

      </properties>  

      <dependencies>  

           <!-- Internal Dependencies -->  
           <dependency>  
                <groupId>swtk-commons</groupId>  
                <artifactId>swtk-commons</artifactId>  
                <version>${swtk-commons.version}</version>  
           </dependency>  
           <dependency>  
                <groupId>org.swtk.ngrams.db</groupId>  
                <artifactId>org.swtk.ngrams.db</artifactId>  
                <version>${swtk-core.ngrams-db.version}</version>  
           </dependency>  

           <!-- External Dependencies (COMMON) -->  
           <dependency>  
                <groupId>junit</groupId>  
                <artifactId>junit</artifactId>  
                <version>${junit.version}</version>  
           </dependency>  
           <dependency>  
                <groupId>org.apache.commons</groupId>  
                <artifactId>commons-lang3</artifactId>  
                <version>${commons-lang.version}</version>  
           </dependency>  
           <dependency>  
                <groupId>log4j</groupId>  
                <artifactId>log4j</artifactId>  
                <version>${log4j.version}</version>  
           </dependency>  
           <dependency>  
                <groupId>com.google.code.gson</groupId>  
                <artifactId>gson</artifactId>  
                <version>${gson.version}</version>  
           </dependency>  

           <!-- External Dependencies (LUCENE) -->  
           <dependency>  
                <groupId>org.apache.lucene</groupId>  
                <artifactId>lucene-core</artifactId>  
                <version>${lucene-core.version}</version>  
           </dependency>  
           <dependency>  
                <groupId>org.apache.lucene</groupId>  
                <artifactId>lucene-analyzers-common</artifactId>  
                <version>${lucene-analyzers.version}</version>  
           </dependency>  
           <dependency>  
                <groupId>org.apache.lucene</groupId>  
                <artifactId>lucene-queryparser</artifactId>  
                <version>${lucene-queryparser.version}</version>  
           </dependency>  
      </dependencies>  
 </project>

Build Project

There will be a non-Java project called "Build"
This will be packaged as a "pom" only by Maven
The purpose of this project is to assemble all the core Java projects and dependencies into a single POM

Sample POM

 <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">  
      <modelVersion>4.0.0</modelVersion>  
   
      <groupId>swtk-core</groupId>  
      <artifactId>swtk-core</artifactId>  
      <version>1.0.0</version>  
      <packaging>pom</packaging>  
   
      <properties>  
   
           <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>  
           <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>  
   
           <!-- Internal Versions -->  
           <swtk-commons.version>1.0.0</swtk-commons.version>  
           <swtk-eng.version>1.0.0</swtk-eng.version>  
           <swtk-core.freebase.version>0.0.1-SNAPSHOT</swtk-core.freebase.version>  
           <swtk-core.feedback.version>0.0.1-SNAPSHOT</swtk-core.feedback.version>  
           <swtk-core.lucene.version>0.0.1-SNAPSHOT</swtk-core.lucene.version>  
           <swtk-core.ngram.version>0.0.1-SNAPSHOT</swtk-core.ngram.version>  
           <swtk-core.patterns.version>0.0.1-SNAPSHOT</swtk-core.patterns.version>  
           <swtk-core.preprocess.version>0.0.1-SNAPSHOT</swtk-core.preprocess.version>  
           <swtk-core.question.version>0.0.1-SNAPSHOT</swtk-core.question.version>  
           <swtk-core.question-training.version>0.0.1-SNAPSHOT</swtk-core.question-training.version>  
           <swtk-core.readability-training.version>0.0.1-SNAPSHOT</swtk-core.readability-training.version>  
   
           <!-- External Versions (COMMON) -->  
           <jaws.version>1.2</jaws.version>  
           <junit.version>4.12-beta-1</junit.version>  
           <log4j.version>1.2.17</log4j.version>  
           <commons-collection.version>3.2.1</commons-collection.version>  
           <commons-lang.version>3.3.2</commons-lang.version>  
           <gson.version>2.3</gson.version>  
   
           <!-- External Dependencies (HIBERNATE) -->  
           <hbn.version>3.5.1-Final</hbn.version>  
           <hbn-ejb.version>3.3.2.Beta1</hbn-ejb.version>  
           <hbn-annotation.version>3.5.5-Final</hbn-annotation.version>  
           <hbn-em.version>3.5.5-Final</hbn-em.version>  
   
           <!-- External Dependencies (HIBERNATE-INDIRECT) -->  
           <dom4j.version>1.6.1</dom4j.version>  
           <javassist.version>3.9.0.GA</javassist.version>  
           <javax-transaction.version>1.1</javax-transaction.version>  
           <antlr.version>2.7.6</antlr.version>  
   
           <!-- External Dependencies (SLF4J) -->  
           <slf4j-api.version>1.5.8</slf4j-api.version>  
           <slf4j-simple.version>1.5.8</slf4j-simple.version>  
           <org-slf4j.version>1.6.4</org-slf4j.version>  
   
      </properties>  
   
      <dependencies>  
   
           <!-- Internal Dependencies -->  
           <dependency>  
                <groupId>swtk-commons</groupId>  
                <artifactId>swtk-commons</artifactId>  
                <version>${swtk-commons.version}</version>  
           </dependency>  
           <dependency>  
                <groupId>swtk-eng</groupId>  
                <artifactId>swtk-eng</artifactId>  
                <version>${swtk-eng.version}</version>  
           </dependency>  
           <dependency>  
                <groupId>org.swtk.core.freebase</groupId>  
                <artifactId>org.swtk.core.freebase</artifactId>  
                <version>${swtk-core.freebase.version}</version>  
           </dependency>  
           <dependency>  
                <groupId>org.swtk.core.feedback</groupId>  
                <artifactId>org.swtk.core.feedback</artifactId>  
                <version>${swtk-core.feedback.version}</version>  
           </dependency>  
           <dependency>  
                <groupId>org.swtk.core.lucene</groupId>  
                <artifactId>org.swtk.core.lucene</artifactId>  
                <version>${swtk-core.lucene.version}</version>  
           </dependency>  
           <dependency>  
                <groupId>org.swtk.core.ngrams.db</groupId>  
                <artifactId>org.swtk.core.ngrams.db</artifactId>  
                <version>${swtk-core.ngram.version}</version>  
           </dependency>  
           <dependency>  
                <groupId>org.swtk.core.patterns</groupId>  
                <artifactId>org.swtk.core.patterns</artifactId>  
                <version>${swtk-core.patterns.version}</version>  
           </dependency>  
           <dependency>  
                <groupId>org.swtk.core.preprocess</groupId>  
                <artifactId>org.swtk.core.preprocess</artifactId>  
                <version>${swtk-core.preprocess.version}</version>  
           </dependency>  
           <dependency>  
                <groupId>org.swtk.core.question</groupId>  
                <artifactId>org.swtk.core.question</artifactId>  
                <version>${swtk-core.question.version}</version>  
           </dependency>  
           <dependency>  
                <groupId>org.swtk.core.question.training</groupId>  
                <artifactId>org.swtk.core.question.training</artifactId>  
                <version>${swtk-core.question-training.version}</version>  
           </dependency>  
           <dependency>  
                <groupId>org.swtk.core.readibility.training</groupId>  
                <artifactId>org.swtk.core.readibility.training</artifactId>  
                <version>${swtk-core.readability-training.version}</version>  
           </dependency>  
   
           <!-- External Dependencies (COMMON) -->  
           <dependency>  
                <groupId>junit</groupId>  
                <artifactId>junit</artifactId>  
                <version>${junit.version}</version>  
           </dependency>  
           <dependency>  
                <groupId>log4j</groupId>  
                <artifactId>log4j</artifactId>  
                <version>${log4j.version}</version>  
           </dependency>  
           <dependency>  
                <groupId>org.apache.commons</groupId>  
                <artifactId>commons-lang3</artifactId>  
                <version>${commons-lang.version}</version>  
           </dependency>  
           <dependency>  
                <groupId>commons-collections</groupId>  
                <artifactId>commons-collections</artifactId>  
                <version>${commons-collection.version}</version>  
           </dependency>  
           <dependency>  
                <groupId>com.google.code.gson</groupId>  
                <artifactId>gson</artifactId>  
                <version>${gson.version}</version>  
           </dependency>  
   
           <!-- External Dependencies (HIBERNATE) -->  
           <dependency>  
                <groupId>org.hibernate</groupId>  
                <artifactId>hibernate</artifactId>  
                <version>${hbn.version}</version>  
                <type>pom</type>  
           </dependency>  
           <dependency>  
                <groupId>org.hibernate</groupId>  
                <artifactId>ejb3-persistence</artifactId>  
                <version>${hbn-ejb.version}</version>  
           </dependency>  
           <dependency>  
                <groupId>org.hibernate</groupId>  
                <artifactId>hibernate-annotations</artifactId>  
                <version>${hbn-annotation.version}</version>  
           </dependency>  
           <dependency>  
                <groupId>org.hibernate</groupId>  
                <artifactId>hibernate-entitymanager</artifactId>  
                <version>${hbn-em.version}</version>  
           </dependency>  
   
           <!-- External Dependencies (HIBERNATE-INDIRECT) -->  
           <dependency>  
                <groupId>dom4j</groupId>  
                <artifactId>dom4j</artifactId>  
                <version>${dom4j.version}</version>  
           </dependency>  
           <dependency>  
                <groupId>javassist</groupId>  
                <artifactId>javassist</artifactId>  
                <version>${javassist.version}</version>  
           </dependency>  
           <dependency>  
                <groupId>javax.transaction</groupId>  
                <artifactId>jta</artifactId>  
                <version>${javax-transaction.version}</version>  
           </dependency>  
           <dependency>  
                <groupId>antlr</groupId>  
                <artifactId>antlr</artifactId>  
                <version>${antlr.version}</version>  
           </dependency>  
   
           <!-- External Dependencies (SLF4J) -->  
           <dependency>  
                <groupId>org.slf4j</groupId>  
                <artifactId>slf4j-api</artifactId>  
                <version>${slf4j-api.version}</version>  
                <type>pom</type>  
                <scope>compile</scope>  
           </dependency>  
           <dependency>  
                <groupId>org.slf4j</groupId>  
                <artifactId>slf4j-simple</artifactId>  
                <version>${slf4j-simple.version}</version>  
           </dependency>  
           <dependency>  
                <groupId>org.slf4j</groupId>  
                <artifactId>slf4j-log4j12</artifactId>  
                <version>${org-slf4j.version}</version>  
           </dependency>  
   
      </dependencies>  
 </project>

Consumer Project

Most workspaces will have at least one consumer project.

The consumer uses the build and adds Servlets and other Web resources to expose this functionality via Web protocols. This may be for further testing by other developers, or for ultimate consumption by end-users. The consumer project will not be target-specific. It will contain resources that conform to the JEE Specification and produce a JAR file.

Sample POM

 <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">  
      <modelVersion>4.0.0</modelVersion>  
   
      <groupId>swtk.core.web</groupId>  
      <artifactId>swtk.core.web</artifactId>  
      <version>1.0.0</version>  
      <packaging>jar</packaging>  
   
      <build>  

           <sourceDirectory>src</sourceDirectory>  
           <testSourceDirectory>test</testSourceDirectory>  

           <plugins>  
                <plugin>  
                     <artifactId>maven-compiler-plugin</artifactId>  
                     <version>3.1</version>  
                     <configuration>  
                          <source>1.7</source>  
                          <target>1.7</target>  
                     </configuration>  
                </plugin>  
           </plugins>  
      </build>  
   
      <properties>  
   
           <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>  
           <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>  
   
           <!-- Internal Versions -->  
           <swtk-commons.version>1.0.0</swtk-commons.version>  
           <swtk-eng.version>1.0.0</swtk-eng.version>  
           <swtk-core.version>1.0.0</swtk-core.version>  
   
           <!-- External Versions (WEB) -->  
           <javax.servlet.version>3.1.0</javax.servlet.version>  
           <httpunit.version>1.7</httpunit.version>  
   
      </properties>  
   
      <dependencies>  
        
           <!-- Internal Dependencies (SWTK) -->  
           <dependency>  
                <groupId>swtk-core</groupId>  
                <artifactId>swtk-core</artifactId>  
                <version>${swtk-core.version}</version>  
           </dependency>  
   
           <!-- External Dependencies (WEB) -->  
           <dependency>  
                <groupId>javax.servlet</groupId>  
                <artifactId>javax.servlet-api</artifactId>  
                <version>${javax.servlet.version}</version>  
           </dependency>  
           <dependency>  
                <groupId>httpunit</groupId>  
                <artifactId>httpunit</artifactId>  
                <version>${httpunit.version}</version>  
           </dependency>  
   
      </dependencies>  
   
      <reporting>  
   
           <plugins>  
                <plugin>  
                     <groupId>org.apache.maven.plugins</groupId>  
                     <artifactId>maven-dependency-plugin</artifactId>  
                     <version>2.9</version>  
                </plugin>  
           </plugins>  
   
      </reporting>  
   
 </project>

Notice the extreme simplicity of this POM file. Virtually all the dependencies are handled by the build project. This POM introduces just enough dependencies to enable generic web support. Container specific dependencies are introduced in the container specific projects (below).

Consumer Project (Container Specific)

For each container being deployed to, a container-specific consumer project will exist. This may a simple as creating "consumer-deploy-tomcat" as a POM file that generates a WAR artifact by depending on the JAR from the generic consumer project.

I believe in the use of container-specific consumer projects via Maven, because

Allows for container-specific settings in each POM file
We might use a Tomcat container for an environment early in the release and deployment management lifecycle (e.g. INT or QA) and a more enterprise container in a PROD environment, like WebSphere Application Server.

In this case, it makes sense to maintain two specific consumer projects

Using Maven, this is extremely lightweight

We are only maintaining one XML file (a POM) for each container

Automation is easier

Assuming the POMs are kept up to date, deployment to a specific environment becomes as simple as typing a plugin goal on the command line.

Sample POM

 <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"  
      xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">  
      <modelVersion>4.0.0</modelVersion>  
   
      <groupId>swtk.core.web.tomcat</groupId>  
      <artifactId>swtk.core.web.tomcat</artifactId>  
      <version>1.0.0</version>  
      <packaging>war</packaging>  
   
      <build>  
           <sourceDirectory>src</sourceDirectory>  
           <plugins>  
             
                <plugin>  
                     <artifactId>maven-compiler-plugin</artifactId>  
                     <version>3.1</version>  
                     <configuration>  
                          <source>1.7</source>  
                          <target>1.7</target>  
                     </configuration>  
                </plugin>  
   
                <plugin>  
                     <artifactId>maven-war-plugin</artifactId>  
                     <version>2.4</version>  
                     <configuration>  
                          <warSourceDirectory>WebContent</warSourceDirectory>  
                          <failOnMissingWebXml>false</failOnMissingWebXml>  
                     </configuration>  
                </plugin>  
   
                <plugin>  
                     <groupId>org.apache.tomcat.maven</groupId>  
                     <artifactId>tomcat7-maven-plugin</artifactId>  
                     <version>2.2</version>  
                     <configuration>  
                          <url>http://localhost:8080/manager/text</url>  
                          <server>TomcatServer</server>  
                          <path>/test</path>  
                          <username>craig</username>  
                          <password>password</password>  
                     </configuration>  
                </plugin>  
                  
                <plugin>  
                     <groupId>org.apache.maven.plugins</groupId>  
                     <artifactId>maven-dependency-plugin</artifactId>  
                     <version>2.2</version>  
                </plugin>  
           </plugins>  
      </build>  
   
      <properties>  
   
           <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>  
           <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>  
   
           <!-- Internal Versions -->  
           <swtk-core-web.version>1.0.0</swtk-core-web.version>  
   
      </properties>  
   
      <dependencies>  
           <!-- Internal Dependencies (SWTK) -->  
           <dependency>  
                <groupId>swtk.core.web</groupId>  
                <artifactId>swtk.core.web</artifactId>  
                <version>${swtk-core-web.version}</version>  
           </dependency>  
      </dependencies>  
   
 </project>

Notice that the dependency section of this POM is extremely simple. The generic POM from the prior section is pulled in. The customized aspects of this POM include the tomcat-specific plugin instructions for automated deployments.

If we were to build this workspace for another container (such as liberty-core), we would create another project with another POM for this purpose.

Other Projects

I recommend creating the following non-Java projects:

docs
dependencies

These projects will not be maintained using Maven and will not generate any artifacts (JARs, WARs, etc) . The dependencies project is used for any third party dependencies that are not Maven-ized and must be installed manually.

Physical Artifacts

Physical artifacts generated from this workspace are represented here:

Dependency Visualization

This article talks about how to visualize project dependencies.

The output from technique produces a diagram that looks like this:

Notice how the container-specific project for Tomcat exists at the base of the visualization. It is a lightweight consumer of the generic consumer project, with additional Maven plugin dependencies introduced for the purpose of deployment automation (and other container specific properties that may be introduced down the road).

Centroids tend to be core projects within the workspace design.

The absence of clusters would lead me to believe the deployed WAR file contained poor separation of concerns among the various projects.

DevOps

Tuesday, December 2, 2014

Workspace Structuring with Maven

Workspace Design Overview

Java Projects

Sample POM

Build Project

Sample POM

Consumer Project

Sample POM

Consumer Project (Container Specific)

Sample POM

Other Projects

Physical Artifacts

Dependency Visualization

1 comment: