Bioinformatics Graz

Maspectras installation procedure

Applicable for Release 0.9

1. Requirements

Hardware requirements:

Server with at least 1 GBytes of main memory
(optional a LINUX cluster)

Software requirements:

Oracle 9i or PostgreSQL 8.0.x or MySQL 4.1.xx for Maspectras and/or the Usermanagement
Java JDK 1.5.x
be sure that JAVA_HOME points to your JDK installation

2. Installation Instructions

Maspectras consists of three components which can run independently on different machines.

AAS: provides the central usermanagement and its access controls

JClusterService: provides a webservice used for external sequence database management,
protein grouping and mass quantification. It is recommended, that those disk- and computing-
intensive services should run on a separate machine or even better on a linux cluster.

Maspectras: is the core application which provides all functionality except the ones mentioned
above

In order to perform a more painless installation and to avoid problems concerning access rights
please execute the following steps as one single user. If you prefer to install Maspectras and its
components on a Linux system create a user like it is described in 9.1. and switch to this user (su - maspectras)
for all the following steps.

Security: The communication between the 3 components of maspectras is based on http connections
which should be secured by SSL layer security (HTTPS). In order to reduce complexity of this
installation instruction HTTPS adaptations were skipped. In our productive environment all
communications between maspectras, usermanagement and jclusterservices are performed over https.

1. Unpacking Maspectras

1.1. Unpack maspectras.tar.gz

2.a MySQL inizialisation (must be InnoDB and support for InnoDB must be active)

2.a.1. Create a new schema called usermanagement for the AAS
Create a new schema called maspectras for maspectras

2.a.2. Create 2 new users

for AAS
username: usermn
password: usermn
for maspectras
username: maspectras
password: maspectras

2.a.3. Grant the usermn user all priviliges for the usermanagement schema
Grant the maspectras user all priviliges for the maspectras schema

2.b PostgreSQL installation

2.b.1. Create 2 new users

for maspectras
username: maspectras
password: maspectras
for AAS
username: usermanagement
password: usermanagement

2.b.2 Create a new schema for called usermanagement (the schema public must already exist, maspectras is mapping by default to public)

2.b.3. Grant the usermanagement user all priviliges for the usermanagement schema
Grant the maspectras user all priviliges for the public schema

2.c Oracle installation

2.c.1. Create 2 new tablespaces (MASPECTRAS and USERMANAGEMENT)

2.c.2. Create 2 new users like 'Scott'

for AAS
username: usermanagement
password: usermanagement
for maspectras
username: maspectras
password: maspectras

3. Configuring the JBoss for the right database

3.a MySQL

for AAS
             Copy mysql-connector-java-3.1.10-bin.jar from database/config/usermanagement/mysql/ to
             $JBOSS_HOME/server/usermanagement/lib/
             Copy mysql-ds.xml from database/config/usermanagement/mysql/ to
             $JBOSS_HOME/server/usermanagement/deploy/
             The mysql-ds.xml is the config file for your database, it is configured for localhost.
             If you have other settings change them in mysql-ds.xml.
for maspectras
             Copy mysql-connector-java-3.1.10-bin.jar from database/config/maspectras/mysql/ to
             $JBOSS_HOME/server/maspectras/lib/
             Copy mysql-ds.xml from database/config/maspectras/mysql/ to
             $JBOSS_HOME/server/maspectras/deploy/
             The mysql-ds.xml is the config file for your database, it is configured for localhost.
             If you have other settings change them in mysql-ds.xml.

3.b PostgreSQL

for AAS
             Copy postgresql-8.0-312.jdbc3.jar from database/config/usermanagement/postgresql/ to
             $JBOSS_HOME/server/usermanagement/lib/
             Copy postgres-ds.xml from database/config/usermanagement/postgresql/ to
             $JBOSS_HOME/server/usermanagement/deploy/
             The postgres-ds.xml is the config file for your database, it is configured for localhost.
             If you have other settings change them in postgres-ds.xml.
for maspectras
             Copy postgresql-8.0-312.jdbc3.jar from database/config/maspectras/postgresql/ to
             $JBOSS_HOME/server/maspectras/lib/
             Copy postgres-ds.xml from database/config/maspectras/postgresql/ to
             $JBOSS_HOME/server/maspectras/deploy/
             The postgres-ds.xml is the config file for your database, it is configured for localhost.
             If you have other settings change them in postgres-ds.xml.

3.c Oracle

for AAS
             Copy ojdbc14.jar from database/config/usermanagement/oracle/ to
             $JBOSS_HOME/server/usermanagement/lib/
             Copy oracle-ds.xml from database/config/usermanagement/oracle/ to
             $JBOSS_HOME/server/usermanagement/deploy/
             In the oracle-ds.xml enter the sid of your server instead of "yoursid"
             The oracle-ds.xml is the config file for your database, it is configured for localhost.
             If you have other settings change them in oracle-ds.xml
for maspectras
             Copy ojdbc14.jar from database/config/maspectras/oracle/ to
             $JBOSS_HOME/server/maspectras/lib/
             Copy postgres-ds.xml from database/config/maspectras/oracle/ to
             $JBOSS_HOME/server/maspectras/deploy/
             In the oracle-ds.xml enter the sid of your server instead of "yoursid"
             The oracle-ds.xml is the config file for your database, it is configured for localhost.
             If you have other settings change them in oracle-ds.xml

4. Starting the AAS

4.1. Go to $JBOSS_HOME/bin

4.2. Start supplied Authentication and Authorization System (AAS) with
     run-usermanagement.bat (for Windows) or
     run-usermanagement.sh (for Linux or Solaris)
  and wait until the JBoss started

4.3 Use from /database/initialize/usermanagement
         initializeAAS_mysql.sql (for MySQL) or
         initializeAAS_postgresql.sql (for PostgreSQL) or
         initializeAAS_oracle.sql (for Oracle)
    to insert required data

5. Create data root

5.1 Create your DATAROOTDIRECTORY. The DATAROOTDIRECTORY is the root path where all the files which are uploaded or can be downloaded from Maspectras are stored. Pay attention! This volume should be very large since all the scanned files are stored there, too!
e.g. /home/maspectrasdata/
Make sure that the maspectras user has read and write access to $DATAROOTDIRECTORY and that the $DATAROOTDIRECTORY ends with a slash
5.2. Copy the folders config and analyses from upload/ to your created DATAROOTDIRECTORY

6. Configuring maspectras for the AAS (only necessary if AAS is not running on the same machine like maspectras, otherwise skip this point)

6.1. Open the file maspectras.properties at $JBOSS_HOME/server/maspectras/ and at $JBOSS_HOME/server/maspectras/deploy/maspectras.war/WEB-INF/

6.2. Change the localhost behind the http_url= to your AAS address

6.3. If your usermanagement is secured with https change the

        protocol=HTTP

        protocol=HTTPS

and adapt the parameters

           ##### HTTPS Protocol #####
           https_url=localhost
           https_port=17433
           keystorefile=/path/to/keystore/keystore

7. Starting maspectras

7.1 Use from /database/initialize/maspectras/
         mysql/insertFKsCreateMySQLAll.sql (for MySQL) or
         postgresql/insertFKsCreatePostgreSQLAll.sql (for PostgreSQL) or
         oracle/insertFKsCreateOracleAll.sql (for Oracle)
    to create some tables

7.2. Go to $JBOSS_HOME/bin

7.3. Start maspectras with
     run-maspectras.bat (for Windows) or
     run-maspectras.sh (for Linux or Solaris)
  and wait until the JBoss started

7.4 Use from /database/initialize/maspectras/
         mysql/sequestInitialValuesMySQL.sql (for MySQL) or
         postgresql/sequestInitialValuesPostgreSQL.sql (for PostgreSQL) or
         oracle/sequestInitialValues.sql (for Oracle)
    to insert required data

8. Change the Maspectras settings and insert the path to the newly created DATAROOTDIRECTORY

8.1. Open http://localhost:8080

8.2. Click on JMX-Console

8.3. Click on service=ServerSettings

8.4. Use the method setProperty

p1 = DATAROOTDIRECTORY
p2 = /home/maspectrasdata/

8.5. Click on invoke

8.6. To check the properties use method „listProperties“

9. Install JClusterService on a UNIX/Linux system (it's not working under windows and not tested under OSX)

9.1. Create a user under which the jclusterservice should execute the commands and create a JCLUSTERSERVICEROOTDIRECTORY e.g. /home/maspectras/maspectrasjclusterservice where the user has access right. If not already done copy and unpack the maspectras.tar.gz again under the home directory of the new user. This creates a maspectras directory with all the necessary files. From there copy all files within maspectras/maspectrasjclusterservice to this directory and make sure that the user can access the folder. e.g.

          groupadd maspectras
          useradd -d /home/maspectras -g maspectras maspectras
          su - maspectras
	  mkdir -p /home/maspectras/maspectrasjclusterservice
          tar -xzf maspectras.tar.gz
	  cp -r %extracted path%/maspectrasjclusterservice/* /home/maspectras/maspectrasjclusterservice 
	  chown -R maspectras:maspectras /home/maspectras 
	  # (just to be sure that every file is accessible for the service user execute this command as root)

If you have not already switched to maspectras user, become maspectras user (su - maspectras)
9.2. Edit jboss_4.0.2/server/jclusterservice/deploy/JClusterWS.ear/JClusterWSWeb.war/WEB-INF/server-config.wsdd and change to:

      <parameter name="attachments.Directory" 
                 value="%installationpath%/jboss-4.0.2/server/JClusterWS/./deploy/JClusterWS.ear/JClusterWS/./deploy/JClusterWS.ear/JClusterWSWeb.war/WEB-INF/attachments"/>
    e.g.
         /home/maspectras/maspectras/jboss_4.0.2/server/jclusterservice/deploy/JClusterWS.ear/JClusterWSWeb.war/WEB-INF/attachments

If you change the server ports for jclusterservice in bindings.xml, change also the JNDI URL in this file:

      <parameter name="jndiURL" value="jnp://localhost:%your_port%"/>

default is:

      <parameter name="jndiURL" value="jnp://localhost:21099"/>

9.3. Adapt the path settings to your local installation in jboss_4.0.2/server/jclusterservice/deploy/JClusterWS.ear/JClusterService.jar/usermanagement-utils.properties

    sum_users_filepath=JCLUSTERSERVICEROOTDIRECTORY/config/users.xml
    e.g.
    sum_users_filepath=/home/maspectras/maspectrasjclusterservice/config/users.xml

9.4. Adapt the path settings to your local installation in jboss_4.0.2/server/jclusterservice/deploy/JClusterWS.ear/JClusterService.jar/cluster.properties
9.4.1. The global path settings:

          clusterservicedefinition_path=JCLUSTERSERVICEROOTDIRECTORY/config/clusterservice-definition.xml
	  job_path=JCLUSTERSERVICEROOTDIRECTORY/jobs
          jobdefinitions_path=JCLUSTERSERVICEROOTDIRECTORY/job_definitions
          e.g.
          clusterservicedefinition_path=/home/maspectras/maspectrasjclusterservice/config/clusterservice-definition.xml
	  job_path=/home/maspectras/maspectrasjclusterservice/jobs
          jobdefinitions_path=/home/maspectras/maspectrasjclusterservice/job_definitions

9.4.2. The queueingsystem specific once:
9.4.2.1. Local execution queue

             local_execution_job_template_path=JCLUSTERSERVICEROOTDIRECTORY/template/jms-job-definition
             e.g.
             local_execution_job_template_path=/home/maspectras/maspectrasjclusterservice/template/jms-job-definition

9.4.2.2. Optional if present ... Sun Grid Engine (SGE) Adapt the paths to the SGE commands
9.4.2.3. Optional if present ... Portable Batch System (PBS/Torque) Adapt the paths to the PBS/Torque commands

9.5. Install the following applications:
9.5.1. Database repository management initialization:

set the JAVA_HOME path in JCLUSTERSERVICEROOTDIRECTORY/applications/bin/repomanager

initialize repository

               JCLUSTERSERVICEROOTDIRECTORY/applications/bin/repomanager -c init -r JCLUSTERSERVICEROOTDIRECTORY/repository
             e.g.
               /home/maspectras/maspectrasjclusterservice/applications/bin/repomanager -c init -r /home/maspectras/maspectrasjclusterservice/repository

9.5.2. NCBI-Blast and NCBI-FormatDB

download from ftp://ftp.ncbi.nih.gov/toolbox/ncbi_tools/ncbi.tar.gz the ncbi toolkit
install it according to the README from NCBI under JCLUSTERSERVICEROOTDIRECTORY/applications/bin
copy "blastall" and "formatdb" from "ncbi/build" into JCLUSTERSERVICEROOTDIRECTORY/applications/bin

9.5.3. CLUSTALW

download from ftp://ftp.ebi.ac.uk/pub/software/unix/clustalw/clustalw1.83.UNIX.tar.gz
install it according to the instructions from EBI under JCLUSTERSERVICEROOTDIRECTORY/applications/bin/
copy clustalw into JCLUSTERSERVICEROOTDIRECTORY/applications/bin

9.6. Adapt the JCLUSTERSERVICEROOTDIRECTORY/config/clusterservice-definition.xml for your local application installation
9.6.1. Change the REPOSITORYMANAGER service definition:

              program-path="JCLUSTERSERVICEROOTDIRECTORY/applications/bin/repomanager"
              e.g.
              program-path="/home/maspectras/maspectrasjclusterservice/applications/bin/repomanager"

and the path parameter:

               <parameter position="0" mandatory="true" name="repository" switch="-r">
                        <parameter-value isdefault="true">JCLUSTERSERVICEROOTDIRECTORY/repository</parameter-value>
                </parameter>
              e.g.
               <parameter position="0" mandatory="true" name="repository" switch="-r">
                        <parameter-value isdefault="true">/home/maspectras/maspectrasjclusterservice/repository</parameter-value>
                </parameter>

9.6.2. Change the NCBI-BLAST/NCBI-FORMATDB service definition:

              program-path="JCLUSTERSERVICEROOTDIRECTORY/applications/bin/blastall"
              e.g.
              program-path="/home/maspectras/maspectrasjclusterservice/applications/bin/blastall"

and

              program-path="JCLUSTERSERVICEROOTDIRECTORY/applications/bin/formatdb"
              e.g.
              program-path="/home/maspectras/maspectrasjclusterservice/applications/bin/formatdb"

9.6.3. Change the CLUSTALW service definition:

              program-path="JCLUSTERSERVICEROOTDIRECTORY/applications/bin/clustalw"
              e.g.
              program-path="/home/maspectras/maspectrasjclusterservice/applications/bin/clustalw"

9.6.4. The default installation does not require any queueingsystem or cluster and therefor every service definition is sent to the built-in local java jms queueing system. If you want to run CLUSTALW, BLAST- and FORMATDB- jobs on a distributed cluster just adapt the configuration parameters described in 3.2 and change in the service definition the parameter queuetype="jms" to queuetype="sge" or queuetype="pbs". In such case the JCLUSTERSERVICEROOTDIRECTORY must be shared (rw) under the same path over all cluster nodes.

9.7. Adapt on the Maspectras side the paths in DATAROOTDIRECTORY/analyses/partitioning/pipeline.xml

      <MCLProgramDirectory>JCLUSTERSERVICEROOTDIRECTORY/applications/bin</MCLProgramDirectory>
      <RemoteBlastDatabaseDirectory>JCLUSTERSERVICEROOTDIRECTORY/tmp/BlastDB</RemoteBlastDatabaseDirectory>
      <RemoteBlastExecutableDirectory></RemoteBlastExecutableDirectory>
      <RemoteInputDirectory>JCLUSTERSERVICEROOTDIRECTORY/tmp/Input</RemoteInputDirectory>
      <RemoteResultDirectory>JCLUSTERSERVICEROOTDIRECTORY/tmp/Output</RemoteResultDirectory>
    e.g.
      <MCLProgramDirectory>/home/maspectras/maspectrasjclusterservice/applications/bin</MCLProgramDirectory>
      <RemoteBlastDatabaseDirectory>/home/maspectras/maspectrasjclusterservice/tmp/BlastDB</RemoteBlastDatabaseDirectory>
      <RemoteBlastExecutableDirectory></RemoteBlastExecutableDirectory>
      <RemoteInputDirectory>/home/maspectras/maspectrasjclusterservice/tmp/Input</RemoteInputDirectory>
      <RemoteResultDirectory>/home/maspectras/maspectrasjclusterservice/tmp/Output</RemoteResultDirectory>

and the connection parameter ServerURL and ServerURLPlusUser in DATAROOTDIRECTORY/analyses/partitioning/cluster.properties. Default connection settings presume that the jclusterservice is running on the same host as maspectras on http. If its runnning on a different machine replace the http://username:password@localhost:21080 by the proper host and port parameters in DATAROOTDIRECTORY/analyses/partitioning/cluster.properties . If possible secure the connection via SSL and replace in ServerURL and ServerURLPlusUser the http by https.

9.8. Start the jclusterservice under $JBOSS_HOME/bin with run-jclusterservice.sh The jclusterservice can run on a different execution host or a cluster system. In that way the applications queued and executed there are not influencing the performance of the webinterface.

10. Installation of the markov cluster

10.1. Download the MCL package from http://micans.org/mcl/ and install it. mcl 1.005, 05-118 has been
    tested with Maspectras
    Maspectras requires the programs clmformat, mcl, and mcxassemble.

10.2. Specify the path to the MCL package in the pipeline.xml file located in the
    $DATAROOTDIRECTORY/analyses/partitioning
    directory:
        e.g.: /usr/local/mcl Make sure that the maspectras user has access to /usr/local/mcl and to $DATAROOTDIRECTORY

11. Get access to the AAS

11.1. Open your Web browser and type http://localhost:17080/UsermanagementWeb

11.2. Login using username: admin password: 12345

(change the password afterwards, since this is the ADMINISTRATOR account for the whole AAS)

11.3. Logout

11.4. Login using username: maspectrasadmin password: 12345

(change the password afterwards, since this is the ADMINISTRATOR account only for maspectras)

12. Get access to Maspectras

12.1. Open your Web browser and type http://localhost:8080/maspectras