Maspectras installation procedure

Applicable for Release 1.1

 

1. Requirements

Hardware requirements:

Server with at least 1 GBytes of main memory (2GBytes are recommended)
     (optional a LINUX cluster)

Software requirements:

Oracle 9i or PostgreSQL 8.0.x or MySQL 4.1.xx/5.0.xx for Maspectras and/or the Usermanagement
Java JDK 1.5.x
be sure that JAVA_HOME points to your JDK installation

 

2. Installation Instructions


Maspectras consists of three components which can run independently on different machines.

  • AAS: provides the central usermanagement and its access controls (can run on Windows/LINUX/Solaris)
  • JClusterService: provides a webservice used for external sequence database management,
    protein grouping and mass quantification. It is recommended, that those disk- and computing-
    intensive services should run on a separate machine or even better on a linux cluster (must run on LINUX / Solaris not tested)
  • Maspectras: is the core application which provides all functionality except the ones mentioned
    above (can run on Windows/LINUX/Solaris)
In order to perform a more painless installation and to avoid problems concerning access rights
please execute the following steps as one single user. If you prefer to install Maspectras and its
components on a Linux system create a user like it is described in 9.1. and switch to this user (su - maspectras)
for all the following steps.

Security: The communication between the 3 components of maspectras is based on http connections
which should be secured by SSL layer security (HTTPS). In order to reduce complexity of this
installation instruction HTTPS adaptations were skipped. In our productive environment all
communications between maspectras, usermanagement and jclusterservices are performed over https.


1. Unpacking Maspectras

1.1. Unpack maspectras.tar.gz

2.a MySQL inizialisation (must be InnoDB and support for InnoDB must be active)

2.a.1. Create a new schema called usermanagement for the AAS

           Create a new schema called maspectras for maspectras


2.a.2. Create 2 new users

  • for AAS
                username: usermn
                password: usermn
  • for maspectras
                 username: maspectras
                 password: maspectras
2.a.3. Grant the usermn user all priviliges for the usermanagement schema
           Grant the maspectras user all priviliges for the maspectras schema

2.b PostgreSQL installation

2.b.1. Create 2 new users

  • for maspectras
                 username: maspectras
                 password: maspectras
  • for AAS
                username: usermanagement
                password: usermanagement
2.b.2 Create a new database called maspectras

2.b.3 Create a new schema called usermanagement (the schema public must already exist, maspectras is mapping by default to public)


2.b.4. Grant the usermanagement user all priviliges for the usermanagement schema
           Grant the maspectras user all priviliges for the public schema

2.c Oracle installation

2.c.1. Create 2 new tablespaces (MASPECTRAS and USERMANAGEMENT)

2.c.2. Create 2 new users like 'Scott'

  • for AAS
                username: usermanagement
                password: usermanagement
  • for maspectras
                 username: maspectras
                 password: maspectras





3. Configuring the JBoss for the right database

3.a MySQL

  • for AAS
                 For MySQL 4.1.xx copy mysql-connector-java-3.1.10-bin.jar from database/config/usermanagement/mysql/ to
                 $JBOSS_HOME/server/usermanagement/lib/
                 For MySQL 5.0.xx copy mysql-connector-java-5.0.4-bin.jar from database/config/usermanagement/mysql/ to
                 $JBOSS_HOME/server/usermanagement/lib/
                 Copy mysql-ds.xml from database/config/usermanagement/mysql/ to
                 $JBOSS_HOME/server/usermanagement/deploy/
                 The mysql-ds.xml is the config file for your database, it is configured for localhost.
                 If you have other settings change them in mysql-ds.xml.
  • for maspectras
                 For MySQL 4.1.xx copy mysql-connector-java-3.1.10-bin.jar from database/config/maspectras/mysql/ to
                 $JBOSS_HOME/server/maspectras/lib/
                 For MySQL 5.0.xx copy mysql-connector-java-5.0.4-bin.jar from database/config/maspectras/mysql/ to
                 $JBOSS_HOME/server/maspectras/lib/
                 Copy mysql-ds.xml from database/config/maspectras/mysql/ to
                 $JBOSS_HOME/server/maspectras/deploy/
                 The mysql-ds.xml is the config file for your database, it is configured for localhost.
                 If you have other settings change them in mysql-ds.xml.

3.b PostgreSQL
  • for AAS
                 Copy postgresql-8.0-312.jdbc3.jar from database/config/usermanagement/postgresql/ to
                 $JBOSS_HOME/server/usermanagement/lib/
                 Copy postgres-ds.xml from database/config/usermanagement/postgresql/ to
                 $JBOSS_HOME/server/usermanagement/deploy/
                 The postgres-ds.xml is the config file for your database, it is configured for localhost.
                 If you have other settings change them in postgres-ds.xml.
  • for maspectras
                 Copy postgresql-8.0-312.jdbc3.jar from database/config/maspectras/postgresql/ to
                 $JBOSS_HOME/server/maspectras/lib/
                 Copy postgres-ds.xml from database/config/maspectras/postgresql/ to
                 $JBOSS_HOME/server/maspectras/deploy/
                 The postgres-ds.xml is the config file for your database, it is configured for localhost.
                 If you have other settings change them in postgres-ds.xml.

3.c Oracle
  • for AAS
                 Copy ojdbc14.jar from database/config/usermanagement/oracle/ to
                 $JBOSS_HOME/server/usermanagement/lib/
                 Copy oracle-ds.xml from database/config/usermanagement/oracle/ to
                 $JBOSS_HOME/server/usermanagement/deploy/
                 In the oracle-ds.xml enter the sid of your server instead of "yoursid"
                 The oracle-ds.xml is the config file for your database, it is configured for localhost.
                 If you have other settings change them in oracle-ds.xml
  • for maspectras
                 Copy ojdbc14.jar from database/config/maspectras/oracle/ to
                 $JBOSS_HOME/server/maspectras/lib/
                 Copy postgres-ds.xml from database/config/maspectras/oracle/ to
                 $JBOSS_HOME/server/maspectras/deploy/
                 In the oracle-ds.xml enter the sid of your server instead of "yoursid"
                 The oracle-ds.xml is the config file for your database, it is configured for localhost.
                 If you have other settings change them in oracle-ds.xml

4. Starting the AAS

4.1. Go to $JBOSS_HOME/bin

4.2. Start supplied Authentication and Authorization System (AAS) with
     run-usermanagement.bat (for Windows) or
     run-usermanagement.sh (for Linux or Solaris)
  and wait until the JBoss started

4.3 Use from /database/initialize/usermanagement
         initializeAAS_mysql.sql (for MySQL 4.1.xx) or
         initializeAAS_mysql_5.0.sql (for MySQL 5.0.xx) or
         initializeAAS_postgresql.sql (for PostgreSQL) or
         initializeAAS_oracle.sql (for Oracle)
    to insert required data

5. Create data root

5.1 Create your DATAROOTDIRECTORY. The DATAROOTDIRECTORY is the root path where all the files which are uploaded or can be downloaded from Maspectras are stored. Pay attention! This volume should be very large since all the scanned files are stored there, too!
e.g. /home/maspectrasdata/
Make sure that the maspectras user has read and write access to $DATAROOTDIRECTORY and that the $DATAROOTDIRECTORY ends with a slash

5.2. Copy the folders config and analyses from upload/ to your created DATAROOTDIRECTORY

6. Configuring maspectras for the AAS (only necessary if AAS is not running on the same machine like maspectras, otherwise skip this point)

6.1. Open the file maspectras.properties at $JBOSS_HOME/server/maspectras/ and at $JBOSS_HOME/server/maspectras/deploy/maspectras.war/WEB-INF/

6.2. Change the localhost behind the http_url= to your AAS address

6.3. If your usermanagement is secured with https change the

        protocol=HTTP
        
to
        protocol=HTTPS
        
and adapt the parameters
           ##### HTTPS Protocol #####
           https_url=localhost
           https_port=17433
           keystorefile=/path/to/keystore/keystore
        

7. Starting maspectras

7.1 Use from /database/initialize/maspectras/
         mysql/insertFKsCreateMySQLAll.sql (for MySQL) or
         postgresql/insertFKsCreatePostgreSQLAll.sql (for PostgreSQL) or
         oracle/insertFKsCreateOracleAll.sql (for Oracle)
    to create some tables

    If you are using MySQL InnoDB must be now the default engine. The my.ini (Windows)/ my.cnf (Linux) must contain the following entry: default-storage-engine=INNODB
    If not MASPECTRAS won't work properly (For AAS MyISAM is sufficient). To correct enter: default-storage-engine=INNODB. Stop the AAS; Restart MySQL; Start the AAS

7.2. Go to $JBOSS_HOME/bin

7.3. Start maspectras with
     run-maspectras.bat (for Windows) or
     run-maspectras.sh (for Linux or Solaris)
  and wait until the JBoss started

7.4 Use from /database/initialize/maspectras/
         mysql/sequestInitialValuesMySQL.sql (for MySQL) or
         postgresql/sequestInitialValuesPostgreSQL.sql (for PostgreSQL) or
         oracle/sequestInitialValues.sql (for Oracle)
    to insert required data

8. Change the Maspectras settings and insert the path to the newly created DATAROOTDIRECTORY

8.1. Open http://localhost:8080

8.2. Click on JMX-Console

8.3. Click on service=ServerSettings

8.4. Use the method setProperty

p1 = DATAROOTDIRECTORY
p2 = /home/maspectrasdata/

Be sure that there is a trailing '/' for Linux or a trailing '\' for Windows otherwise you will have problems with the upload

8.5. Click on invoke

8.6. To check the properties use method „listProperties“

9. Install JClusterService on a UNIX/Linux system (it's not working under windows and not tested under OSX)

9.1. Create a user under which the jclusterservice should execute the commands and create a JCLUSTERSERVICEROOTDIRECTORY e.g. /home/maspectras/maspectrasjclusterservice where the user has access right. If not already done copy and unpack the maspectras.tar.gz again under the home directory of the new user. This creates a maspectras directory with all the necessary files. From there copy all files within maspectras/maspectrasjclusterservice to this directory and make sure that the user can access the folder. e.g.

          groupadd maspectras
          useradd -d /home/maspectras -g maspectras maspectras
          su - maspectras
	  mkdir -p /home/maspectras/maspectrasjclusterservice
          tar -xzf maspectras.tar.gz
	  cp -r %extracted path%/maspectrasjclusterservice/* /home/maspectras/maspectrasjclusterservice 
	  chown -R maspectras:maspectras /home/maspectras 
	  # (just to be sure that every file is accessible for the service user execute this command as root)
      

If you have not already switched to maspectras user, become maspectras user (su - maspectras)
9.2. Edit jboss_4.0.5.GA/server/jclusterservice/deploy/JClusterWS.ear/JClusterWSWeb.war/WEB-INF/server-config.wsdd and change to:

      <parameter name="attachments.Directory" 
                 value="%installationpath%/jboss_4.0.5.GA/server/JClusterWS/./deploy/JClusterWS.ear/JClusterWS/./deploy/JClusterWS.ear/JClusterWSWeb.war/WEB-INF/attachments"/>
    e.g.
         /home/maspectras/maspectras/jboss_4.0.5.GA/server/jclusterservice/deploy/JClusterWS.ear/JClusterWSWeb.war/WEB-INF/attachments
If you change the server ports for jclusterservice in bindings.xml, change also the JNDI URL in this file:
      <parameter name="jndiURL" value="jnp://localhost:%your_port%"/>
default is:
      <parameter name="jndiURL" value="jnp://localhost:21099"/>

9.3. Adapt the path settings to your local installation in jboss_4.0.5.GA/server/jclusterservice/deploy/JClusterWS.ear/JClusterService.jar/usermanagement-utils.properties

    sum_users_filepath=JCLUSTERSERVICEROOTDIRECTORY/config/users.xml
    e.g.
    sum_users_filepath=/home/maspectras/maspectrasjclusterservice/config/users.xml

9.4. Adapt the path settings to your local installation in jboss_4.0.5.GA/server/jclusterservice/deploy/JClusterWS.ear/JClusterService.jar/cluster.properties
  9.4.1. The global path settings:

          clusterservicedefinition_path=JCLUSTERSERVICEROOTDIRECTORY/config/clusterservice-definition.xml
	  job_path=JCLUSTERSERVICEROOTDIRECTORY/jobs
          jobdefinitions_path=JCLUSTERSERVICEROOTDIRECTORY/job_definitions
          e.g.
          clusterservicedefinition_path=/home/maspectras/maspectrasjclusterservice/config/clusterservice-definition.xml
	  job_path=/home/maspectras/maspectrasjclusterservice/jobs
          jobdefinitions_path=/home/maspectras/maspectrasjclusterservice/job_definitions

  9.4.2. The queueingsystem specific once:
    9.4.2.1. Local execution queue
             local_execution_job_template_path=JCLUSTERSERVICEROOTDIRECTORY/template/jms-job-definition
             e.g.
             local_execution_job_template_path=/home/maspectras/maspectrasjclusterservice/template/jms-job-definition
    9.4.2.2. Optional if present ... Sun Grid Engine (SGE) Adapt the paths to the SGE commands
    9.4.2.3. Optional if present ... Portable Batch System (PBS/Torque) Adapt the paths to the PBS/Torque commands

9.5. Install the following applications:
  9.5.1. Database repository management initialization:

  • set the JAVA_HOME path in JCLUSTERSERVICEROOTDIRECTORY/applications/bin/repomanager (open the repomanager file and type e.g. JAVA_HOME="/usr/java/jdk1.5.0_11" at the beginning of the file)
  • initialize repository
                   JCLUSTERSERVICEROOTDIRECTORY/applications/bin/repomanager -c init -r JCLUSTERSERVICEROOTDIRECTORY/repository
                 e.g.
                   /home/maspectras/maspectrasjclusterservice/applications/bin/repomanager -c init -r /home/maspectras/maspectrasjclusterservice/repository
  9.5.2. NCBI-Blast and NCBI-FormatDB
  • download from ftp://ftp.ncbi.nih.gov/toolbox/ncbi_tools/ncbi.tar.gz the ncbi toolkit
  • install it according to the README from NCBI under JCLUSTERSERVICEROOTDIRECTORY/applications/bin
  • copy "blastall" and "formatdb" from "ncbi/build" into JCLUSTERSERVICEROOTDIRECTORY/applications/bin
  • copy the whole "data" directory from "ncbi/data" to the location JCLUSTERSERVICEROOTDIRECTORY/applications/data

  9.5.3. CLUSTALW
  • download from ftp://ftp.ebi.ac.uk/pub/software/unix/clustalw/clustalw1.83.UNIX.tar.gz
  • install it according to the instructions from EBI under JCLUSTERSERVICEROOTDIRECTORY/applications/bin/
  • copy clustalw into JCLUSTERSERVICEROOTDIRECTORY/applications/bin

9.6. Adapt the JCLUSTERSERVICEROOTDIRECTORY/config/clusterservice-definition.xml for your local application installation
  9.6.1. Change the REPOSITORYMANAGER service definition:

              program-path="JCLUSTERSERVICEROOTDIRECTORY/applications/bin/repomanager"
              e.g.
              program-path="/home/maspectras/maspectrasjclusterservice/applications/bin/repomanager"
and the path parameter:
               <parameter position="0" mandatory="true" name="repository" switch="-r">
                        <parameter-value isdefault="true">JCLUSTERSERVICEROOTDIRECTORY/repository</parameter-value>
                </parameter>
              e.g.
               <parameter position="0" mandatory="true" name="repository" switch="-r">
                        <parameter-value isdefault="true">/home/maspectras/maspectrasjclusterservice/repository</parameter-value>
                </parameter>

  9.6.2. Change the NCBI-BLAST/NCBI-FORMATDB service definition:
              program-path="JCLUSTERSERVICEROOTDIRECTORY/applications/bin/blastall"
              e.g.
              program-path="/home/maspectras/maspectrasjclusterservice/applications/bin/blastall"
and
              program-path="JCLUSTERSERVICEROOTDIRECTORY/applications/bin/formatdb"
              e.g.
              program-path="/home/maspectras/maspectrasjclusterservice/applications/bin/formatdb"

  9.6.3. Change the CLUSTALW service definition:
              program-path="JCLUSTERSERVICEROOTDIRECTORY/applications/bin/clustalw"
              e.g.
              program-path="/home/maspectras/maspectrasjclusterservice/applications/bin/clustalw"

  9.6.4. The default installation does not require any queueingsystem or cluster and therefor every service definition is sent to the built-in local java jms queueing system. If you want to run CLUSTALW, BLAST- and FORMATDB- jobs on a distributed cluster just adapt the configuration parameters described in 3.2 and change in the service definition the parameter queuetype="jms" to queuetype="sge" or queuetype="pbs". In such case the JCLUSTERSERVICEROOTDIRECTORY must be shared (rw) under the same path over all cluster nodes.

9.7. Adapt on the Maspectras side the paths in DATAROOTDIRECTORY/analyses/partitioning/pipeline.xml

      <MCLProgramDirectory>/home/maspectras/mcl</MCLProgramDirectory>
      <RemoteBlastDatabaseDirectory>JCLUSTERSERVICEROOTDIRECTORY/tmp/BlastDB</RemoteBlastDatabaseDirectory>
      <RemoteBlastExecutableDirectory>JCLUSTERSERVICEROOTDIRECTORY/applications/bin</RemoteBlastExecutableDirectory>
      <RemoteInputDirectory>JCLUSTERSERVICEROOTDIRECTORY/tmp/Input</RemoteInputDirectory>
      <RemoteResultDirectory>JCLUSTERSERVICEROOTDIRECTORY/tmp/Output</RemoteResultDirectory>
    e.g.
      <MCLProgramDirectory>/home/maspectras/mcl</MCLProgramDirectory>
      <RemoteBlastDatabaseDirectory>/home/maspectras/maspectrasjclusterservice/tmp/BlastDB</RemoteBlastDatabaseDirectory>
      <RemoteBlastExecutableDirectory>/home/maspectras/maspectrasjclusterservice/applications/bin</RemoteBlastExecutableDirectory>
      <RemoteInputDirectory>/home/maspectras/maspectrasjclusterservice/tmp/Input</RemoteInputDirectory>
      <RemoteResultDirectory>/home/maspectras/maspectrasjclusterservice/tmp/Output</RemoteResultDirectory>
and the connection parameter ServerURL and ServerURLPlusUser in DATAROOTDIRECTORY/analyses/partitioning/cluster.properties. Default connection settings presume that the jclusterservice is running on the same host as maspectras on http. If its runnning on a different machine replace the http://username:password@localhost:21080 by the proper host and port parameters in DATAROOTDIRECTORY/analyses/partitioning/cluster.properties . If possible secure the connection via SSL and replace in ServerURL and ServerURLPlusUser the http by https.

9.8. Start the jclusterservice under $JBOSS_HOME/bin with run-jclusterservice.sh The jclusterservice can run on a different execution host or a cluster system. In that way the applications queued and executed there are not influencing the performance of the webinterface.


10. Installation of the markov cluster

10.1. Download the MCL package from http://micans.org/mcl/ and install it. mcl 1.005, 05-118 has been
    tested with Maspectras
    Maspectras requires the programs clmformat, mcl, and mcxassemble.

10.2. Specify the path to the MCL package in the pipeline.xml file located in the
    $DATAROOTDIRECTORY/analyses/partitioning
    directory:
        e.g.: /home/maspectras/mcl Make sure that the maspectras user has access to /home/maspectras/mcl and to $DATAROOTDIRECTORY

11. Get access to the AAS

11.1. Open your Web browser and type http://localhost:17080/UsermanagementWeb

11.2. Login using username: admin password: 12345

(change the password afterwards, since this is the ADMINISTRATOR account for the whole AAS)

11.3. Logout

11.4. Login using username: maspectrasadmin password: 12345

(change the password afterwards, since this is the ADMINISTRATOR account only for maspectras)

12. Get access to Maspectras

12.1. Open your Web browser and type http://localhost:8080/maspectras