Bioinformatics Graz

Maspectras installation procedure

Applicable for Release 2.0

1. Requirements

Hardware requirements:

Server with at least 1 GBytes of main memory (2GBytes are recommended)
(optional a LINUX cluster)

Software requirements:

Oracle (9i or 10g) or PostgreSQL 8.1.x or MySQL 4.1.xx/5.0.xx for Maspectras and/or the Usermanagement
Java JDK 1.5.x or JDK 1.6.x
be sure that JAVA_HOME points to your JDK installation

2. Installation Instructions

Maspectras consists of three components which can run independently on different machines.

AAS: provides the central usermanagement and its access controls (can run on Windows/LINUX/Solaris)

JClusterService: provides a webservice used for external sequence database management,
protein grouping and mass quantification. It is recommended, that those disk- and computing-
intensive services should run on a separate machine or even better on a linux cluster (must run on
LINUX (for Ubuntu /bin/bash has to be installed)/ Solaris not tested)

Maspectras: is the core application which provides all functionality except the ones mentioned
above (can run on Windows/LINUX/Solaris)

In order to perform a more painless installation and to avoid problems concerning access rights
please execute the following steps as one single user. If you prefer to install Maspectras and its
components on a Linux system create a user like it is described in 9.1. and switch to this user (su - maspectras)
for all the following steps.

Security: The communication between the 3 components of maspectras is based on http connections
which should be secured by SSL layer security (HTTPS). In order to reduce complexity of this
installation instruction HTTPS adaptations were skipped. In our productive environment all
communications between maspectras, usermanagement and jclusterservices are performed over https.

1. Unpacking Maspectras

1.1. Unpack maspectras.tar.gz

2.a MySQL inizialisation (must be InnoDB and support for InnoDB must be active)

2.a.1. Create a new schema called usermanagement for the AAS
Create a new schema called maspectras for maspectras

2.a.2. Create 2 new users

for AAS
username: usermn
password: usermn
for maspectras
username: maspectras
password: maspectras

2.a.3. Grant the usermn user all priviliges for the usermanagement schema
Grant the maspectras user all priviliges for the maspectras schema

2.b PostgreSQL installation

2.b.1. Create 2 new users

for maspectras
username: maspectras
password: maspectras
for AAS
username: usermanagement
password: usermanagement

2.b.2 Create a new database called maspectras

2.b.3 Create a new schema called usermanagement (the schema public must already exist, maspectras is mapping by default to public)

2.b.4. Grant the usermanagement user all priviliges for the usermanagement schema
Grant the maspectras user all priviliges for the public schema

2.c Oracle installation

2.c.1. Create 2 new tablespaces (MASPECTRAS and USERMANAGEMENT)

2.c.2. Create 2 new users like 'Scott'

for AAS
username: usermanagement
password: usermanagement
for maspectras
username: maspectras
password: maspectras

3. Configuring the JBoss for the right database

3.a MySQL

for AAS
             For MySQL 4.1.xx copy mysql-connector-java-3.1.10-bin.jar from database/config/usermanagement/mysql/ to
             $JBOSS_HOME/server/usermanagement/lib/
             For MySQL 5.0.xx copy mysql-connector-java-5.0.4-bin.jar from database/config/usermanagement/mysql/ to
             $JBOSS_HOME/server/usermanagement/lib/
             Copy mysql-ds.xml from database/config/usermanagement/mysql/ to
             $JBOSS_HOME/server/usermanagement/deploy/
             The mysql-ds.xml is the config file for your database, it is configured for localhost.
             If you have other settings change them in mysql-ds.xml.
for maspectras
             For MySQL 4.1.xx copy mysql-connector-java-3.1.10-bin.jar from database/config/maspectras/mysql/ to
             $JBOSS_HOME/server/maspectras/lib/
             For MySQL 5.0.xx copy mysql-connector-java-5.0.4-bin.jar from database/config/maspectras/mysql/ to
             $JBOSS_HOME/server/maspectras/lib/
             Copy mysql-ds.xml from database/config/maspectras/mysql/ to
             $JBOSS_HOME/server/maspectras/deploy/
             The mysql-ds.xml is the config file for your database, it is configured for localhost.
             If you have other settings change them in mysql-ds.xml.

3.b PostgreSQL

for AAS
             Copy postgresql-8.1-412.jdbc3.jar from database/config/usermanagement/postgresql/ to
             $JBOSS_HOME/server/usermanagement/lib/
             Copy postgres-ds.xml from database/config/usermanagement/postgresql/ to
             $JBOSS_HOME/server/usermanagement/deploy/
             The postgres-ds.xml is the config file for your database, it is configured for localhost.
             If you have other settings change them in postgres-ds.xml.
for maspectras
             Copy postgresql-8.1-412.jdbc3.jar from database/config/maspectras/postgresql/ to
             $JBOSS_HOME/server/maspectras/lib/
             Copy postgres-ds.xml from database/config/maspectras/postgresql/ to
             $JBOSS_HOME/server/maspectras/deploy/
             The postgres-ds.xml is the config file for your database, it is configured for localhost.
             If you have other settings change them in postgres-ds.xml.

3.c Oracle

for AAS
             Copy ojdbc14.jar from database/config/usermanagement/oracle/ to
             $JBOSS_HOME/server/usermanagement/lib/
             Copy oracle-ds.xml from database/config/usermanagement/oracle/ to
             $JBOSS_HOME/server/usermanagement/deploy/
             In the oracle-ds.xml enter the sid of your server instead of "yoursid"
             The oracle-ds.xml is the config file for your database, it is configured for localhost.
             If you have other settings change them in oracle-ds.xml
for maspectras
             Copy ojdbc14.jar from database/config/maspectras/oracle/ to
             $JBOSS_HOME/server/maspectras/lib/
             Copy postgres-ds.xml from database/config/maspectras/oracle/ to
             $JBOSS_HOME/server/maspectras/deploy/
             In the oracle-ds.xml enter the sid of your server instead of "yoursid"
             The oracle-ds.xml is the config file for your database, it is configured for localhost.
             If you have other settings change them in oracle-ds.xml

4. Starting the AAS

4.1. Go to $JBOSS_HOME/bin

4.2. Start supplied Authentication and Authorization System (AAS) with
     run-usermanagement.bat (for Windows) or
     run-usermanagement.sh (for Linux or Solaris)
  and wait until the JBoss started

4.3 Use from /database/initialize/usermanagement
         initializeAAS_mysql.sql (for MySQL 4.1.xx) or
         initializeAAS_mysql_5.0.sql (for MySQL 5.0.xx) or
         initializeAAS_postgresql.sql (for PostgreSQL) or
         initializeAAS_oracle.sql (for Oracle)
    to insert required data

5. Create data root

5.1 Create your DATAROOTDIRECTORY. The DATAROOTDIRECTORY is the root path where all the files which are uploaded or can be downloaded from Maspectras are stored. Pay attention! This volume should be very large since all the scanned files are stored there, too!
e.g. /home/maspectrasdata/
Make sure that the maspectras user has read and write access to $DATAROOTDIRECTORY and that the $DATAROOTDIRECTORY ends with a slash
5.2. Copy the folders config and analyses from upload/ to your created DATAROOTDIRECTORY

6. Configuring maspectras for the AAS (only necessary if AAS is not running on the same machine like maspectras, otherwise skip this point)

6.1. Open the file maspectras.properties at $JBOSS_HOME/server/maspectras/ and at $JBOSS_HOME/server/maspectras/deploy/maspectras.war/WEB-INF/

6.2. Change the localhost behind the http_url= to your AAS address

6.3. If your usermanagement is secured with https change the

        protocol=HTTP

        protocol=HTTPS

and adapt the parameters

           ##### HTTPS Protocol #####
           https_url=localhost
           https_port=17433
           keystorefile=/path/to/keystore/keystore

7. Starting maspectras

7.1 Use from /database/initialize/maspectras/
         mysql/insertFKsCreateMySQLAll.sql (for MySQL) or
         postgresql/insertFKsCreatePostgreSQLAll.sql (for PostgreSQL) or
         oracle/insertFKsCreateOracleAll.sql (for Oracle)
    to create some tables

    If you are using MySQL InnoDB must be now the default engine. The my.ini (Windows)/ my.cnf (Linux) must contain the following entry: default-storage-engine=INNODB
    If not MASPECTRAS won't work properly (For AAS MyISAM is sufficient). To correct enter: default-storage-engine=INNODB. Stop the AAS; Restart MySQL; Start the AAS

7.2. Go to $JBOSS_HOME/bin

7.3. Start maspectras with
     run-maspectras.bat (for Windows) or
     run-maspectras.sh (for Linux or Solaris)
  and wait until the JBoss started

7.4 Use from /database/initialize/maspectras/
         mysql/sequestInitialValuesMySQL.sql (for MySQL) or
         postgresql/sequestInitialValuesPostgreSQL.sql (for PostgreSQL) or
         oracle/sequestInitialValues.sql (for Oracle)
    to insert required data

8. Change the Maspectras settings and insert the path to the newly created DATAROOTDIRECTORY

8.1. Open http://localhost:8080

8.2. Click on JMX-Console

8.3. Click on service=ServerSettings

8.4. Use the method setProperty

p1 = DATAROOTDIRECTORY
p2 = /home/maspectrasdata/

Be sure that there is a trailing '/' for Linux or a trailing '\' for Windows otherwise you will have problems with the upload

8.5. Click on invoke

8.6. To check the properties use method „listProperties“

9. Install JClusterService on a UNIX/Linux system (it's not working under windows and not tested under OSX)

9.1. Create a user under which the jclusterservice should execute the commands and create a JCLUSTERSERVICEROOTDIRECTORY e.g. /home/maspectras/maspectrasjclusterservice where the user has access right. If not already done copy and unpack the maspectras.tar.gz again under the home directory of the new user. This creates a maspectras directory with all the necessary files. From there copy all files within maspectras/maspectrasjclusterservice to this directory and make sure that the user can access the folder. e.g.

          groupadd maspectras
          useradd -d /home/maspectras -g maspectras maspectras
          su - maspectras
	  mkdir -p /home/maspectras/maspectrasjclusterservice
          tar -xzf maspectras.tar.gz
	  cp -r %extracted path%/maspectrasjclusterservice/* /home/maspectras/maspectrasjclusterservice 
	  chown -R maspectras:maspectras /home/maspectras 
	  # (just to be sure that every file is accessible for the service user execute this command as root)

If you have not already switched to maspectras user, become maspectras user (su - maspectras)
9.2. Edit jboss-4.2.2.GA/server/jclusterservice/deploy/JClusterWS.ear/JClusterWSWeb.war/WEB-INF/server-config.wsdd and change to:

      <parameter name="attachments.Directory" 
                 value="%installationpath%/jboss-4.2.2.GA/server/JClusterWS/./deploy/JClusterWS.ear/JClusterWS/./deploy/JClusterWS.ear/JClusterWSWeb.war/WEB-INF/attachments"/>
    e.g.
         /home/maspectras/maspectras/jboss-4.2.2.GA/server/jclusterservice/deploy/JClusterWS.ear/JClusterWSWeb.war/WEB-INF/attachments

If you change the server ports for jclusterservice in bindings.xml, change also the JNDI URL in this file:

      <parameter name="jndiURL" value="jnp://localhost:%your_port%"/>

default is:

      <parameter name="jndiURL" value="jnp://localhost:21099"/>

9.3. Adapt the path settings to your local installation in jboss-4.2.2.GA/server/jclusterservice/deploy/JClusterWS.ear/JClusterService.jar/usermanagement-utils.properties

    sum_users_filepath=JCLUSTERSERVICEROOTDIRECTORY/config/users.xml
    e.g.
    sum_users_filepath=/home/maspectras/maspectrasjclusterservice/config/users.xml

9.4. Adapt the path settings to your local installation in jboss-4.2.2.GA/server/jclusterservice/deploy/JClusterWS.ear/JClusterService.jar/cluster.properties
9.4.1. The global path settings:

          clusterservicedefinition_path=JCLUSTERSERVICEROOTDIRECTORY/config/clusterservice-definition.xml
	  job_path=JCLUSTERSERVICEROOTDIRECTORY/jobs
          jobdefinitions_path=JCLUSTERSERVICEROOTDIRECTORY/job_definitions
          e.g.
          clusterservicedefinition_path=/home/maspectras/maspectrasjclusterservice/config/clusterservice-definition.xml
	  job_path=/home/maspectras/maspectrasjclusterservice/jobs
          jobdefinitions_path=/home/maspectras/maspectrasjclusterservice/job_definitions

9.4.2. The queueingsystem specific once:
9.4.2.1. Local execution queue

             local_execution_job_template_path=JCLUSTERSERVICEROOTDIRECTORY/template/jms-job-definition
             e.g.
             local_execution_job_template_path=/home/maspectras/maspectrasjclusterservice/template/jms-job-definition

9.4.2.2. Optional if present ... Sun Grid Engine (SGE) Adapt the paths to the SGE commands
9.4.2.3. Optional if present ... Portable Batch System (PBS/Torque) Adapt the paths to the PBS/Torque commands

9.5. Install the following applications:
9.5.1. Database repository management initialization:

set the JAVA_HOME path in JCLUSTERSERVICEROOTDIRECTORY/applications/bin/repomanager (open the repomanager file and type e.g. JAVA_HOME="/usr/java/jdk1.5.0_11" at the beginning of the file)

initialize repository

               JCLUSTERSERVICEROOTDIRECTORY/applications/bin/repomanager -c init -r JCLUSTERSERVICEROOTDIRECTORY/repository
             e.g.
               /home/maspectras/maspectrasjclusterservice/applications/bin/repomanager -c init -r /home/maspectras/maspectrasjclusterservice/repository

9.5.2. NCBI-Blast and NCBI-FormatDB

download from ftp://ftp.ncbi.nih.gov/toolbox/ncbi_tools/ncbi.tar.gz the ncbi toolkit
install it according to the README from NCBI under JCLUSTERSERVICEROOTDIRECTORY/applications/bin
copy "blastall" and "formatdb" from "ncbi/build" into JCLUSTERSERVICEROOTDIRECTORY/applications/bin
copy the whole "data" directory from "ncbi/data" to the location JCLUSTERSERVICEROOTDIRECTORY/applications/data

9.5.3. CLUSTALW

download from ftp://ftp.ebi.ac.uk/pub/software/unix/clustalw/clustalw1.83.UNIX.tar.gz
install it according to the instructions from EBI under JCLUSTERSERVICEROOTDIRECTORY/applications/bin/
copy clustalw into JCLUSTERSERVICEROOTDIRECTORY/applications/bin

9.6. Adapt the JCLUSTERSERVICEROOTDIRECTORY/config/clusterservice-definition.xml for your local application installation
9.6.1. Change the REPOSITORYMANAGER service definition:

              program-path="JCLUSTERSERVICEROOTDIRECTORY/applications/bin/repomanager"
              e.g.
              program-path="/home/maspectras/maspectrasjclusterservice/applications/bin/repomanager"

and the path parameter:

               <parameter position="0" mandatory="true" name="repository" switch="-r">
                        <parameter-value isdefault="true">JCLUSTERSERVICEROOTDIRECTORY/repository</parameter-value>
                </parameter>
              e.g.
               <parameter position="0" mandatory="true" name="repository" switch="-r">
                        <parameter-value isdefault="true">/home/maspectras/maspectrasjclusterservice/repository</parameter-value>
                </parameter>

9.6.2. Change the NCBI-BLAST/NCBI-FORMATDB service definition:

              program-path="JCLUSTERSERVICEROOTDIRECTORY/applications/bin/blastall"
              e.g.
              program-path="/home/maspectras/maspectrasjclusterservice/applications/bin/blastall"

and

              program-path="JCLUSTERSERVICEROOTDIRECTORY/applications/bin/formatdb"
              e.g.
              program-path="/home/maspectras/maspectrasjclusterservice/applications/bin/formatdb"

9.6.3. Change the CLUSTALW service definition:

              program-path="JCLUSTERSERVICEROOTDIRECTORY/applications/bin/clustalw"
              e.g.
              program-path="/home/maspectras/maspectrasjclusterservice/applications/bin/clustalw"

9.6.4. Change the MASSQUANTIFICATION service definition:

              program-path="JCLUSTERSERVICEROOTDIRECTORY/applications/bin/msQuantificationCmd"
              e.g.
              program-path="/home/maspectras/maspectrasjclusterservice/applications/bin/msQuantificationCmd"

9.6.5. The default installation does not require any queueingsystem or cluster and therefore every service definition is sent to the built-in local java jms queueing system. If you want to run CLUSTALW, BLAST- and FORMATDB- jobs on a distributed cluster just adapt the configuration parameters described in 3.2 and change in the service definition the parameter queuetype="jms" to queuetype="sge" or queuetype="pbs". In such case the JCLUSTERSERVICEROOTDIRECTORY must be shared (rw) under the same path over all cluster nodes.

9.7. Adapt on the Maspectras side the paths in DATAROOTDIRECTORY/analyses/partitioning/pipeline.xml

      <MCLProgramDirectory>/home/maspectras/mcl</MCLProgramDirectory>
      <RemoteBlastDatabaseDirectory>JCLUSTERSERVICEROOTDIRECTORY/tmp/BlastDB</RemoteBlastDatabaseDirectory>
      <RemoteBlastExecutableDirectory>JCLUSTERSERVICEROOTDIRECTORY/applications/bin</RemoteBlastExecutableDirectory>
      <RemoteInputDirectory>JCLUSTERSERVICEROOTDIRECTORY/tmp/Input</RemoteInputDirectory>
      <RemoteResultDirectory>JCLUSTERSERVICEROOTDIRECTORY/tmp/Output</RemoteResultDirectory>
    e.g.
      <MCLProgramDirectory>/home/maspectras/mcl</MCLProgramDirectory>
      <RemoteBlastDatabaseDirectory>/home/maspectras/maspectrasjclusterservice/tmp/BlastDB</RemoteBlastDatabaseDirectory>
      <RemoteBlastExecutableDirectory>/home/maspectras/maspectrasjclusterservice/applications/bin</RemoteBlastExecutableDirectory>
      <RemoteInputDirectory>/home/maspectras/maspectrasjclusterservice/tmp/Input</RemoteInputDirectory>
      <RemoteResultDirectory>/home/maspectras/maspectrasjclusterservice/tmp/Output</RemoteResultDirectory>

and the connection parameter ServerURL and ServerURLPlusUser in DATAROOTDIRECTORY/analyses/partitioning/cluster.properties. Default connection settings presume that the jclusterservice is running on the same host as maspectras on http. If its runnning on a different machine replace the http://username:password@localhost:21080 by the proper host and port parameters in DATAROOTDIRECTORY/analyses/partitioning/cluster.properties . If possible secure the connection via SSL and replace in ServerURL and ServerURLPlusUser the http by https.

9.8. Start the jclusterservice under $JBOSS_HOME/bin with run-jclusterservice.sh The jclusterservice can run on a different execution host or a cluster system. In that way the applications queued and executed there are not influencing the performance of the webinterface.

10. Installation of the markov cluster

10.1. Download the MCL package from http://micans.org/mcl/ and install it. mcl 1.005, 05-118 has been
    tested with Maspectras
    Maspectras requires the programs clmformat, mcl, and mcxassemble.

10.2. Specify the path to the MCL package in the pipeline.xml file located in the
    $DATAROOTDIRECTORY/analyses/partitioning
    directory:
        e.g.: /home/maspectras/mcl Make sure that the maspectras user has access to /home/maspectras/mcl and to $DATAROOTDIRECTORY

11. Get access to the AAS

11.1. Open your Web browser and type http://localhost:17080/UsermanagementWeb

11.2. Login using username: admin password: 12345

(change the password afterwards, since this is the ADMINISTRATOR account for the whole AAS)

11.3. Logout

11.4. Login using username: maspectrasadmin password: 12345

(change the password afterwards, since this is the ADMINISTRATOR account only for maspectras)

12. Get access to Maspectras

12.1. Open your Web browser and type http://localhost:8080/maspectras

13. Test Maspectras

13.1. Open a new browser and type https://maspectras.genome.tugraz.at

13.2. Login as guest user (username: guest ; password: guest)

13.3. In the left menu click "Management" -> "File Upload"

13.4. Click on "Query", type "Name" "=" "ICPL_Protmix_1lizu1he_A_c1_ms2_Mascot" and download the Mascot file (for help see user manual 1.2.1 and 1.2.3)

13.5. Click on "Query", type "Name" "=" "ICPL_Protmix_1lizu1he_A_c1_Centroid" and download the mzXML file containing the raw data (check if the file suffix is mzXML and not mzXML.xml, if not change the suffix to mzXML)

13.6. Logout as guest user and continue working at http://localhost:8080/maspectras

13.7. Upload and initialize the database doc/testdatabase/karl_04090683392.fa with the parsing rules of doc/parsingRules/giDatabase_parserules like in the user manual described at chapter 2.1

13.8. Upload the Mascot file (File Type = mascot) and the mzXML file (File Type = Raw-File) like in the user manual described at chapter 2.2

13.9. When you started the upload of the mzXML file the file is translated into a file format more convenient for calculation. Check the status of this translation at Uploadstatus (user manual 2.3). Wait until "TRANSLATING" is replaced by "TRANSLATION FINISHED"

13.10. Create a new Massspecexperiment (for help see user manual 6.1) and assign the mzXML as RawFile

13.11. Start the parsing of the Mascot file (for help see user manual 6.2)

13.12. When the parsing is finished check your results (for help see user manual 7 and 7.1). In the clustered protein view must be 16 protein(-clusters), then the clustering worked.

14. Installation of Java3D for the client machine

To use the real 3D view Java3D must be installed on the client machine. To download Java3D visit the web-site http://java.sun.com/products/java-media/3D/download.html. It is recommended to increase the memory size for the applet, for more information visit the Maspectras user manual 7.5.1.