ESPRIT :: Installation :: Cluster

Obtain the source code


On *nix, Steps to install ESPRIT
$ unzip ESPRIT_distribution.zip
$ cd ESPRIT_distribution


Read, esprit_user_guide.pdf and README.txt
$ cd source
$ vim Makefile
Choose the platform by uncomment/comment


To make the package
$ make esprit_cc


Its always better to 
$ make clean 
$ make esprit_cc


Precaution: 
- make sure that the fasta file has header in one and the sequence in one line
- if the sequence is in multiple lines, convert the file to contain just one line of sequence


Pseudocode is followed here using shell scripting and clusterjobmanager
Copy the sequence file here, If you have more than one than group them.
$ cp /path/to/sequence.fas .


To run preproc
$ /path/to/ESPRIT_distribution/source/preproc -f sequence.fas
160794 Seqs Match Primer
160794 Seqs Valid Len


31072 Seqs After Process
      1.63 secs in Purging Strings.


flag:
-f this prevents the program from trimming.


Files created:
sequence_Clean.fas
sequence_Clean.frq


To check
$ awk -F' ' '{ s+=$2 } END { print s }' sequence_Clean.frq
160794
$ grep -c '>' sequence.fas
160794
- Make sure that these numbers are same.


To run kmerdist_par
$ cat submit_kmer_jobs.sh
for i in $(seq 1 10)

do

for j in $(seq $i 10) 

do

job="/path/to/ESPRIT_distribution/source/kmerdist_par sequence_Clean.fas 10 $i $j\n ";

RANDOM=10

num=$RANDOM

echo -e $job > kmer_job_$num.clusterJob

clusterJobSubmission < kmer_job_$num.clusterJob

done

done

- where clusterJobSubmission is your cluster job submission manager

- the extension .clusterJob can be replaced with the extension required

- variable job can include other details if required.
$ cat jobs.clusterJob
## ..
sh submit_kmer_jobs.sh

$ jobsubmit < jobs. clusterJob
- this will submit the job.


Output:

sequence_Clean_[*]_[*].dist

- make sure that numbers are 1_[1-10], 2_[2-10], 3_[3-10], 4_[4-10], 5_[5-10], 6_[6-10], 7_[7-10], 8_[8-10], 9_[9-10], 10_10

Merge all the .dist files

$ cat sequence_Clean_*.dist >> kmer.dist

Split the kmer files into 100 files

$ /path/to/ESPRIT_distribution/source/splitdist -s 100 kmer.dist

Counting Total Records....

71249223 Records Found, Splitting...

Output:

kmer.dist_[0-99]

Submit parallel jobs for needle_dist

$ cat submit_needle_job.sh

for i in $(seq 0 99)

do

job="/path/to/ESPRIT_distribution/source/needledist sequence_Clean.fas kmer.dist\_$i needle.dist\_$i\n ";

RANDOM=10

num=$RANDOM

echo -e $job > needle_job_$num.clusterJob

clusterJobSubmission < needle_job_$num.clusterJob

done

Output

needle.dist_[0-99]

Group all the needle.dist files

$ cat needle.dist_* >> sequence.ndist


To run hcluster
$ /path/to/ESPRIT_distribution/source/hcluster -t 15000 sequence.ndist sequence_Clean.frq
- flag -t is used to increase the size of the linked table, default is 10000


Output
sequence.ndist_sort
sequence.OTU
sequence.Outliers
sequence.Rarefaction
sequence.Cluster
sequence.Cluster_List
sequence.ACE
sequence.CHAO1