# PPSS- a handy tool for parallel processing

**We were looking for a light Linux shell tool that could process given commands using advantage of multicore system (parallel processing). Two of the tools that we came across are worth writing about - parallel and PPSS. This blog discusses PPSS, while parallel can be food for a future blog.**

PPSS can be downloaded from <https://code.google.com/p/ppss/> (there are deb and rpm files too). PPSS is a shell script that can be used to run any command, script, or program in parallel. All it needs is a source (file or dir) and a command to execute.

Example-.

 ./ppss -d -c '' -p  ./ppss -f -c '' -p  <a name="more"></a> If source is a directory, then it executes the command on each file in the directory and if the source is a file, it executes the command on each line in the file.Note: command should always be enclosed in single quotes. Argument (each file in source directory or each line in source file) can be accessed in command by the variable - "$ITEM". At any time, number of items being processed will never increase the cores available, say while processing 50GB of data.

$~/bin/ppss -f list\_of\_files\_to\_be\_processed.txt -c 'zgrep ""

 ../../"$ITEM"' -p 2 Jul 26 09:11:22: =========================================================  Jul 26 09:11:22: |P|P|S|S| Jul 26 09:11:22: Distributed Parallel Processing Shell Script vers. 2.97  Jul 26 09:11:22: ========================================================= Jul 26 09:11:22: Hostname: domU-12-31-39-00-EC-96  Jul 26 09:11:22: --------------------------------------------------------- Jul 26 09:11:22: CPU: Dual-Core AMD Opteron(tm) Processor 2218 HE  Jul 26 09:11:22: Starting 2 parallel workers. Jul 26 09:11:22: --------------------------------------------------------- Jul 26 13:18:18: 70% complete. Processed 10199 of 14400. Failed 158/14400 Jul 26 13:16:33: ETA: Thu Jul 26 13:11:23 UTC 2012Output of each command executed on each item is logged in a single file named after the item. The log file is available at location - './ppss\_dir/job\_log/'. Command execution status for the item can be obtained from log file and it displays as- "Status: FAILURE / Status: Success "

For more information on ppss, follow the links below.

 https://code.google.com/p/ppss/ https://code.google.com/p/ppss/downloads/detail?name=ppss-2.85.tgz