Monday, April 18, 2011

Deleting files from a directory.

On a linux shell deleting files is quite straightforward as it is expected just do rm fileName..

And if you have group of files which has a particular pattern and if you want to delete them, you do rm * < pattern >. But what do you do if you want to retain files with a particular patter and remove all other files in that directory.. This happens to me when I run some code for my work. My folder gets completely loaded with lot of log and tmp files which are produced by the main program. So how do I remove them ?

On a zshell its quite easy.. Say if you want to retain the files starting with the name mypetfile.

just do rm ^mypetfile*..

all the files which are not starting with the mypetfile is gone !!!

Parallel processing on bash shell

Straight to the idea:
suppose you are working on a machine which has some 16 cores. And you have a heavy program which generates particle events and outputs to a file. You can make the whole thing parallel process on the shell.

Here we go.
The program name toyMC
the program is run with an argumen n_events. The program reads this number and generates "n_evetns" number of MonteCarlo evetns and writes it into and output file evGen.txt

How do we parallelize this

Say we run 100 parallel instances of this program:

first creat 100 directories and then "cd" to these directories and run the code.

It is safe to do it this way.. otherwise multiple instances of the program might access the file and it will be a great mess.

The Full command on bash:

for i in `seq 1 1 100; do (mkdir job${i}; cd job{$i}; ../toMC 100 & ); done

thats it...

warning: the macro is not tested.