suppose you are working on a machine which has some 16 cores. And you have a heavy program which generates particle events and outputs to a file. You can make the whole thing parallel process on the shell.
Here we go.
The program name toyMC
the program is run with an argumen n_events. The program reads this number and generates "n_evetns" number of MonteCarlo evetns and writes it into and output file evGen.txt
How do we parallelize this
Say we run 100 parallel instances of this program:
first creat 100 directories and then "cd" to these directories and run the code.
It is safe to do it this way.. otherwise multiple instances of the program might access the file and it will be a great mess.
The Full command on bash:
for i in `seq 1 1 100; do (mkdir job${i}; cd job{$i}; ../toMC 100 & ); done
thats it...
warning: the macro is not tested.
Say you only want to run one job per CPU but still have 100 jobs. With GNU Parallel you can do:
ReplyDeleteseq 100 | parallel mkdir job{}\; cd job{}\; ../toMC 100
Watch the intro video to learn more: http://www.youtube.com/watch?v=OpaiGYxkSuQ
hi Tange
ReplyDeleteThanks for the suggestion. GNU parallel is something that I wasn't aware of.