Page 1 of 1

-j multi-threaded compression

Posted: Sat Jun 19, 2010 12:20 am
by Coastie
If you have a 2 core cpu (and plenty of ram) that can process 4 threads, do you use the -j option for cores, or threads?

-j2 (cores)
0r
-j4 (threads)

Re: -j multi-threaded compression

Posted: Mon Jun 21, 2010 12:20 pm
by admin
I guess you have a DualCore CPU with Hyper Threading. Then the system will see 4 logical CPUS (cat /proc/cpuinfo)
In that case it makes sense to use "-j4", but you can also prefer "-j3" if you want to leave one thread for the other applications so that your systems is more responsive during the backup.

Re: -j multi-threaded compression

Posted: Wed Jun 23, 2010 3:33 pm
by floor9
I have noticed that on our laptops, every compression level performs best at three threads. We currently use Core i5-520Ms (dual-core with hyperthreading) with 4GB of DDR3 on a 64-bit Ubuntu Server 10.04 console-only installation.

In my experience:

2 threads = approximately double the performance of a single thread
3 threads = significant improvement over 2 threads, but always well under a 50% gain
4 threads = occasionally faster than 3 threads by a few seconds, but almost always slower

This is compressing a single Windows 7 partition that has not been prepared at all. Our final .FSA size is around 11 GB. Note that we operating somewhat unusually in that we are archiving one partition onto another on the same drive (our goal is to allow the user to perform a "99% hands-off" reimage of their Windows partition). This obviously generates a negative hit on our performance, but with verbosity off, we're still usually under 15 minutes (and never over 20) to decompress.

Re: -j multi-threaded compression

Posted: Sat Jul 03, 2010 12:12 am
by Coastie
floor9 wrote:Note that we operating somewhat unusually in that we are archiving one partition onto another on the same drive (our goal is to allow the user to perform a "99% hands-off" reimage of their Windows partition).
That is what I use it for as well. Disaster recovery on the road. Thanks for the core info.