UPX

Message boards : Number crunching : UPX

To post messages, you must log in.

AuthorMessage
Piotr Skrodzewicz

Send message
Joined: 21 Oct 06
Posts: 19
Credit: 273,234
RAC: 0
Message 35755 - Posted: 30 Jan 2007, 15:15:32 UTC

1. There is newer version of UPX - 2.92 (Rosetta 5.45 uses 2.91).
2. I suggest trying LZMA compression if not used already.
ID: 35755 · Rating: 2 · rate: Rate + / Rate - Report as offensive    Reply Quote
Christoph

Send message
Joined: 10 Dec 05
Posts: 57
Credit: 1,512,386
RAC: 0
Message 35764 - Posted: 30 Jan 2007, 19:25:16 UTC

I compressed the current 5.45 release with the new UPX version 2.92 and maximum compression. The new executable has now 2,10 MB, that's 23% of the original!
ID: 35764 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile David E K
Volunteer moderator
Project administrator
Project developer
Project scientist

Send message
Joined: 1 Jul 05
Posts: 1018
Credit: 4,334,829
RAC: 0
Message 35765 - Posted: 30 Jan 2007, 19:47:45 UTC

We'll switch to 2.92 and use maximum compression for the next release. Thanks!
ID: 35765 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile David E K
Volunteer moderator
Project administrator
Project developer
Project scientist

Send message
Joined: 1 Jul 05
Posts: 1018
Credit: 4,334,829
RAC: 0
Message 35769 - Posted: 30 Jan 2007, 20:06:47 UTC

Has anyone had luck compressing Mac ppc and intel executables?
ID: 35769 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Piotr Skrodzewicz

Send message
Joined: 21 Oct 06
Posts: 19
Credit: 273,234
RAC: 0
Message 35771 - Posted: 30 Jan 2007, 20:18:14 UTC - in response to Message 35764.  

I compressed the current 5.45 release with the new UPX version 2.92 and maximum compression. The new executable has now 2,10 MB, that's 23% of the original!


Maximum compression ?

You mean "upx.exe --lzma rosetta_5.45_windows_intelx86.exe" or "upx.exe --best rosetta_5.45_windows_intelx86.exe" ?


"--lzma" option is better than "--best".
ID: 35771 · Rating: 1 · rate: Rate + / Rate - Report as offensive    Reply Quote
Christoph

Send message
Joined: 10 Dec 05
Posts: 57
Credit: 1,512,386
RAC: 0
Message 35801 - Posted: 31 Jan 2007, 13:18:03 UTC - in response to Message 35771.  

I compressed the current 5.45 release with the new UPX version 2.92 and maximum compression. The new executable has now 2,10 MB, that's 23% of the original!


Maximum compression ?

You mean "upx.exe --lzma rosetta_5.45_windows_intelx86.exe" or "upx.exe --best rosetta_5.45_windows_intelx86.exe" ?


"--lzma" option is better than "--best".

I used this command: upx.exe -9 --all-methods --all-filters
It took quite long, and I think --ultra-brute would take even longer.


ID: 35801 · Rating: 1 · rate: Rate + / Rate - Report as offensive    Reply Quote
Christoph

Send message
Joined: 10 Dec 05
Posts: 57
Credit: 1,512,386
RAC: 0
Message 35805 - Posted: 31 Jan 2007, 14:40:13 UTC

I made a few more tests, and the --lzma method, as Piotr already said, is one of the fastest and best compression options. There are a few more options with even better compression, but most of them take more than half an hour to complete.
ID: 35805 · Rating: 1 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Feet1st
Avatar

Send message
Joined: 30 Dec 05
Posts: 1755
Credit: 4,690,520
RAC: 0
Message 35810 - Posted: 31 Jan 2007, 15:51:31 UTC

...a half hour to compress, but, just curious, how long to DEcompress? That is the part the client PCs would be doing. Decompress is often faster. If it only takes 5min to decompress, it might still be worth doing.

And how did the sizes compare?
Add this signature to your EMail:
Running Microsoft's "System Idle Process" will never help cure cancer, AIDS nor Alzheimer's. But running Rosetta@home just might!
https://boinc.bakerlab.org/rosetta/
ID: 35810 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Piotr Skrodzewicz

Send message
Joined: 21 Oct 06
Posts: 19
Credit: 273,234
RAC: 0
Message 35817 - Posted: 31 Jan 2007, 16:38:50 UTC

AFAIK --lzma does not have additional options aimed for increasing compression.
Decompression is fast. Does anyone have not compressed (not: unpacked), original version of the latest Rosetta ?
ID: 35817 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Patrick May

Send message
Joined: 1 Feb 06
Posts: 2
Credit: 15,373
RAC: 0
Message 35943 - Posted: 2 Feb 2007, 0:39:06 UTC - in response to Message 35771.  

I compressed the current 5.45 release with the new UPX version 2.92 and maximum compression. The new executable has now 2,10 MB, that's 23% of the original!


Maximum compression ?

You mean "upx.exe --lzma rosetta_5.45_windows_intelx86.exe" or "upx.exe --best rosetta_5.45_windows_intelx86.exe" ?


"--lzma" option is better than "--best".


Thats not true. If you only use "--lzma", LZMA/7 will be applied. But if you use "--lzma --best", LZMA/10 will be applied. Slightly better Ratio 22.64%. And it ist still faster than only using "--best".

If you only use "--best", the old NRV2E/10 will be applied. Slower and the Ratio is only 27.09% on the rosetta_5.45_windows_intelx86.exe.

Using "--lzma --brute" or "--lzma --ultra-brute" is not worth. You have very much slower compression (depending on file) and often no better ratio.

However, it is best to use "--lzma --best --exact" to get a byte-identical file after decompression with option -d. There is no loss in compression ratio in this case.



ID: 35943 · Rating: 1 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Feet1st
Avatar

Send message
Joined: 30 Dec 05
Posts: 1755
Credit: 4,690,520
RAC: 0
Message 35945 - Posted: 2 Feb 2007, 0:58:03 UTC

My point was just that in the big scheme of things, the compress is only done once... by BakerLab. And if the difference in the time to decompress is smaller then the difference in the download time for a 56k modem user, then perhaps it would be worth using a method that has a long compress time if it saves a few more minutes of download time.

It's the decompress time that is most important. That is what is being done tens of thousands of times.
Add this signature to your EMail:
Running Microsoft's "System Idle Process" will never help cure cancer, AIDS nor Alzheimer's. But running Rosetta@home just might!
https://boinc.bakerlab.org/rosetta/
ID: 35945 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Patrick May

Send message
Joined: 1 Feb 06
Posts: 2
Credit: 15,373
RAC: 0
Message 35987 - Posted: 2 Feb 2007, 3:45:50 UTC
Last modified: 2 Feb 2007, 3:48:46 UTC

Okay, perhaps I was a little bit offtopic. Of course, BakerLab will do the compression of the app and send it out. Otherwise it would be senseless. But the decompression speed of UPX is very fast. According to http://upx.sourceforge.net/ the decompression is ~10 MB/sec on an ancient Pentium 133, ~200 MB/sec on an Athlon XP 2000+. Everyone with an 56k Modem will save much time.
Depending on the processor, you will perhaps lose one or maybe two seconds.




ID: 35987 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Christoph

Send message
Joined: 10 Dec 05
Posts: 57
Credit: 1,512,386
RAC: 0
Message 36002 - Posted: 2 Feb 2007, 15:19:04 UTC - in response to Message 35945.  

My point was just that in the big scheme of things, the compress is only done once... by BakerLab. And if the difference in the time to decompress is smaller then the difference in the download time for a 56k modem user, then perhaps it would be worth using a method that has a long compress time if it saves a few more minutes of download time.

It's the decompress time that is most important. That is what is being done tens of thousands of times.

The compression time would be 10 times higher, but the size 50 KB less. It is not worth the effort.
ID: 36002 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile dcdc

Send message
Joined: 3 Nov 05
Posts: 1829
Credit: 116,582,820
RAC: 77,078
Message 36003 - Posted: 2 Feb 2007, 15:22:42 UTC

surely the compression only needs to be done once per file at the bakerlab end?
ID: 36003 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Piotr Skrodzewicz

Send message
Joined: 21 Oct 06
Posts: 19
Credit: 273,234
RAC: 0
Message 36072 - Posted: 3 Feb 2007, 23:13:36 UTC - in response to Message 35943.  
Last modified: 3 Feb 2007, 23:14:14 UTC

However, it is best to use "--lzma --best --exact" to get a byte-identical file after decompression with option -d. There is no loss in compression ratio in this case.

I also checked compression ratio:
Original "rosetta_5.45_windows_intelx86.exe" - 3206144 bytes.
Repacked one - first with decompress (-d) then packed (with "--lzma --best --exact") - 2208256 bytes.
ID: 36072 · Rating: 1 · rate: Rate + / Rate - Report as offensive    Reply Quote
Piotr Skrodzewicz

Send message
Joined: 21 Oct 06
Posts: 19
Credit: 273,234
RAC: 0
Message 36818 - Posted: 15 Feb 2007, 8:16:18 UTC - in response to Message 35765.  

We'll switch to 2.92 and use maximum compression for the next release. Thanks!


Rosetta 5.46 released... and UPX 2.91 used ! Probably with old packer options too !
ID: 36818 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Piotr Skrodzewicz

Send message
Joined: 21 Oct 06
Posts: 19
Credit: 273,234
RAC: 0
Message 36829 - Posted: 15 Feb 2007, 9:21:12 UTC - in response to Message 36818.  

We'll switch to 2.92 and use maximum compression for the next release. Thanks!


Rosetta 5.46 released... and UPX 2.91 used ! Probably with old packer options too !


3206144-2208256=997888
997888[gain from compression]*267373[hosts]=266808308224 bytes
266808308224/1024=260554988.5 kilobytes
=254448.230 megabytes
=248.484 gigabytes

WASTED
ID: 36829 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Piotr Skrodzewicz

Send message
Joined: 21 Oct 06
Posts: 19
Credit: 273,234
RAC: 0
Message 44608 - Posted: 2 Aug 2007, 11:02:08 UTC

UPX 3.01 FINAL released:
http://upx.sourceforge.net/

I suggest trying following commandline:
upx.exe --lzma --best --ultra-brute FILE.exe

I achieved 0.69% gain vs previous wersion. Plus 3.01 is a FINAL and STABLE version, since 2.92 was a beta IIRC. 0.69% is not much ? It is 17920 bytes for 5.72.
17920*376584[hosts]=6748385280 bytes=6590220 megabytes=6435 gigbytes~6.28 terabytes bandwidth save !!!

1 kilobyte = 1024 bytes
ID: 44608 · Rating: 1 · rate: Rate + / Rate - Report as offensive    Reply Quote
Rhiju
Volunteer moderator

Send message
Joined: 8 Jan 06
Posts: 223
Credit: 3,546
RAC: 0
Message 44684 - Posted: 4 Aug 2007, 18:15:40 UTC

Hi -- this looks great, we'll use this compression for the next update (sorry I missed it for these last couple). It wil be nice to compress the Mac apps now!

ID: 44684 · Rating: 1 · rate: Rate + / Rate - Report as offensive    Reply Quote

Message boards : Number crunching : UPX



©2024 University of Washington
https://www.bakerlab.org