Posts by Piotr Skrodzewicz

1) Message boards : Number crunching : Rosetta CPU optimization - how ? (Message 47738) Posted 14 Oct 2007 by Piotr Skrodzewicz Post: SSE4.1 is comming !!!!!!!!! BUT Rosetta still use SSE1 OMG!!!! IIRC Rosetta does not use even SSE !
2) Message boards : Number crunching : Rosetta CPU optimization - how ? (Message 45286) Posted 23 Aug 2007 by Piotr Skrodzewicz Post: Is the deal between Rosetta and Intel going to put AMD machines at a disadvantage, or maybe even phase out AMD machines in the future? Hmm, using ICC10 should speed up computations on both Intel and AMG CPUs. Althrough ICC is known to have some anti-amd rountines (causes program to run not at full speed on AMD). But 1. Maybe in version 10 this problem does not occur 2. There are patches that cause programs to run on full speed on AMD CPUs. Anyway ICC is worth trying ! Peter
3) Message boards : Number crunching : Rosetta CPU optimization - how ? (Message 44932) Posted 13 Aug 2007 by Piotr Skrodzewicz Post: Hey, there are free and academic (much lower cost) licenses: http://www.intel.com/cd/software/products/asmo-na/eng/219771.htm http://www.intel.com/cd/software/products/asmo-na/eng/219603.htm#academic This compiler IS THE fastest one. I really can't understand why most of/all boinc projects don't use ICC - benefits are clear - more processing power, more satisfaction of users (faster credits, more units computed etc.). EDIT: Guys, I found a person who will compile Rosetta with SSE, SSE2, SSE3, SSSE3. Now I must "only" obtain source code ;) Non-commercial version of ICC will be used.
4) Message boards : Number crunching : Rosetta CPU optimization - how ? (Message 44911) Posted 12 Aug 2007 by Piotr Skrodzewicz Post: waiting.... for ... Rosetta will be compiled with new Intel Compiler 10.0(599$) Yes, I know, it IS expensive. But I see at least 4 solutions: 1) Buy compiler. Currently project has much funds IMHO (terabytes of transfer, giga(tera?)bytes of HD, HQ servers etc.). Also, CONSIDER DONATIONS WITH DONOR LIST. 2) Find someone with compiler and ask for compilation. 3) Use pirated/warez version - this option is of course out of consideration 4) Ask Intel for free version. In exchange there will be LOGO of Intel and/or some text on rosetta page. Someone could ask: is it worth cash and effort ? Heh, yes. For example SIMAP after release of a compiled version ran out of workunits - participants calculated too fast.
5) Message boards : Number crunching : Rosetta CPU optimization - how ? (Message 44842) Posted 9 Aug 2007 by Piotr Skrodzewicz Post: There is ICC 10.0. How about compile rosetta with SSE1/SSE2 ? Favor fast code. First test version could be available on web site (not downloaded automagically), so test period will be safe. Anything new about this topic ?
6) Message boards : Number crunching : 5.10.13 (Message 44609) Posted 2 Aug 2007 by Piotr Skrodzewicz Post: BOINC 5.10.13 has an option, to enable crunching only when user is inactive. Following option caused many VALIDATE ERROR with Rosetta for me. Please confirm or deny that.
7) Message boards : Number crunching : UPX (Message 44608) Posted 2 Aug 2007 by Piotr Skrodzewicz Post: UPX 3.01 FINAL released: http://upx.sourceforge.net/ I suggest trying following commandline: upx.exe --lzma --best --ultra-brute FILE.exe I achieved 0.69% gain vs previous wersion. Plus 3.01 is a FINAL and STABLE version, since 2.92 was a beta IIRC. 0.69% is not much ? It is 17920 bytes for 5.72. 17920*376584[hosts]=6748385280 bytes=6590220 megabytes=6435 gigbytes~6.28 terabytes bandwidth save !!! 1 kilobyte = 1024 bytes
8) Message boards : Number crunching : Save ~50% of bandwidth :) Switching from gzip to PPMD (Message 38012) Posted 19 Mar 2007 by Piotr Skrodzewicz Post: -- Just remember.. it should be something that all Rosetta O.S. supports have in the O.S. by default... or be something that gets downloaded with the Rosetta binary... Downloading would create a bunch of headaches with supporting multiple version of GLIBC on linux... Not downloading would require linuces to download and install additional software in order to run Rosetta (would likely have chilling effect on the number of Linux crunchers)... Would probably be best to leave well enough alone.... IMO, you are wrong. Just implement LZMA using http://www.7-zip.org/sdk.html into rosetta executable. EXE will handle compression. No additional software needed. Renderfarm@Home uses LZMA...
9) Message boards : Number crunching : Save ~50% of bandwidth :) Switching from gzip to PPMD (Message 37977) Posted 18 Mar 2007 by Piotr Skrodzewicz Post: Normally LZMA has better compression than PPMD. PPMD is good for text files, LZMA for binaries. I think there's only a LZMA SDK, no PPMD SDK. You can find it here. Anyway, that big file bbdep02.May.sortlib is only donwloaded once, so those 2~3 MB won't really matter. ;) bbdep02.May.7z - 3599799 bytes (3.5mb/sec compression speed - Core Duo 2Ghz, PPMD 16mb dictionary size). bbdep02.May.rk - 3030160 bytes (8 min compression/10 sec decompression - P3 1.5 MHz). - YEAH, RK even beats PPMD !!! 1. All project files (exept exes) are in fact text files, so compression format change will still give ~50% gain. 2. There is no PPMD SDK AFAIK, but how about implementing PPMD w/o SDK ? 3. If PPMD is too big problem, try LZMA. 4. RK is AFAIK closed source. EDIT: bbdep02.May.sortlib.bz2 - 3688826 bytes - but bzip2 is slow... bbdep02.May.sortlib.7z - 2949796 bytes !!! - 7-ZIP LZMA 16mb dictionary. Forget RK, forget Winrar, forget PPMD - use 7-zip LZMA !!! Both Winrar and 7-zip use LZMA, so I thought that 7-zip will not be noticeably better than Winrar... I was wrong. 7-ZIP LZMA destroys all competition. So, don't wait, switch to LZMA ASAP !!!
10) Message boards : Number crunching : Save ~50% of bandwidth :) Switching from gzip to PPMD (Message 37906) Posted 17 Mar 2007 by Piotr Skrodzewicz Post:
11) Message boards : Number crunching : UPX (Message 36829) Posted 15 Feb 2007 by Piotr Skrodzewicz Post: We'll switch to 2.92 and use maximum compression for the next release. Thanks! Rosetta 5.46 released... and UPX 2.91 used ! Probably with old packer options too ! 3206144-2208256=997888 997888[gain from compression]*267373[hosts]=266808308224 bytes 266808308224/1024=260554988.5 kilobytes =254448.230 megabytes =248.484 gigabytes WASTED
12) Message boards : Number crunching : UPX (Message 36818) Posted 15 Feb 2007 by Piotr Skrodzewicz Post: We'll switch to 2.92 and use maximum compression for the next release. Thanks! Rosetta 5.46 released... and UPX 2.91 used ! Probably with old packer options too !
13) Message boards : Number crunching : UPX (Message 36072) Posted 3 Feb 2007 by Piotr Skrodzewicz Post: However, it is best to use "--lzma --best --exact" to get a byte-identical file after decompression with option -d. There is no loss in compression ratio in this case. I also checked compression ratio: Original "rosetta_5.45_windows_intelx86.exe" - 3206144 bytes. Repacked one - first with decompress (-d) then packed (with "--lzma --best --exact") - 2208256 bytes.
14) Message boards : Number crunching : Rosetta CPU optimization - how ? (Message 35829) Posted 31 Jan 2007 by Piotr Skrodzewicz Post: Problem with ICC is it cost money for a license.... Another problem with SSE3 is not all CPU's use it and currently Windows does not correctly report it in the kernel (VISTA does, XP doesn't) so it is harder to flag up which computer would be able to support it. So maybe I ask someone with legal ICC9.1 to compile SSE2 version for windows ? Where is a sourcecode of Rosetta ? EDIT: I just realised, that Rosetta is a closed source app. So, I hope that Rosetta developers will release optimized app.
15) Message boards : Number crunching : Rosetta CPU optimization - how ? (Message 35822) Posted 31 Jan 2007 by Piotr Skrodzewicz Post: As far as i know rosetta doesn't use sse at all, that's why no sse2/3 optimizations for the application are being develeoped. I might be wrong though. Well, I suggest trying SSE3 ICC9.1 compilation.
16) Message boards : Number crunching : Rosetta CPU optimization - how ? (Message 35818) Posted 31 Jan 2007 by Piotr Skrodzewicz Post: AFAIK Rosetta utilises SSE. 1. Which version of SSE is used ? If SSE1 only, is there any chance for SSE2/3 ? 2 How it is optimized ? Just special flag in a compiler ? 3. Which compiler is used. AFAIK ICC (known also as ICL) 9.1 is the fastest one.
17) Message boards : Number crunching : UPX (Message 35817) Posted 31 Jan 2007 by Piotr Skrodzewicz Post: AFAIK --lzma does not have additional options aimed for increasing compression. Decompression is fast. Does anyone have not compressed (not: unpacked), original version of the latest Rosetta ?
18) Message boards : Number crunching : UPX (Message 35771) Posted 30 Jan 2007 by Piotr Skrodzewicz Post: I compressed the current 5.45 release with the new UPX version 2.92 and maximum compression. The new executable has now 2,10 MB, that's 23% of the original! Maximum compression ? You mean "upx.exe --lzma rosetta_5.45_windows_intelx86.exe" or "upx.exe --best rosetta_5.45_windows_intelx86.exe" ? "--lzma" option is better than "--best".
19) Message boards : Number crunching : UPX (Message 35755) Posted 30 Jan 2007 by Piotr Skrodzewicz Post: 1. There is newer version of UPX - 2.92 (Rosetta 5.45 uses 2.91). 2. I suggest trying LZMA compression if not used already.