Server update/upgrade

Message boards : Number crunching : Server update/upgrade

To post messages, you must log in.

1 · 2 · 3 · 4 · Next

AuthorMessage
Profile [VENETO] boboviz

Send message
Joined: 1 Dec 05
Posts: 1686
Credit: 6,629,809
RAC: 98
Message 79091 - Posted: 20 Nov 2015, 8:48:59 UTC
Last modified: 20 Nov 2015, 8:53:45 UTC

Rjs5 said:
The applications seem to be built on Red Hat RHEL4 which is not too old and still in corporate use


1)Yesterday RH released Red Hat Enterprise Linux 7.2, with a LOT of new features and resolved bugs. Here the release notes.

2)Here how to upgrade boinc server code, with latest changes. Why not to try it on Ralph, that is unused for most of the time?
(P.S. I'm tired of problems of this forum)

3)Latest version of GCC is 5.2, released July 16, 2015.

4)Latest version of Visual Studio is 2015 (with SP1 CPT/RC, near to stable release and already available for download and test).

I know, every single update of the list involves some works and may be not easy. But i think that 2016 may be the year for a big jump of project.
ID: 79091 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
rjs5

Send message
Joined: 22 Nov 10
Posts: 272
Credit: 19,755,483
RAC: 682
Message 79104 - Posted: 21 Nov 2015, 21:31:31 UTC - in response to Message 79091.  

Rjs5 said:
The applications seem to be built on Red Hat RHEL4 which is not too old and still in corporate use


1)Yesterday RH released Red Hat Enterprise Linux 7.2, with a LOT of new features and resolved bugs. Here the release notes.

2)Here how to upgrade boinc server code, with latest changes. Why not to try it on Ralph, that is unused for most of the time?
(P.S. I'm tired of problems of this forum)

3)Latest version of GCC is 5.2, released July 16, 2015.

4)Latest version of Visual Studio is 2015 (with SP1 CPT/RC, near to stable release and already available for download and test).

I know, every single update of the list involves some works and may be not easy. But i think that 2016 may be the year for a big jump of project.


Why would Rosetta do any preliminary testing on RHEL7.2 ? Just curious.

I doubt the distributions make any difference in Rosetta. The Rosetta app for Linux is 64-bit v3.67 and windows is 32-bit v3.65. They are essentially the same. The versions differ because they fixed the dynamic library problem with Linux.

Testing a new release is not really that hard. For me, I just CLONE my VirtualBox GUEST environment (with all the Boinc sources) on my main Windows machine and install the new release. It takes about an hour. With my 5930x, I could probably test 5 or so Linux distributions simultaneously.

One machine with enough memory and some extra disk space is all you need.
8-)


ID: 79104 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile [VENETO] boboviz

Send message
Joined: 1 Dec 05
Posts: 1686
Credit: 6,629,809
RAC: 98
Message 79107 - Posted: 23 Nov 2015, 9:28:46 UTC - in response to Message 79104.  
Last modified: 23 Nov 2015, 9:29:13 UTC

Why would Rosetta do any preliminary testing on RHEL7.2 ? Just curious.

'Cause they said that Rosetta server is "hard-customized" and it's difficult to update.

I doubt the distributions make any difference in Rosetta.

I know, but i don't know if, for example, they make security updates on their servers. New distro is bugfix, new tools, better security, better performance, etc. At the bottom of this thread there is: "Last Modified: 3 Dec 2007 20:36:17 UTC".
I think it's time for upgrade

Testing a new release is not really that hard. For me, I just CLONE my VirtualBox GUEST environment (with all the Boinc sources) on my main Windows machine and install the new release.

They have Ralph. Use it.
ID: 79107 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
rjs5

Send message
Joined: 22 Nov 10
Posts: 272
Credit: 19,755,483
RAC: 682
Message 79108 - Posted: 24 Nov 2015, 16:09:20 UTC - in response to Message 79107.  

Why would Rosetta do any preliminary testing on RHEL7.2 ? Just curious.

'Cause they said that Rosetta server is "hard-customized" and it's difficult to update.

I doubt the distributions make any difference in Rosetta.

I know, but i don't know if, for example, they make security updates on their servers. New distro is bugfix, new tools, better security, better performance, etc. At the bottom of this thread there is: "Last Modified: 3 Dec 2007 20:36:17 UTC".
I think it's time for upgrade

Testing a new release is not really that hard. For me, I just CLONE my VirtualBox GUEST environment (with all the Boinc sources) on my main Windows machine and install the new release.

They have Ralph. Use it.


Sorry. I was thinking about "qualifying" the 7.2 "execution" environment. Making sure that Rosetta ran properly on 7.2 and not what you mean ... moving their internal build and operational infrastructure forward.








ID: 79108 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile [VENETO] boboviz

Send message
Joined: 1 Dec 05
Posts: 1686
Credit: 6,629,809
RAC: 98
Message 79113 - Posted: 26 Nov 2015, 13:36:16 UTC
Last modified: 26 Nov 2015, 13:37:13 UTC

I beg your, rjs5, pardon if i have misspoken (i'm italian).
I think that the list in my first post is not in the correct order (i've insert it casually). In order of importance (IMHO):
1) Update compilers, tools, design software, etc, to have a better way to develop code (and, eventually, optimize it). Good code, good science.
2) Update the Boinc server code, to have a better web interface for volunteers (and better scheduler, validator and so on).
3) Update the server, for better security, bugfix, etc.
ID: 79113 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
rjs5

Send message
Joined: 22 Nov 10
Posts: 272
Credit: 19,755,483
RAC: 682
Message 79119 - Posted: 27 Nov 2015, 22:25:07 UTC - in response to Message 79113.  

I beg your, rjs5, pardon if i have misspoken (i'm italian).
I think that the list in my first post is not in the correct order (i've insert it casually). In order of importance (IMHO):
1) Update compilers, tools, design software, etc, to have a better way to develop code (and, eventually, optimize it). Good code, good science.
2) Update the Boinc server code, to have a better web interface for volunteers (and better scheduler, validator and so on).
3) Update the server, for better security, bugfix, etc.


Thanks for the clarification. I assumed it was my reading problem. I appreciate you taking the time to clear up my confusion. At first I thought you were talking about only 1) but another reading and comments made me think you were talking about 2) and 3).

I have been working some with RHEL7.2 and I have had some problems with it. RHEL7.2 seems especially sensitive when trying to operate in graphical (not command line) mode. I am not sure that Rosetta will see the same problems on their machines if they upgrade, but they have been very unlucky when trying new stuff. 8-)

thanks again.


ID: 79119 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile [VENETO] boboviz

Send message
Joined: 1 Dec 05
Posts: 1686
Credit: 6,629,809
RAC: 98
Message 79122 - Posted: 28 Nov 2015, 9:52:07 UTC - in response to Message 79119.  

thanks again.


Thanks to you. Is thank to your posts this summer that the "optimization wave" is started. We expect results :-)

ID: 79122 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
rjs5

Send message
Joined: 22 Nov 10
Posts: 272
Credit: 19,755,483
RAC: 682
Message 79127 - Posted: 29 Nov 2015, 2:38:49 UTC - in response to Message 79122.  

thanks again.


Thanks to you. Is thank to your posts this summer that the "optimization wave" is started. We expect results :-)

ID: 79127 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
rjs5

Send message
Joined: 22 Nov 10
Posts: 272
Credit: 19,755,483
RAC: 682
Message 79128 - Posted: 29 Nov 2015, 3:05:30 UTC - in response to Message 79122.  

thanks again.


Thanks to you. Is thank to your posts this summer that the "optimization wave" is started. We expect results :-)



If you would like to see changes that were made during DENIS optimization, then the small DENIS source code would be an interesting example. The execution time was reduced from over 1 hour (default) to 10 minutes (optimized) on my 5930x.

The overview of the project shows the initial release plus the changes to optimize the code by Sesef.

https://github.com/DENISproject/denis-boinc-baseapp/network


The actual DENIS changes are outlined at the URL below. Performance improvement came in the form of:
1. compile time options, especially "whole-program".
2. CODE MODIFICATION: constant term factoring ... pulling out the repeatedly calculated values and assigning them to a single "tmpX" variable that is computed once and then just reused during the task.
3. CODE MODIFICATION: You will see the EXPLICIT use of INTRINSIC statements (_mm_*, _mm256_*, _mm512_*, ...) define the SIZE of the registers. The "pd" at the end of the instruction sequence below means PACKED DOUBLE .... 2 operations per instruction. In ROSETTA you would see "sd" for SCALAR double or 1 operation per instruction.

stmp5 = _mm_mul_pd(stmp3, l1);
stmp5 = _mm_sub_pd(stmp5, _mm_loaddup_pd(&constants[44]));
stmp5 = _mm_div_pd(stmp5, _mm_sub_pd(stmp3, ones));
stmp5 = _mm_mul_pd(stmp5, state0);
stmp5 = _mm_mul_pd(stmp5, _mm_loadu_pd(&data[64]));
stmp5 = _mm_mul_pd(stmp5, gtmp1);
stmp5 = _mm_mul_pd(stmp5, _mm_sub_pd(ones, _mm_loadu_pd(&states[17])));


https://github.com/sesef/denis-boinc-baseapp/commit/359b68d1eefb2b39f8a2b79e60c11dda8fdf7e66



ID: 79128 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile [VENETO] boboviz

Send message
Joined: 1 Dec 05
Posts: 1686
Credit: 6,629,809
RAC: 98
Message 79148 - Posted: 2 Dec 2015, 9:08:17 UTC - in response to Message 79128.  
Last modified: 2 Dec 2015, 9:17:07 UTC

If you would like to see changes that were made during DENIS optimization, then the small DENIS source code would be an interesting example. The execution time was reduced from over 1 hour (default) to 10 minutes (optimized) on my 5930x.


But Denis has only one kind of simulation (up to now).
Rosetta@home runs, over Rosetta framework, a lot of different simulations.
So, the problem is to "strike a balance" between science and optimizations.
6x acceleration is a dream, i think 20% it's a big gain!!

It is certainly strange that Rosetta's admins have not thought, until now, to use all the potential of our cpus
ID: 79148 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile [VENETO] boboviz

Send message
Joined: 1 Dec 05
Posts: 1686
Credit: 6,629,809
RAC: 98
Message 79258 - Posted: 20 Dec 2015, 14:17:14 UTC

This is an old post from admis, October 2014.
I heard we will get some funding for hardware upgrades. Not sure about the timeline but once we get the hardware, we'll look into the software and coordinate the update.


2016 is here.....
ID: 79258 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile [VENETO] boboviz

Send message
Joined: 1 Dec 05
Posts: 1686
Credit: 6,629,809
RAC: 98
Message 79288 - Posted: 23 Dec 2015, 14:37:47 UTC

Little OT.
Does anyone has tried to use this?
https://code.visualstudio.com/
ID: 79288 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Chilean
Avatar

Send message
Joined: 16 Oct 05
Posts: 711
Credit: 26,669,447
RAC: 0
Message 79291 - Posted: 23 Dec 2015, 16:14:00 UTC

Yeah. I don't think anything is going to happen soon at this rate. It's sad.
DENNIS remade their website, and they are a very small team.
ID: 79291 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Dr. Merkwürdigliebe
Avatar

Send message
Joined: 5 Dec 10
Posts: 81
Credit: 2,657,273
RAC: 0
Message 79292 - Posted: 23 Dec 2015, 17:31:38 UTC - in response to Message 79291.  

Yeah. I don't think anything is going to happen soon at this rate. It's sad.
DENNIS remade their website, and they are a very small team.


They're probably too busy deleting my posts but on a serious note:

rjs5 said he's looking into the source code right now, maybe something will come up.

Why they are not providing a binary compiled with a recent version of gcc is beyond me. I would get it if the last optimized binary would have yield false results but the only problem seemed to be that ancient CPUs choked on it.

Probably because Professor Ray would have to retire his 386SX.
ID: 79292 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
sgaboinc

Send message
Joined: 2 Apr 14
Posts: 282
Credit: 208,966
RAC: 0
Message 79293 - Posted: 23 Dec 2015, 18:42:49 UTC
Last modified: 23 Dec 2015, 18:54:11 UTC

actually come to think about it, the number of different or different types of X86 or even X86_64 cpu is simply *staggering* & so many of them have different *features* especially those 'esoteric' newer CPU features :o :p lol

and you don't really need to look further than the most recent state-of-the-art processors, even the most recent families has distinct and different cpu features between them all :o :p lol
http://ark.intel.com/
http://www.amd.com/en-us/products/processors

for all that history, there are probably more mutations of X86 or even X86_64 cpus than there are different mutations of dna :o :p lol
ID: 79293 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile [VENETO] boboviz

Send message
Joined: 1 Dec 05
Posts: 1686
Credit: 6,629,809
RAC: 98
Message 79296 - Posted: 23 Dec 2015, 19:41:08 UTC - in response to Message 79292.  
Last modified: 23 Dec 2015, 20:08:40 UTC

Why they are not providing a binary compiled with a recent version of gcc is beyond me. I would get it if the last optimized binary would have yield false results but the only problem seemed to be that ancient CPUs choked on it.


I remember problems with linux app.....
Ancient cpu is not a problem. Boinc client reads the info about cpu: my wife notebook, for example, is
| Processor: 2 GenuineIntel Pentium(R) Dual-Core CPU T4400 @ 2.20GHz [Family 6 Model 23 Stepping 10]
| Processor features: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss htt tm pni ssse3 cx16 nx lm tm2 pbe)

and send these infomations to server (server scheduler, at the end, send "correct" wus to cpu). You can also force it.
ID: 79296 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile [VENETO] boboviz

Send message
Joined: 1 Dec 05
Posts: 1686
Credit: 6,629,809
RAC: 98
Message 79464 - Posted: 1 Feb 2016, 20:06:57 UTC - in response to Message 79258.  

This is an old post from admis, October 2014.
I heard we will get some funding for hardware upgrades. Not sure about the timeline but once we get the hardware, we'll look into the software and coordinate the update.


2016 is here.....


Is really here.
And no news about server update (nor optimizations).

ID: 79464 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile David E K
Volunteer moderator
Project administrator
Project developer
Project scientist

Send message
Joined: 1 Jul 05
Posts: 1018
Credit: 4,334,829
RAC: 0
Message 79468 - Posted: 2 Feb 2016, 18:46:25 UTC - in response to Message 79464.  

This is an old post from admis, October 2014.
I heard we will get some funding for hardware upgrades. Not sure about the timeline but once we get the hardware, we'll look into the software and coordinate the update.


2016 is here.....


Is really here.
And no news about server update (nor optimizations).


No update on optimizations but rjs5 has been quite busy working on it.

The server will not be upgraded anytime soon as far as I know. We are currently gearing up for the next CASP.
ID: 79468 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Chilean
Avatar

Send message
Joined: 16 Oct 05
Posts: 711
Credit: 26,669,447
RAC: 0
Message 79508 - Posted: 10 Feb 2016, 15:20:21 UTC - in response to Message 79468.  

This is an old post from admis, October 2014.
I heard we will get some funding for hardware upgrades. Not sure about the timeline but once we get the hardware, we'll look into the software and coordinate the update.


2016 is here.....


Is really here.
And no news about server update (nor optimizations).


No update on optimizations but rjs5 has been quite busy working on it.

The server will not be upgraded anytime soon as far as I know. We are currently gearing up for the next CASP.


When is CASP going to start exactly? (if there's an exact date at all...)

ID: 79508 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile [VENETO] boboviz

Send message
Joined: 1 Dec 05
Posts: 1686
Credit: 6,629,809
RAC: 98
Message 79509 - Posted: 10 Feb 2016, 17:26:12 UTC - in response to Message 79508.  

When is CASP going to start exactly? (if there's an exact date at all...)


From Predictioncenter site:
"CASP12 will be held in May-August 2016."

ID: 79509 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
1 · 2 · 3 · 4 · Next

Message boards : Number crunching : Server update/upgrade



©2023 University of Washington
https://www.bakerlab.org