canonical result

Message boards : Number crunching : canonical result

To post messages, you must log in.

1 · 2 · 3 · Next

AuthorMessage
Profile FZB

Send message
Joined: 17 Sep 05
Posts: 84
Credit: 4,948,999
RAC: 0
Message 211 - Posted: 19 Sep 2005, 22:05:14 UTC

from what i see, a result get only sent to a single pc. this is fine as long as you had it running internaly, where you can be sure no "cheating" is done, but you might consider sending it to two+ pc's now that you are public. just a thought, not sure if it helps your needs or if you can easily tell from the results if they are "valid"
--
Florian
www.domplatz1.de
ID: 211 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile David E K
Volunteer moderator
Project administrator
Project developer
Project scientist

Send message
Joined: 1 Jul 05
Posts: 1018
Credit: 4,334,829
RAC: 0
Message 214 - Posted: 19 Sep 2005, 23:22:33 UTC - in response to Message 211.  
Last modified: 19 Sep 2005, 23:31:10 UTC

from what i see, a result get only sent to a single pc. this is fine as long as you had it running internaly, where you can be sure no "cheating" is done, but you might consider sending it to two+ pc's now that you are public. just a thought, not sure if it helps your needs or if you can easily tell from the results if they are "valid"


We will be able to tell if the results are valid. Right now, we need as much compute cycles as we can get. If there is cheating, we will find out and remove the cheaters from this project and I will let all other projects know also.

It will be unfortunate if we have to resort to redundancy.
ID: 214 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile [B^S] Paul@home
Avatar

Send message
Joined: 18 Sep 05
Posts: 34
Credit: 393,096
RAC: 0
Message 224 - Posted: 20 Sep 2005, 8:30:04 UTC

If you can validate the results returned based on 1 PC then I guess that is the way to go. No point duplicating work that is not required.

People could stll artificially bump up their claimed credits... even when returning perfectly valid results. I know its not a major issue since credits dont mean anything except for the bit of competition, but is it possible for you to watch for this sort of thing too?

ie get the system to flag any claimed credits that are above expectation?


cheers,

Paul.
Wanna visit BOINC Synergy team site? Click below!

Join BOINC Synergy Team
ID: 224 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
RDC

Send message
Joined: 16 Sep 05
Posts: 43
Credit: 101,644
RAC: 0
Message 226 - Posted: 20 Sep 2005, 9:19:34 UTC - in response to Message 224.  

ie get the system to flag any claimed credits that are above expectation?


One thing I worry about with something like that and that is what time is expected? I've done 2 WU's so far. One 36 hours long and one 19 hours long and both done while stalling on other things I need to do on my computer so as not to drag out the processing time. That's a pretty big gap between the two times and could easily be flagged as suspicious due to the huge difference between my two WU's. Plus, if I do anything with my PC such as play a game or do something with graphic programs, it has the unplanned problem of artificially inflating my CPU time during the time I am doing something other than dedicating the PC to BOINC. No, it's not cheating but it is a problem with how BOINC benchmarks and how the BOINC/project application uses and records the CPU cycles.

Credit, while lower in priority, is one reason out of many many projects do require multiple WU's be processed.
ID: 226 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile [B^S] Paul@home
Avatar

Send message
Joined: 18 Sep 05
Posts: 34
Credit: 393,096
RAC: 0
Message 230 - Posted: 20 Sep 2005, 11:59:16 UTC
Last modified: 20 Sep 2005, 12:00:58 UTC

I'm not sure that using your CPU for other processes will inflate your credit all that much. The claimed credit should be based on CPU time spent on a given work unit, not the elapsed time between starting and finishing the unit.

For example, if you are using 50% CPU for something other than the science app, the CPU time count will increase at about half the rate that it would if your CPU was giving 100% to the science app. This means that even though more time would have elapsed, the total CPU time will remain about the same and your claimed credit will remain the same.


I was thinking more along the lines of users fiddling benchmark scores... :(


I suppose the guys may be able to make an estimate of claimed credit if they can increase the accuracy of the expected run time. Any claime that falls more that x% outside that estimate could be flagged as suspicious by the validator, even if the result is valid.

It would take quite a bit of tweaking to get it right but it would help alleviate people’s worries...

Cheers,

Paul

Wanna visit BOINC Synergy team site? Click below!

Join BOINC Synergy Team
ID: 230 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
RDC

Send message
Joined: 16 Sep 05
Posts: 43
Credit: 101,644
RAC: 0
Message 246 - Posted: 20 Sep 2005, 19:17:46 UTC - in response to Message 230.  

I'm not sure that using your CPU for other processes will inflate your credit all that much. The claimed credit should be based on CPU time spent on a given work unit, not the elapsed time between starting and finishing the unit.



If I play a game like Mech Warrior III or Master of Orion without shutting BOINC down first, I can literally double to triple the time taken to process and double to triple the credit claimed on a WU. Not all games cause this behavior but the above ones do and a 2 hour stint in one of those games will cause the WU to claim 2 or more hours additional time and 2 or more hours additional credit. If I leave the game up longer, I can claim more credits with that WU. I try to remember to shut BOINC down first but I do forget at times. I shut it down because those games also greatly increase the risk the WU having a computational error and thereby wasting the time spend crunching it but at least half of the screwed up WU's are taken as valid WU's but the credits get tossed in the quorum validation process.

In short, the more you play certain games the more you can stretch out the time and credit. I have brought this up before to the BOINC programmers and nobody seemed to think it is a real problem but if I can double or triple my claimed credit without cheating but by simply playing a game for a few hours, there is a large potential problem. Maybe I'm not making myself clear on the issue, I don't know. The fact is that you can greatly inflate the WU credits without intending to, without any form of cheating and without touching one BOINC/Project file.
ID: 246 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile FZB

Send message
Joined: 17 Sep 05
Posts: 84
Credit: 4,948,999
RAC: 0
Message 248 - Posted: 20 Sep 2005, 20:34:17 UTC

this is definately strange, but i think boinc devs have little impact there. seems like windows is messing something up there. will try to recreate the situation here, have master of orion 3 at home. windows should count the time the process actually got, not the time that past by in real world.
what windows version do you see this with?
--
Florian
www.domplatz1.de
ID: 248 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
RDC

Send message
Joined: 16 Sep 05
Posts: 43
Credit: 101,644
RAC: 0
Message 250 - Posted: 20 Sep 2005, 20:54:23 UTC - in response to Message 248.  

this is definately strange, but i think boinc devs have little impact there. seems like windows is messing something up there. will try to recreate the situation here, have master of orion 3 at home. windows should count the time the process actually got, not the time that past by in real world.
what windows version do you see this with?


Win98 SE and WinXP Home Edition SP2. On both systems, BOINC seems to count elapsed time when a high CPU intensive game is run. I've had similar happen with high CPU intensive graphics programs as well but not to the extent that games have.
ID: 250 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
RDC

Send message
Joined: 16 Sep 05
Posts: 43
Credit: 101,644
RAC: 0
Message 251 - Posted: 20 Sep 2005, 21:03:10 UTC

Here's a couple of examples of what I mean on Predictor that show the doubling + of claimed credit:

With a game running:
http://predictor.scripps.edu/workunit.php?wuid=1388891 claimed 15.91 granted 6.85
Without a game running:
http://predictor.scripps.edu/workunit.php?wuid=1388889 claimed 8.62 granted 8.11


With a game running:
http://predictor.scripps.edu/workunit.php?wuid=1313326 claimed 18.14 granted 6.66
Without a game running:
http://predictor.scripps.edu/workunit.php?wuid=1313336 claimed 6.62 granted 6.62

ID: 251 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile KWSN Sir Clark

Send message
Joined: 18 Sep 05
Posts: 46
Credit: 387,432
RAC: 0
Message 253 - Posted: 20 Sep 2005, 21:07:34 UTC

I think the doubling effect is why they use quorums and discard the top and bottom result and use the average so that's it's fairer in that sense.

With Rosetta currently assigning each WU to just one user, this allows cheating to go ahead but people should not care about the credit (I admit it does look good). But if they are crunching for credit, they are crunching for the wrong reasons. They should be crunching for the science and the quicker a workunit is crunched the sooner a new WU is sent to them and the more science they can get done.
ID: 253 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile [B^S] Paul@home
Avatar

Send message
Joined: 18 Sep 05
Posts: 34
Credit: 393,096
RAC: 0
Message 254 - Posted: 20 Sep 2005, 21:15:13 UTC
Last modified: 20 Sep 2005, 21:21:06 UTC

That is odd alright... and not what should happen at all!

What version of the CC are you running? is it a custom compiled or direct from berekeley.

This sounds more like a windows problem then a BOINC problem but since it's affecting 2 of your computers...

I have graphical and CPU intensive games and never seen this kind of jump in CPU time. Elapsed time increases alrigt but not WU time... (not noticably anyway)

Do your games run OK when this happens? I can see how this might occur (processor in continuous state change), but that shouldn't happen. And if it did, your games will go like pigs too.

does your CPU do any sort of virtual hyper threading (or whatever the AMD equivilant is!)? I see your machine registers with 1 CPU but if something like that was running in the background when you started our games it could cause something like this to happen.

What % CPU does the BOINC science app get when you are playing games?

You are hardly running some kind of virtual machine for BOINC and/or your games?

cheers,
Paul


Wanna visit BOINC Synergy team site? Click below!

Join BOINC Synergy Team
ID: 254 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
RDC

Send message
Joined: 16 Sep 05
Posts: 43
Credit: 101,644
RAC: 0
Message 255 - Posted: 20 Sep 2005, 21:19:49 UTC - in response to Message 253.  

I think the doubling effect is why they use quorums and discard the top and bottom result and use the average so that's it's fairer in that sense.

With Rosetta currently assigning each WU to just one user, this allows cheating to go ahead but people should not care about the credit (I admit it does look good). But if they are crunching for credit, they are crunching for the wrong reasons. They should be crunching for the science and the quicker a workunit is crunched the sooner a new WU is sent to them and the more science they can get done.


Exactly to both statements. I'm in this for the science primarily but do enjoy watching my credits go up and helping my team out. Science comes first however and that's the main reason I've brought the point up. It's something that does need to be addressed by the developers instead of just depending on the quorum system of validation to keep such things under control.

ID: 255 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
RDC

Send message
Joined: 16 Sep 05
Posts: 43
Credit: 101,644
RAC: 0
Message 256 - Posted: 20 Sep 2005, 21:23:39 UTC - in response to Message 254.  
Last modified: 20 Sep 2005, 21:31:41 UTC

What version of the CC are you running? is it a custom compiled or direct from berekeley.

This sounds more like a windows problem then a BOINC problem but since it's affecting 2 of your computers...


CC 4.45 direct from Berkeley.

I initially thought it was just a Win98 thing as that O/S was a mess to start with but tried the games on my WinXP PC and had the same issues.

No performance issues with the games. The CPU is pegged at 100% for both the game and Boinc application program (P@H, S@H, etc) when this occurs. I have no form of hyperthreading available on either PC.

(edit) Priority is set for low on the various BOINC programs so it should give way to the higher priority game but for some reason Windows isn't releasing the BOINC application but is attempting to share it with other non-related applications. (/edit)

ID: 256 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile [B^S] Paul@home
Avatar

Send message
Joined: 18 Sep 05
Posts: 34
Credit: 393,096
RAC: 0
Message 257 - Posted: 20 Sep 2005, 21:40:22 UTC - in response to Message 256.  

The CPU is pegged at 100% for both the game and Boinc application program (P@H, S@H, etc) when this occurs. I have no form of hyperthreading available on either PC.

OK, well thats not right... at all at all! 2x processes at 100%? hmmm.. windows not managing processes correctly? not sur what processes look after that!



(edit) Priority is set for low on the various BOINC programs so it should give way to the higher priority game but for some reason Windows isn't releasing the BOINC application but is attempting to share it with other non-related applications. (/edit)


boinc.exe should be 'normal' priority... it needs to be able to stop/start the science apps when it needs to.


Paul

Wanna visit BOINC Synergy team site? Click below!

Join BOINC Synergy Team
ID: 257 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
RDC

Send message
Joined: 16 Sep 05
Posts: 43
Credit: 101,644
RAC: 0
Message 259 - Posted: 20 Sep 2005, 22:09:21 UTC - in response to Message 257.  

boinc.exe is set for normal. I was referring to the science applications that are set for low. All of the priorities are as they should be on installation.

It has to be something with the games themselves and how they interact with BOINC in a Windows environment. Not all games cause BOINC to behave in such a manner. CPU intensive things do increase times slightly but what I've experienced is far from a slight increase. Everquest is another game that will cause increases in crunching time but not to the extent that Mech Warrior and Masters of Orion do.

BOINC should release the science application's use of the CPU when something else that has higher priority is using it. With some games, the CPU appears to be shared at full load with little to no effect on the game but a huge effect on the science application.

I e-mailed the devs about this problem a couple of years ago before other projects were out there and the e-mail I received back stated that I was misunderstanding how BOINC worked. I tend to think that they misunderstood what the problem is.
ID: 259 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile [B^S] Paul@home
Avatar

Send message
Joined: 18 Sep 05
Posts: 34
Credit: 393,096
RAC: 0
Message 260 - Posted: 20 Sep 2005, 22:21:54 UTC

The CPU can't be at full load for 2 applications. it's gotta be either 1 or the other... or split across the 2. If it was split, you shold see the game and science app at 50% CPU each. Since the game is working ok, it has to be something to do with how BOINC and Windows are talking to each other. Win thinks its giving BOINC 100% so CPU time is increasing even though it is getting little or no crunching done.

Could it be some sort of conflict between your games and BOINC? Something that BOINC wants to unload/stop but the games wont let it and so it apears BOINC is still taking CPU time?

Anything in your Event Viewer when you are starting + running the games?

What happens if you shut down BOINC, start the game, let it run up 100% CPU and then restart BOINC. Does the same thing happen?
Wanna visit BOINC Synergy team site? Click below!

Join BOINC Synergy Team
ID: 260 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
mrwizer
Avatar

Send message
Joined: 18 Sep 05
Posts: 23
Credit: 507,085
RAC: 0
Message 262 - Posted: 20 Sep 2005, 22:30:32 UTC

First, although many including myself crunch for the science aspect, you cannot deny the appeal of statistics and their mass draw to the genera public. Many people do crunch for stats, and we should not turn them away because of this. CPU time is CPU time, regardless of the reason behind it. That being said, stats should be taken seriously.

With only one work unit being sent out, you are ignoring many of the lessons learned from Seti Classic. The reason that validation and duplication was set up was to handle the inevitable problem of cheating. Various methods can be used for this, and I am not talking about games or loading your computing with other tasks. Their are methods used to manipulate computer time passage. If you computer is running at full load, yet the time is being manipulated, you are able to change the credit granted. You do not have to modify benchmarks, just the time. I am not sure if this is possible in Windows, but I have heard of it with other projects using Linux, such as FaD. And it does happen, again look at Seti Classic.

To keep the integrity of the project and its data, I think that you have to have redundancy. It is unfortunate and a waste of computer time, but a reality.
ID: 262 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
RDC

Send message
Joined: 16 Sep 05
Posts: 43
Credit: 101,644
RAC: 0
Message 264 - Posted: 20 Sep 2005, 22:52:58 UTC - in response to Message 260.  

Anything in your Event Viewer when you are starting + running the games?

What happens if you shut down BOINC, start the game, let it run up 100% CPU and then restart BOINC. Does the same thing happen?


Everything appears normal except that it shows two things running at 98-100%.

Doing what you suggested doesn't solve the problem, both continue to try and claim full useage of the CPU. I would say it's a Windows 98 problem but I'm having the same issues on WinXP as well. I couldn't try the request on my XP unit yet as I have a Rosetta WU being crunched currently on it but I am expecting the same results as I experienced on the Win98 unit with both reporting full CPU useage.


ID: 264 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Paul D. Buck

Send message
Joined: 17 Sep 05
Posts: 815
Credit: 1,812,737
RAC: 0
Message 277 - Posted: 21 Sep 2005, 14:33:18 UTC

The whole series of Windows from Win95 through ME are not really Operating Systems (by Paul's definition) because they do not in fact track CPU use per process or application. So, BOINC, which needs this tracking will basically track wall clock time. It has no choice in the matter.

Windows NT and all that are based on that, Win 2000, XP, are "true" OS in that they do actively manage and track CPU use by process and application. They are not very good at multi tasking, but that is another issue.

So, you are right, playing any CPU intensive game that has a "busy wait" loop, waiting on you to do something, will do the same thing. So, good, bad, or indifferent, this is a given. You will have distorted times.

And, yes, the throw out the high low average mechanism is there to help to accommodate this ...
ID: 277 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile [B^S] Paul@home
Avatar

Send message
Joined: 18 Sep 05
Posts: 34
Credit: 393,096
RAC: 0
Message 279 - Posted: 21 Sep 2005, 15:00:48 UTC - in response to Message 277.  



So, you are right, playing any CPU intensive game that has a "busy wait" loop, waiting on you to do something, will do the same thing. So, good, bad, or indifferent, this is a given. You will have distorted times.


But will (should?) a busy wait loop in the game process not stop BOINC taking CPU? What would be the point of busy wait if it did not reserve the CPU for the application? As far as BOINC would be concerned, the game ('normal priority') is taking 100% CPU.... even if it is just spinning the processor waiting for a mouse click!

What RDC is describing is both the game and BOINC sitting at 100% CPU time... I could kind of understand that happening if BOINC was actually getting CPU time during these busy loops but if that was the case, then surely we would not see such increases in processing time? ie, it would continue to crunch results in the usual time (or perhaps slightly more)

cheers,
Paul.
Wanna visit BOINC Synergy team site? Click below!

Join BOINC Synergy Team
ID: 279 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
1 · 2 · 3 · Next

Message boards : Number crunching : canonical result



©2024 University of Washington
https://www.bakerlab.org