Internet traffic and necessary data

Message boards : Number crunching : Internet traffic and necessary data

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · Next

AuthorMessage
Lee Carre

Send message
Joined: 6 Oct 05
Posts: 96
Credit: 79,331
RAC: 0
Message 10193 - Posted: 29 Jan 2006, 16:10:56 UTC - in response to Message 10189.  
Last modified: 29 Jan 2006, 16:23:20 UTC

ID: 10193 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile carl.h
Avatar

Send message
Joined: 28 Dec 05
Posts: 555
Credit: 183,449
RAC: 0
Message 10195 - Posted: 29 Jan 2006, 16:20:21 UTC

40:1 home....
Not all Czech`s bounce but I`d like to try with Barbar ;-)

Make no mistake This IS the TEDDIES TEAM.
ID: 10195 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Lee Carre

Send message
Joined: 6 Oct 05
Posts: 96
Credit: 79,331
RAC: 0
Message 10196 - Posted: 29 Jan 2006, 16:23:47 UTC - in response to Message 10195.  

40:1 home....

yup
ID: 10196 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile carl.h
Avatar

Send message
Joined: 28 Dec 05
Posts: 555
Credit: 183,449
RAC: 0
Message 10197 - Posted: 29 Jan 2006, 16:30:04 UTC

Yep..That`s what we get.... but I think I`m 39 of those
Not all Czech`s bounce but I`d like to try with Barbar ;-)

Make no mistake This IS the TEDDIES TEAM.
ID: 10197 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Morphy375
Avatar

Send message
Joined: 2 Nov 05
Posts: 86
Credit: 1,629,758
RAC: 0
Message 10212 - Posted: 30 Jan 2006, 0:57:15 UTC

ID: 10212 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
pisi78

Send message
Joined: 25 Oct 05
Posts: 2
Credit: 199,062
RAC: 0
Message 10217 - Posted: 30 Jan 2006, 12:04:57 UTC
Last modified: 30 Jan 2006, 12:08:54 UTC


ID: 10217 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile nasher

Send message
Joined: 5 Nov 05
Posts: 98
Credit: 618,288
RAC: 0
Message 10252 - Posted: 31 Jan 2006, 9:16:21 UTC

honestly i didnt realize that there were so many countries and restrictions on the brodband brodcasts and such world wide...

guess im just a spoiled being in the US

yes i believe there are always ways to improve the programing and compresion of files and i always hoped they would but never saw a reason why they would care.. well i guess alot of people in other countries care and especialy for all of you i hope they find a better compresion and such for you soon.

of corse i dont know the exchange rates and right now im too tired to bother to look them up.. but with my cable modem and all the cable channels i get and such i pay about $100 US a month. but i get and use alot of bandwith on my cable modem haveing 4-6 computers connected 24/7 and useing a bunch of them for playing online games as well as Distributed computing.
ID: 10252 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Lee Carre

Send message
Joined: 6 Oct 05
Posts: 96
Credit: 79,331
RAC: 0
Message 10257 - Posted: 31 Jan 2006, 11:57:18 UTC - in response to Message 10252.  

useing a bunch of them for playing online games

gaming doesn't use that much in the way of bandwidth (but i'm guessing newer games use more, due to more in-game data to account for, the main thing with games is response times, how long it takes data to get from one end to the other, that's why you always see "latency" numbers, so show the "speed" of your connection to the host
ID: 10257 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
BennyRop

Send message
Joined: 17 Dec 05
Posts: 555
Credit: 140,800
RAC: 0
Message 10309 - Posted: 1 Feb 2006, 9:13:31 UTC

The cable modem setup here on Kodiak that I was using was 512k down/128k up with an 8 gig/mo usage cap. $20/gig for every gig over 8.
2 machines playing Desert Combat (another online app in serious need of minimizing network traffic) for 4 hours a day while running team speak, and we went over the 8 gig cap; and have since moved to online games that were built to be played over dial up connections. ;)

With Distributed Folding, we'd download a 2-4Meg file for each protein - and work on that for 1 week during Casp trials, or around a month during normal times. We'd send tiny bits of data back every 4 hours on some of the machines I was running at the time.

With Folding @ Home, we download a reasonably small file (1-2 megs?) that takes most of a day for the short runs, and 2-3 days for the longer runs on my 1.5-1.8 Ghz Athlons. It sends back a small amount of data when finished and gets another job.

Given that the system should know the last week's performance of our machines, is it possible to use that information and give out 1 job per machine with enough tasks for that job to keep a system busy for 1 or 2 days? And sending back results every few hours?

Reducing the bandwidth by using much better compression, and setting up much smaller results files which would compress even further would also be appreciated. Even if my single cpu is no longer behind a connection with a usage cap.


ID: 10309 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Nite Owl
Avatar

Send message
Joined: 2 Nov 05
Posts: 87
Credit: 3,019,449
RAC: 0
Message 10310 - Posted: 1 Feb 2006, 9:38:43 UTC - in response to Message 10252.  
Last modified: 1 Feb 2006, 9:41:23 UTC

Nasher wrote:
honestly i didnt realize that there were so many countries and restrictions on the brodband brodcasts and such world wide...

guess I'm just a spoiled being in the US.

Being in the US doesn't solve the problem for everybody.... It's where in the US you're located. Where I am my only option (other than phone line) is a Two-Way Satellite, and that's a problem......
ID: 10310 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile ecafkid

Send message
Joined: 5 Oct 05
Posts: 40
Credit: 15,177,319
RAC: 0
Message 10413 - Posted: 3 Feb 2006, 15:41:29 UTC
Last modified: 3 Feb 2006, 15:43:51 UTC

Nite Owl, Here is a link for wireless Broadband in your area. As I sell this type of service in the Midwest and utilize it myself you may want to check it out for other options. Just a thought. Not trying to stick my nose where it doen's belong.

http://www.spectrumsciences.com/somd/index.html




ID: 10413 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Nite Owl
Avatar

Send message
Joined: 2 Nov 05
Posts: 87
Credit: 3,019,449
RAC: 0
Message 10422 - Posted: 3 Feb 2006, 17:27:33 UTC - in response to Message 10413.  

Nite Owl, Here is a link for wireless Broadband in your area. As I sell this type of service in the Midwest and utilize it myself you may want to check it out for other options. Just a thought. Not trying to stick my nose where it doen's belong.

http://www.spectrumsciences.com/somd/index.html




Thanks for the link E-kid! :thumbsup: I checked out their website... Their coverage area is too far North to include me... Anyhoo, since I dumped Rosetta and switched to WCG I haven't had the first problem with their work units NOR my Satellite broadband service... Crunching 24/7 on all 30 machines...
ID: 10422 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Carlos_Pfitzner
Avatar

Send message
Joined: 22 Dec 05
Posts: 71
Credit: 138,867
RAC: 0
Message 10689 - Posted: 12 Feb 2006, 4:22:47 UTC - in response to Message 9308.  
Last modified: 12 Feb 2006, 4:30:15 UTC

ok .....i'm out apart from a few jobs left to run

testing predictor on one machine at the moment.....most are shut down

.....i can hear the tv clearly now


SETI-Beta is the lowest traffic / credits/h -there will be a astro-pulse app too
and there is need of more cruhchers to test / validate the app

I am the founder of FaDBeens team at Seti-beta, so u keep the same team u are in

-x-x-x-x-x-

I verifyed that from all files that rosetta download for each WU
that the largest file is the xxxx.pdb. (2 to 7 mb each pdb)

*and this large file is *not* used by the crunching app
*I am believing that this file is used only to show the "native" protein fold,
on the screen_saver -or- when we click "show graphics"

If so, here a suggestion for rosetta options ops preferences
to skip downloading this file at users choice

*Ofcourse who opt to not d/l it will not see the "native" fold on screen.

And Here the comprobation that it is not used :!:
https://boinc.bakerlab.org/rosetta/forum_thread.php?id=1035
look for lsof -u boinc
*all* files that are used by that userid while rosetta run are listed following.
*Is a Linux system, however file usage, should be the same for windows

Click signature for global team stats
ID: 10689 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Lee Carre

Send message
Joined: 6 Oct 05
Posts: 96
Credit: 79,331
RAC: 0
Message 10694 - Posted: 12 Feb 2006, 14:20:27 UTC - in response to Message 10689.  
Last modified: 12 Feb 2006, 14:22:34 UTC

I verifyed that from all files that rosetta download for each WU
that the largest file is the xxxx.pdb. (2 to 7 mb each pdb)

*and this large file is *not* used by the crunching app
*I am believing that this file is used only to show the "native" protein fold,
on the screen_saver -or- when we click "show graphics"

If so, here a suggestion for rosetta options ops preferences
to skip downloading this file at users choice

*Ofcourse who opt to not d/l it will not see the "native" fold on screen.

as far as i know .pdb files are so that when/if the accociated app crashes, it produces a meaningful error rather than just an error code and a memory address (which isn't that helpful)

yes, strictly pdb files aren't needed, but i'm sure most users will appreciate having an error message rather than a hex code

however i may be wrong, and it could be for the screensaver and not a symbol file (but in my experience pdb files are generally symbol files for app crashes)
ID: 10694 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Carlos_Pfitzner
Avatar

Send message
Joined: 22 Dec 05
Posts: 71
Credit: 138,867
RAC: 0
Message 10701 - Posted: 12 Feb 2006, 21:23:44 UTC
Last modified: 12 Feb 2006, 21:37:09 UTC

If it was for app symbol table debug, it needs be sent *only*
when the app changes eg: 4.81 -> 4.82

and then no need to send to the cruncher a new pdb at each new WU

Thus, I still believe that it is for graphics usage "native fold"

*Like of PDB's? Download how many u want here -:)
http://www.rcsb.org/pdb/Welcome.do;jsessionid=RinuB0Wrql+Qg5+NkM-EGA**

Ps: I am sure that the user I quoted, would appreciate hex codes ...
*He is about to left rosetta forever,
(cause size of WUs to download versus size $$$ of his Telephone Bill!)
Click signature for global team stats
ID: 10701 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Dimitris Hatzopoulos

Send message
Joined: 5 Jan 06
Posts: 336
Credit: 80,939
RAC: 0
Message 10703 - Posted: 12 Feb 2006, 23:21:24 UTC

1/ The obvious first move would be to increase nstruct, from 10 to something considerably higher, like 20 or 50 or 100. Currently it only takes a couple of hours to process the average Rosetta WU on a 3yr old P4 computer. And a new WU means a 2-3MBytes download. All these downloads quickly add up to a Gigabyte per month per P4 CPU, which lead to people dropping from Rosetta.

2/ The next step to minimize donor overheads would be using 7zip/bzip2/etc instead of GZip, but the impact in overall traffic reduction will be small compared to #1 imo.

3/ And as long as R is using mostly single precision floats, they could consider compiling with SSE for a 3-4 times speedup.

There have been a few experienced programmers offering help in the past, so I hope that something is in the works.
Best UFO Resources
Wikipedia R@h
How-To: Join Distributed Computing projects that benefit humanity
ID: 10703 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Lee Carre

Send message
Joined: 6 Oct 05
Posts: 96
Credit: 79,331
RAC: 0
Message 10705 - Posted: 13 Feb 2006, 1:10:52 UTC - in response to Message 10701.  

If it was for app symbol table debug, it needs be sent *only*
when the app changes eg: 4.81 -> 4.82

and then no need to send to the cruncher a new pdb at each new WU

Thus, I still believe that it is for graphics usage "native fold"

ah, you're right then, i must admit, i don't monitor what's downloaded and network usage, because bandwidth isn't a problem for me, so i don't really pay too much attention to what kind of files are downloaded
ID: 10705 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Lee Carre

Send message
Joined: 6 Oct 05
Posts: 96
Credit: 79,331
RAC: 0
Message 10706 - Posted: 13 Feb 2006, 1:15:10 UTC - in response to Message 10703.  
Last modified: 13 Feb 2006, 1:15:45 UTC

2/ The next step to minimize donor overheads would be using 7zip/bzip2/etc instead of GZip, but the impact in overall traffic reduction will be small compared to #1 imo.

different compression methods will only be adopted if they work across all platforms, if they don't, then they're not appropriate for BOINC use

3/ And as long as R is using mostly single precision floats, they could consider compiling with SSE for a 3-4 times speedup.

or do as SETI Beta are doing, and re-write the app so that it uses a basic method by default, but uses SSE when it detects that the processor is capable of handling that instruction set, best of both world then, because i'm sure you'd get a lot of people complaining that rosetta no longer runs on their older computers,
and besides, rosetta is still seeking more processing power last time i checked, so excluding hosts is a bad thing, especially if the app is just going to error out on a non-compatible host
ID: 10706 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
BennyRop

Send message
Joined: 17 Dec 05
Posts: 555
Credit: 140,800
RAC: 0
Message 10707 - Posted: 13 Feb 2006, 1:22:03 UTC

Every move that will decrease network traffic to half or less than its current rate would be a move in the right direction; be it compressing things to 1/2 or less of their current size by using much more efficient compression engines and options; increasing nstruct so that the machine crunches one job all day, etc.

There's been suggestions of only downloading certain non changing files (such as the PDBs) once per machine - and for networked users, it would be nice to have all the non changing files stored and grabbed from a network share; not pulled off the internet. (Verify time/date/size/pgp key to prove it hasn't been corrupted)

Any idea when we'll start seeing some of these options that the project leaders thought were good - put into a client that we can start using?
ID: 10707 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Dimitris Hatzopoulos

Send message
Joined: 5 Jan 06
Posts: 336
Credit: 80,939
RAC: 0
Message 10708 - Posted: 13 Feb 2006, 1:48:02 UTC - in response to Message 10706.  

different compression methods will only be adopted if they work across all platforms, if they don't, then they're not appropriate for BOINC use


Well, that's why I suggested bzip2 as it's an open-source, plug-in replacement for gzip. Works in all platforms.


or do as SETI Beta are doing, and re-write the app so that it uses a basic method by default, but uses SSE when it detects that the processor is capable of handling that instruction set, best of both world then, because i'm sure you'd get a lot of people complaining that rosetta no longer runs on their older computers,
and besides, rosetta is still seeking more processing power last time i checked, so excluding hosts is a bad thing, especially if the app is just going to error out on a non-compatible host


Agreed, but as you correct said more processing power, NOT necessarily more HOSTS. Have a look at CPU stats

Personally, I'd be happy with offering a beta-SSE-enabled Rosetta executable, as optional install, like many people install optimised BOINC app.
Best UFO Resources
Wikipedia R@h
How-To: Join Distributed Computing projects that benefit humanity
ID: 10708 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Previous · 1 · 2 · 3 · 4 · Next

Message boards : Number crunching : Internet traffic and necessary data



©2024 University of Washington
https://www.bakerlab.org