Internet traffic and necessary data

Message boards : Number crunching : Internet traffic and necessary data

To post messages, you must log in.

1 · 2 · 3 · 4 · Next

AuthorMessage
Profile SwZ
Avatar

Send message
Joined: 1 Jan 06
Posts: 37
Credit: 169,775
RAC: 0
Message 8284 - Posted: 3 Jan 2006, 18:40:03 UTC

The balance is between the amount of network traffic and the length of the jobs. Each WU is doing ten independent folding trajectories. If we cut this down to two, for example, jobs would run 5 fold faster, but the traffic would go up five fold. Once we have more hardware in place we should be able to deal with more traffic. But the work shouldn't be completely lost--if you have already written out five structures when you turn off your computer off, the next time you turn it on it should only have to make five more (I need to check on this).


So we need to reduce transfering data per WU. Me not confuse from necessity continuously PC working all day and night, but very confuse traffic. About 50Mb per day - it too large traffic. Usualy I spend about 300Mb per month, which cost about $30. And 50Mb/day*30day=1.5Gb, which cost about $150. This is half of my salary!!!
If we want to use "rent-free" PC resources in all world, we must understand, that "rent-free" in DC is CPU time, but not internet traffic. And reasonable traffic size about 1-2Mb/day/one CPU core.

ID: 8284 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile SwZ
Avatar

Send message
Joined: 1 Jan 06
Posts: 37
Credit: 169,775
RAC: 0
Message 8289 - Posted: 3 Jan 2006, 18:55:33 UTC

I try simple pack text file aa1dtjA09_05.400_v1_3 with fragments data
" position: 1 neighbors: 400

1f06 A 162 V L -83.317 116.745 175.741
1f06 A 163 Q L -85.736 -44.654 174.056
1f06 A 164 K E -149.188 143.809 -179.095
1f06 A 165 A E -160.412 159.686 176.956
1f06 A 166 V E -154.213 147.815 181.314
1f06 A 167 Q E -126.751 129.608 177.521
1f06 A 168 Y E -107.000 136.290 175.811
1f06 A 169 T E -92.039 117.444 -177.064
1f06 A 170 L E -121.951 129.001 179.049

1jz8 A 250 L L -82.940 116.696 -172.679
1jz8 A 251 R L -144.692 153.380 -175.553
....................
"
with RAR instead GZIP
and obtain archive size 1632Kb instead 3446Kb.
May by use RAR or 7ZIP instead GZIP?
ID: 8289 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Paul D. Buck

Send message
Joined: 17 Sep 05
Posts: 815
Credit: 1,812,737
RAC: 0
Message 8290 - Posted: 3 Jan 2006, 18:56:56 UTC

All of which may mean that projects like CPDN, Einstein@Home and Rosetta@Home may not be for you due to the connection costs.

You have to balance what you are willing to spend with which projects you wish to do. SETI@Home is nice in that the data files in both directions are fairly small. BUt this is why the projects, and the wiki try to outline the typical sizes of things so that you can make intelligent choices.
ID: 8290 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile SwZ
Avatar

Send message
Joined: 1 Jan 06
Posts: 37
Credit: 169,775
RAC: 0
Message 8291 - Posted: 3 Jan 2006, 19:11:32 UTC - in response to Message 8290.  

All of which may mean that projects like CPDN, Einstein@Home and Rosetta@Home may not be for you due to the connection costs.

You have to balance what you are willing to spend with which projects you wish to do. SETI@Home is nice in that the data files in both directions are fairly small. BUt this is why the projects, and the wiki try to outline the typical sizes of things so that you can make intelligent choices.


Yes, OK. I want to say, that we can easy reduce traffic in 2-5 times. And emphazed problem, but not want complain to own hardships.

So if using 7zip we have 1479 Mb instead 3446Kb.

Next, if apply discrete numbers we can pack each line in six bytes, so after packing with RAR obtain about 800 Mb instead 3.4Mb.
ID: 8291 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile SwZ
Avatar

Send message
Joined: 1 Jan 06
Posts: 37
Credit: 169,775
RAC: 0
Message 8292 - Posted: 3 Jan 2006, 19:18:06 UTC

Next, if I right understude, 10 trajectories per protein is not limit. May be it possible to make this number optional, so that user can change it and tune traffic and WU frequency.
ID: 8292 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile SwZ
Avatar

Send message
Joined: 1 Jan 06
Posts: 37
Credit: 169,775
RAC: 0
Message 8293 - Posted: 3 Jan 2006, 19:24:16 UTC - in response to Message 8290.  

All of which may mean that projects like CPDN, Einstein@Home and Rosetta@Home may not be for you due to the connection costs.


Other projects not perspective, but Protein Folding - mankind future. ;-) (Joke)
So I will participate in this project!

ID: 8293 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Divide Overflow

Send message
Joined: 17 Sep 05
Posts: 82
Credit: 921,382
RAC: 0
Message 8298 - Posted: 3 Jan 2006, 20:55:51 UTC - in response to Message 8284.  

Each BOINC project will have different requirements, including network traffic volume. If Rosetta is asking for too much bandwidth for you, please consider one of the other projects out there.
ID: 8298 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Nothing But Idle Time

Send message
Joined: 28 Sep 05
Posts: 209
Credit: 139,545
RAC: 0
Message 8314 - Posted: 4 Jan 2006, 0:20:00 UTC

Swz:

File compression was suggested before in these threads; I mention it here in case you have not seen them before --
thread1 and thread2
ID: 8314 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
FluffyChicken
Avatar

Send message
Joined: 1 Nov 05
Posts: 1260
Credit: 369,635
RAC: 0
Message 8343 - Posted: 4 Jan 2006, 11:04:09 UTC

Thanks NBIT saves me doing the search to link to them (do you use NBIT ?).


Although to the people that say move elsewhere, I see this post as feedback to the developers as ways to overcome this problem are given.

CPDN is actually not too bad, bandwitdh may look large in the first instance, but since that is pretty much it apart from some small trickles for half a year or so, it's overall negligable ;-)


Team mauisun.org
ID: 8343 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Nothing But Idle Time

Send message
Joined: 28 Sep 05
Posts: 209
Credit: 139,545
RAC: 0
Message 8351 - Posted: 4 Jan 2006, 14:09:28 UTC - in response to Message 8343.  

...(do you use NBIT ?).


Just a coincidence. I chose NBIT as a moniker because I'm retired; me and my computer have nothing but idle time on our hands! :)
ID: 8351 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Lee Carre

Send message
Joined: 6 Oct 05
Posts: 96
Credit: 79,331
RAC: 0
Message 8522 - Posted: 7 Jan 2006, 6:18:16 UTC - in response to Message 8351.  
Last modified: 7 Jan 2006, 6:18:32 UTC

...(do you use NBIT ?).


Just a coincidence. I chose NBIT as a moniker because I'm retired; me and my computer have nothing but idle time on our hands! :)

i'd hope that your computer isn't idle ;)
surely it's at 100% constantly
ID: 8522 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
tw56

Send message
Joined: 13 Dec 05
Posts: 7
Credit: 519,161
RAC: 0
Message 8959 - Posted: 13 Jan 2006, 18:27:36 UTC - in response to Message 8293.  

All of which may mean that projects like CPDN, Einstein@Home and Rosetta@Home may not be for you due to the connection costs.


Other projects not perspective, but Protein Folding - mankind future. ;-) (Joke)
So I will participate in this project!

One possibility. Set preference to download work less often then disconnect network cable while computer runs work. Connect back up to send results download new work.

ID: 8959 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile SwZ
Avatar

Send message
Joined: 1 Jan 06
Posts: 37
Credit: 169,775
RAC: 0
Message 8960 - Posted: 13 Jan 2006, 18:33:18 UTC - in response to Message 8959.  


One possibility. Set preference to download work less often then disconnect network cable while computer runs work. Connect back up to send results download new work.

Pay cost for megabytes and disconnecting not change paying.
See https://boinc.bakerlab.org/rosetta/forum_thread.php?id=756
ID: 8960 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
FluffyChicken
Avatar

Send message
Joined: 1 Nov 05
Posts: 1260
Credit: 369,635
RAC: 0
Message 9011 - Posted: 14 Jan 2006, 15:17:13 UTC - in response to Message 8959.  

All of which may mean that projects like CPDN, Einstein@Home and Rosetta@Home may not be for you due to the connection costs.


Other projects not perspective, but Protein Folding - mankind future. ;-) (Joke)
So I will participate in this project!

One possibility. Set preference to download work less often then disconnect network cable while computer runs work. Connect back up to send results download new work.



Actually you can just tell 'boinc' to suspend network activity, which is much easier and far more prctical then running around pulling and reconnecting cables ;-)
And if you need to do it on alot of computers at one or even individually from one location BoincView makes it easy.

Team mauisun.org
ID: 9011 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile hob.
Avatar

Send message
Joined: 4 Nov 05
Posts: 64
Credit: 250,683
RAC: 0
Message 9293 - Posted: 18 Jan 2006, 18:54:43 UTC

it is with great reluctance that i now have to quit rosetta .......at least for the time being.

the size of the downloads has made it impossable to run the project anymore on my dialup connection.

with my offpeak connection the best i can manage to download is 40 Mb per day, just one of my machines is now trying to download 95Mb 8-(.........i have 11 machines running here.......downloading ~ 1 gig a week is just NOT an option.

i would suggest to rosetta that unless thay can limit in some way download sizes they are going to lose a lot of volunteers.

i have NO choice but to take my 25.6 Ghz of processors somewhere else, pity because the management here are so good in so many other ways.

hob
46 years dc so far

join team FaDbeens
join us

ID: 9293 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile River~~
Avatar

Send message
Joined: 15 Dec 05
Posts: 761
Credit: 285,578
RAC: 0
Message 9297 - Posted: 18 Jan 2006, 19:23:05 UTC - in response to Message 9293.  

it is with great reluctance that i now have to quit rosetta .......at least for the time being.


hi hob,

thanks for letting us know - and do keep an eye on these boards once in a while as I am sure it will be posted when the trafic issue is made more manageable.

I also would not expect it to be all that soon, with so many other issues that need sorting out.

I'm sure with your long experience of DC you will find another worthy project for your cpu cycles

River~~
ID: 9297 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile carl.h
Avatar

Send message
Joined: 28 Dec 05
Posts: 555
Credit: 183,449
RAC: 0
Message 9299 - Posted: 18 Jan 2006, 19:29:57 UTC

Hob, been a long time brother ..nearly 5 year now...I`ll catch up with you elsewhere

Rosey you just lost one of the best all time crunchers , see ya buddy ..

Something for UW to think on....

In the UK a lot of our Broadband contracts (cheap) are restricted to 2GB a month, I don`t think any of us wish to pay more purely for the pleasure of Rosetta. Compression of WU is a priority issue.
Not all Czech`s bounce but I`d like to try with Barbar ;-)

Make no mistake This IS the TEDDIES TEAM.
ID: 9299 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Morphy375
Avatar

Send message
Joined: 2 Nov 05
Posts: 86
Credit: 1,629,758
RAC: 0
Message 9301 - Posted: 18 Jan 2006, 19:36:35 UTC

Sad news hob, really sad news.... :-((
Teddies....
ID: 9301 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile hob.
Avatar

Send message
Joined: 4 Nov 05
Posts: 64
Credit: 250,683
RAC: 0
Message 9308 - Posted: 18 Jan 2006, 21:39:07 UTC

ok .....i'm out apart from a few jobs left to run

testing predictor on one machine at the moment.....most are shut down

.....i can hear the tv clearly now
46 years dc so far

join team FaDbeens
join us

ID: 9308 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Scribe
Avatar

Send message
Joined: 2 Nov 05
Posts: 284
Credit: 157,359
RAC: 0
Message 9324 - Posted: 19 Jan 2006, 6:58:54 UTC

Hob, not too sure Predictor is worthwhile, lots of complaints about it...check out here and their forum.....
ID: 9324 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
1 · 2 · 3 · 4 · Next

Message boards : Number crunching : Internet traffic and necessary data



©2024 University of Washington
https://www.bakerlab.org