Hosted CPUs

Message boards : Number crunching : Hosted CPUs

To post messages, you must log in.

AuthorMessage
Paul

Send message
Joined: 29 Oct 05
Posts: 193
Credit: 65,819,857
RAC: 2,378
Message 68892 - Posted: 26 Dec 2010, 13:39:23 UTC

Has anyone located a hosted service like Amazon Web Service for crunching Rosetta? I have heard about hosted CPUs for as little as $15 per month. The Amazon service could be a good option as they can sell excess capacity at low prices.

It would be great to learn from everyones research and see if we can find a way to get thousands of additional processors on the project.

It would also be helpful if someone can help explain how to get BOINC client installed on hosted systems. I do not understand how to build a linux configuration for BOINC client for hosted systems. I am happy to build a linux configuration if someone can help me understand the requirements.

I hope everyone enjoyed Christmas. Maybe one more year of crunching will get us closer to the answer.

Thank you.
Thx!

Paul

ID: 68892 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Chris Holvenstot
Avatar

Send message
Joined: 2 May 10
Posts: 220
Credit: 9,106,918
RAC: 0
Message 68895 - Posted: 26 Dec 2010, 20:43:46 UTC

The Linux configuration is real easy to work with - the install only takes you a few moments. Are you looking to do this on a "hosted" Linux system, or on a system you build and run yourself?

I guess that is the important question.

Before jumping into a "hosted" solution for a CPU intensive application like Rosetta, I think that I would communicate with the provider and find out some specifics - I suspect that what they are selling are "virtual" systems and if so then you would need to get answers to the following:

1. How many CPUs will your image see?

2. How much real memory is available to you image?

3. What, if any throttling to they impose?

To be realistic, a modern quad-core system running at home will take about $30 a month to feed (power and cooling) when running any BOINC application - they do generate some heat.

Another question to ask is do they provide "user" mode terminal or shell access to the system they are leasing you - it would make the install real easy. Root access is not required however.

Let me know what they say - I am a bit curious - due to power and heat limitations in my residence I have held off on deploying any new systems for a while and instead have resorted to squeezing a bit more out of my existing systems using custom kernels - however this is only getting me about 5 to 7% so far and I am getting the itch to grow again. I'm looking at trying to optimize glibc but its potential will be dependent on how the Rosetta apps are linked - static or not.

CH



ID: 68895 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Paul

Send message
Joined: 29 Oct 05
Posts: 193
Credit: 65,819,857
RAC: 2,378
Message 68920 - Posted: 29 Dec 2010, 12:23:07 UTC - in response to Message 68895.  

Nice RAC!

Most of the providers run virtual machines on multicore computers but they guarantee at least 1 core of performance. You usually get 1GB of RAM per core and you can buy more.

Amazon Web Service will let you purchase more RAM, more CPU, etc. For a High CPU medium (2 cores), the price is 17 cents per hour or $28.56 per week. They also sell unused capacity called SPOT Instances for the same high CPU instance is just under 6 cents per hour or $10.08 per week. You can get it for even less sometimes.

I just don't understand how to upload an image or control the device. I am sure you get telnet so I guess I could to the install from the command line but that is ugly for me.

If anyone has specifics, it would be great to learn from you.

thanks

Thx!

Paul

ID: 68920 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Matt40k
Avatar

Send message
Joined: 7 Dec 10
Posts: 6
Credit: 2,361,176
RAC: 0
Message 68922 - Posted: 29 Dec 2010, 12:50:44 UTC

The current "free" service looks interesting :)
ID: 68922 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Feet1st
Avatar

Send message
Joined: 30 Dec 05
Posts: 1755
Credit: 4,690,520
RAC: 0
Message 68927 - Posted: 29 Dec 2010, 17:18:33 UTC

I've run Rosetta on Amazon Linux instances before. I've also studied many other hosting services and never found one that met the needs of Rosetta for anything near the $15/month figure you mentioned (except for a single CPU Amazon Spot priced instance when they go for around 2 cents an hour). They either don't allow you to install software on their server, they don't give you any significant CPU time, they cost more then Amazon full price, or all of the above.

Amazon provides full SSH access to the instance. What I did was just wget the BOINC code and install and run it as per the Linux instructions from BOINC. But, and especially with the spot instances, you have to remember that your running image could be shut down at any time. I used EBS for BOINC's storage to partially prepare for that eventuality. I also configured BOINC preferences to immediately report completed results (but that's producing more hits to the project servers).

So, you don't need to create an image at all, you can just use one of the existing AMIs, but when the instance is ended (i.e. someone out bids you for spot pricing) it is a manual process to get started again. The next step on what I've done is essentially to facilitate a farm of hosts and automate the steps. Create an image that boots itself by loading up a current (or stored) BOINC version, checks with some central host to see if any partially completed images are available, and then either picks up where another ended instance left off by attaching to the corresponding EBS (so long as the instance types are identical, BOINC will just think the machine was rebooted), or create a new one with files set up in a mannar to automatically connect to the project with your EMail address and password. There's some BOINC doc on doing that in the wiki at Berkeley.
Add this signature to your EMail:
Running Microsoft's "System Idle Process" will never help cure cancer, AIDS nor Alzheimer's. But running Rosetta@home just might!
https://boinc.bakerlab.org/rosetta/
ID: 68927 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Matt40k
Avatar

Send message
Joined: 7 Dec 10
Posts: 6
Credit: 2,361,176
RAC: 0
Message 68929 - Posted: 29 Dec 2010, 19:14:56 UTC

Have you seen prgmr.com?
ID: 68929 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Feet1st
Avatar

Send message
Joined: 30 Dec 05
Posts: 1755
Credit: 4,690,520
RAC: 0
Message 68934 - Posted: 29 Dec 2010, 21:35:00 UTC - in response to Message 68929.  

Have you seen prgmr.com?


I see nothing there indicating any definition of the amount of CPU time guarenteed nor expected. The SLA simply refers to up time.

If I asked you to host a CPU for me, with internet connectivity, some disk space, etc. how much would you have to charge to break-even? Power, cooling, floorspace, routers, firewalls, software updates, insurance, internet bandwidth...

The only way to offer something for less then the electricity costs of a single CPU is to provide less then a single CPU.
Add this signature to your EMail:
Running Microsoft's "System Idle Process" will never help cure cancer, AIDS nor Alzheimer's. But running Rosetta@home just might!
https://boinc.bakerlab.org/rosetta/
ID: 68934 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Matt40k
Avatar

Send message
Joined: 7 Dec 10
Posts: 6
Credit: 2,361,176
RAC: 0
Message 68937 - Posted: 30 Dec 2010, 8:29:34 UTC

If you look on there, Luke has posted the cpu scheduling, so you can see how nuch cpu time you would get.

The beneift of this over Amazon would be you don't get charged when a lot of cpu time is free, which last time I checked, was a lot of the time.
Also the cost of 1 core on a cpu is less than all the cores on a cpu.

This would an easy way to give a stable stream, with no start costs, unlike buying servers and colocation. Anyway, just my honest opinon and what I've got planned.

Going my fact the no1 team is full of desktops, I assume the "best" idea is to use huge amounts of computers, and run it in the background whilst they are idle, ie they are using that shinny i7 to surf the internet :)
ID: 68937 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote

Message boards : Number crunching : Hosted CPUs



©2024 University of Washington
https://www.bakerlab.org