Index  | Recent Threads  | Unanswered Threads  | Who's Active  | Guidelines  | Search
 

Quick Go ยป
No member browsing this thread
Thread Status: Active
Total posts in this thread: 15
Posts: 15   Pages: 2   [ 1 2 | Next Page ]
[ Jump to Last Post ]
Post new Thread
Author
Previous Thread This topic has been viewed 1417 times and has 14 replies Next Thread
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Distribute Storage

I was speaking of this in the NRW forum but figured I'd just put it here as a "documented idea" for the record. It is complicated but is used today. For the researchers with extremely tight budgets and might not be able to afford storage space right away for some of their completed work (ie, NRW)..that work could be archived until needed or space is freed up by using the free space on volunteers' hard drives.

A user could donate whatever amount of hard disk space they would like and it would be distributed and replicated over many many computers. So if one crashes or is offline when the data is needed, other computers would have that same data for redundancy.

I can't remember the service's name but there is a website you can go to now to do this. You can either buy space and upload your files which would then be downloaded to other members' computers until needed. Or you can donate say 50GB worth of your drive space and be able to use 50GB worth of drive space on the members' computers. The latter would mainly just be for redundancy of data.

Just a thought as WCG expands. Might be an awesome thing to offer the researchers. Granted, it would take a large amount of development manhours...but might be worth exploring at some point in the future. If even just a fraction of the WCG members donated space...you're looking at many many terabytes worth of available storage space.
[Dec 31, 2008 3:37:23 PM]   Link   Report threatening or abusive post: please login first  Go to top 
kskjold
Senior Cruncher
Norway
Joined: May 20, 2008
Post Count: 469
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Distribute Storage

I support this suggestion.
If this was possible too day, I would lend out about 10 Gb. And this number will only raise whenever you buy a new computer with greater space....
Thumbs up cool
----------------------------------------
----------------------------------------
[Edit 1 times, last edit by kskjold at Dec 31, 2008 5:16:33 PM]
[Dec 31, 2008 5:16:09 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: Distribute Storage

Yea, I have about 100GB I could easily spare on my server that runs WCG. I also have a 500GB HD that I could bring online just for WCG as well.
[Dec 31, 2008 5:32:04 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: Distribute Storage

As I said in the other thread, we have discussed this before.
[Dec 31, 2008 5:43:30 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: Distribute Storage

I know, I did a search but couldn't find it. So I figured I'd post it anyway. No harm no foul.
[Dec 31, 2008 5:49:36 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: Distribute Storage

I support this suggestion.


please feel free to work as a volunteer for WCG and implement the suggestion. It sounds like all you have to do is put in a few years of labour. I'll be waiting for the Beta WU
[Dec 31, 2008 5:52:40 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: Distribute Storage

Heh heh, I'll get right on it! I wonder if I could do it as a vbs script? That's about the extent of my programming knowledge....and a little VB6
[Dec 31, 2008 6:10:53 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: Distribute Storage

BOINC is supposed to be able to deal with distributed storage. However no project uses it and a recent report said the code was broken on both the server and client.
[Jan 1, 2009 8:16:00 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: Distribute Storage

Looks like there is something already out there for helping to store petabytes worth of scientific research data. It is called Storage@Home. I just came across a small article about it but haven't been able to find much more.

"Storage@home is a distributed storage infrastructure developed to solve the problem of backing up and sharing petabytes of scientific results using a distributed model of volunteer managed hosts. Data is maintained by a mixture of replication and monitoring, with repairs done as needed. By the time of publication, the system should be out of testing, in use, and available for volunteer participation."

This was published 6/11/07
[Jan 2, 2009 1:46:16 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: Distribute Storage

Hello Knightro2,
Thanks for pointing out Storage@Home. Here is the December 2008 wiki entry: http://en.wikipedia.org/wiki/Storage@home
Just off the top of my head, I don't think we are ready for this right now. Not in a world where so many people have 10-gigabytes per month communication limits. Obviously that will rapidly change. I still remember when ISDN sounded like a glorious future.

Lawrence
[Jan 2, 2009 4:31:03 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Posts: 15   Pages: 2   [ 1 2 | Next Page ]
[ Jump to Last Post ]
Post new Thread