| Index | Recent Threads | Unanswered Threads | Who's Active | Guidelines | Search |
 |
World Community Grid Forums
Category: Active Research
Forum: OpenPandemics - COVID-19 Project
Thread: One terabyte written to my SSD since the stress test began |
| No member browsing this thread |
|
Thread Status: Active Total posts in this thread: 81
|
|
| Author |
|
|
Andyman
Cruncher Joined: Apr 9, 2021 Post Count: 17 Status: Offline Project Badges: 
|
This method seemed easier
https://boinc.berkeley.edu/forum_thread.php?id=13012 Exit BOINC (BOINC Manager->File->Exit BOINC Manager->check "Stop running tasks when exiting the BOINC Manager"->OK) Navigate to your BOINC Data directory (default in Windows a hidden directory at C:\Programdata\BOINC), then go up one folder so you can copy the BOINC folder. Copy the BOINC folder to a folder on your D: drive (e.g. D:\BOINC) Uninstall BOINC. Reinstall BOINC. In the third screen of the installer, click Advanced. Change the path for the Data Directory from C:\Programdata\BOINC to D:\BOINC (or wherever you copied the directory to). Continue installing BOINC. Start BOINC Manager (Start->All Programs->BOINC->BOINC Manager) We do it this way and not via the registry to make sure that all permissions on the new BOINC folder are set correctly. If you miss the installer of BOINC, you can get 7.14.2 from here. If you have trouble uninstalling or reinstalling BOINC, use the Microsoft Windows Program Uninstaller. The only thing I'd suggest for a project like CPDN is to take it slowly - especially at the first step, "Stop running tasks when exiting the BOINC Manager". CPDN can take a little time to flush everything to disk and shut down the applications in an orderly fashion. |
||
|
|
GunZi
Senior Cruncher Joined: Apr 1, 2020 Post Count: 160 Status: Offline Project Badges: 
|
Woah I'm seeing a similar thing. 11-20MB/s...
|
||
|
|
spRocket
Senior Cruncher Joined: Mar 25, 2020 Post Count: 277 Status: Offline Project Badges: 
|
Based on the reports, I can't help but wonder if this is a Windows-specific problem, and/or perhaps ZFS is doing some better caching behind the scenes that mitigates the issue?
----------------------------------------FWIW, my main cruncher is on Ubuntu 18.04.5 LTS, running kernel 4.15.0-142-generic with the stock ZFS module supplied by Canonical. Is there anyone with BSD, Mac, or Linux with other kernels/filesystems who can chime in? Meanwhile, I tried zpool iostat -v 60 for a 60-second interval, and once again saw 3.9-9.3 MB/s (varying as time went on) per drive, as before with two GPU tasks (edit: Nvidia with a GTX 960) and 14 CPU tasks. The load on the storage pool is not constant, but it seems that 7-8 MB/s per mirrored drive is common. Not as horrible as 20+, but I'm still glad it's on spinning rust. [Edit 1 times, last edit by spRocket at Apr 30, 2021 9:10:16 AM] |
||
|
|
biini
Senior Cruncher Finland Joined: Jan 25, 2007 Post Count: 334 Status: Offline Project Badges: 
|
For those running windows 10 and frustrated with this... I have been able to successfully migrate my Data Directory to an old spinning Disk... Try the following at your own risk... 1. Exit BONIC - and make sure it is not running in the system tray. 2. Edit the registry: HKEY_LOCAL_MACHINE\SOFTWARE\Space Sciences Laboratory, U.C. Berkeley\BOINC Setup\DATADIR --Change the Data value to the path you WANT to move your data to i.e. E:\BONIC\ 3. LAUNCH Bonic - it will error out, but in the process will build out the E:\BONIC folder, which must have some special permissions, as if you manually create it, it won't use it... 4. EXIT Bonic and again make sure it is not in the system tray 5. Browse to C:\Programdata\BONIC and copy all files and folders to your target directory. ie. (E:\BONIC) and overwrite the files in that location! Note: c:\programdata is hidden, just copy/paste the folder path or just type it in... you'll get there... 6. Re-Launch Bonic - it will take a minute to read the data as it is SLOWER than your SSD... 7. Open Task manager and select the performance tab and validate you now see your HDD get busy... My NVME is only about a 18 months old, but as per Magician, I have already hit 87TB of writes... so this had to be figured out... Hope this helps at least one SSD owner make it through the pandemic. I think this is a bit easier.. 1) Shutdown boinc and all running instances 2) move c:\programdata\boinc to desired location (ie d:\boinc) 3) mklink /j c:\programdata\boinc d:\boinc ---------------------------------------- [Edit 1 times, last edit by biini at Apr 30, 2021 9:30:31 AM] |
||
|
|
tux93
Cruncher Germany Joined: Jan 5, 2012 Post Count: 9 Status: Offline Project Badges: 
|
Is there anyone with BSD, Mac, or Linux with other kernels/filesystems who can chime in? Not as horrible as 20+, but I'm still glad it's on spinning rust. # iostat -hd 60 /dev/sdc That is with two GPU tasks on a GTX 750 Ti and five additional CPU tasks iotop -a shows this: Total DISK READ : 0.00 B/s | Total DISK WRITE : 0.00 B/s  Primary: Intel i7-4790 + nVidia GTX 1060 Secondary: Intel i7-2600 + nVidia GTX 750 Ti OS: openSUSE Tumbleweed [Edit 1 times, last edit by tux93 at Apr 30, 2021 9:32:42 AM] |
||
|
|
goben_2003
Advanced Cruncher Joined: Jun 16, 2006 Post Count: 146 Status: Offline Project Badges: 
|
I noticed the high writes during the beta. I switched to running off of a ramdisk since I have plenty of ram. I use ImDisk. In case it helps anywone, here is my post from another thread of how I set it up:
----------------------------------------Note: If you are short on RAM, you should be able to save most of the writes with less RAM by only having the slots directory in memory. To do this, follow the guide below except instead of with the c:\ProgramData\BOINC folder, do it with c:\ProgramData\BOINC\slots folder. Note: It is a good idea to give a little time to ensure the ramdisk finish initializes before BOINC starts, especially if you have a large ramdisk and/or put the whole BOINC data folder on it. Short version of one way to do that is after the ImDisk instructions. Note: If you experience a sudden power loss, you will lose your downloaded work and work in progress(if whole BOINC folder is on ramdisk) or just the work in progress(if only the slots folder is on ramdisk). I run this on a laptop, so it has a battery backup and does not experience this issue. Data is synced to disk on shutdown or restart. If you have a UPS hooked up to a desktop that triggers shutdown, the data should be saved to disk. Note for if you put the whole BOINC data folder on the ramdisk: Some BOINC settings need to be adjusted to ensure the whole ramdisk is usable by BOINC. It can be done locally or in the profile you have assigned to that device. "Use no more than:" "Gigabytes of disk space" This should be set larger than your ramdisk. "Leave at least:" 0.0 "Gigabytes of disk space free" "Use no more than:" 100.0 "% of total disk space" Yeah, it really does thrash the SSD. That is why I switched to running off of a ramdisk that syncs to disk on shutdown/reboot. It definitely cut down the writing to my SSD a lot. Would you mind sharing how you set that up? I mean I probably could stick something together on my own but why reinvent the wheel if there's a working solution already ^^ I'm also wondering if it would be enough to have the ./slots dir in memory or it has the whole boinc working directory I installed ImDisk. IIRC, there is an option about a service in the install. This is required for it to get the shutdown/restart notification and sync to disk. Here are the steps I followed to set it up: 1. Close all programs that might be accessing BOINC. (should be c:\ProgramData\BOINC) 2. Rename BOINC to BOINC.imdisk (the actual name is not important as long as it's different). 3. Create a new directory named BOINC in the same location. 4. Run Imdisk configuration 4a. Set size to what you want. Note that the GPU workunits require a lot more space to be allowed to download than they actually take up. So definitely oversize it otherwise you will get messages in the log that there is not enough disk space. Note: "Allocate Memory Dynamically" is no longer recommended as it may be causing issues. Just putting the slots folder in RAM with "Allocate Memory Dynamically" unchecked is probably a better idea if you are short on RAM. You can use the "Allocate Memory Dynamically" option checked if you do not have a ton of ram, so the ram is only used if the space actually gets used. 4b. Launch at Windows Startup 4c. I picked a drive letter, but I do not know that it is required since I used a folder mount point instead of as a drive, so it does not show up as a drive. 4d. Advanced -> Use Mount Point -> set to your boinc folder (should be c:\ProgramData\BOINC) 4e. Load Content from Image File or Folder: set this to the folder that you renamed BOINC to (c:\ProgramData\BOINC.imdisk in my example) 4f. Click OK You can resize later by using the ImDisk Virtual Disk Driver (It is in the Start Menu and is in the control panel). I have only used it to increase the size. It normally works, but one time froze, so I normally change the size with boinc shutdown and the drive saved. There are many ways to make BOINC startup with a delay. I use scheduled tasks. The general gist of how to do with scheduled tasks is: scheduled tasks->Create Task Name it whatever you want, I used "BOINC" Run only when user is logged in Triggers tab -> At log on any user delay task for 1 minute enabled Actions tab -> New -> Start a program, Program/script = "C:\Program Files\BOINC\boincmgr.exe" Conditions tab -> I do not have anything checked If anyone needs more details, I will try to do a more detailed write up. edit: Edited to add the step to create the empty BOINC directory. Thanks to gibbcorp for pointing it out. edit: Added a note about just putting the slots directory in ram for people who want to use less ram and still save most of the writes. Thanks to highCon for pointing out that some people may want to use less RAM. edit: Added note about what happens if you suddenly lose power to the computer. edit: Added a warning about the "Allocate Memory Dynamically" option edit: Added a note about BOINC disk settings. Thanks to MJH333 for pointing out that BOINC was not using the whole ramdisk with them not set appropriately for the ramdisk. Changed to note that it only applies if whole BOINC data folder is on the ramdisk. edit: Changed power loss note to reflect that losing the slots only loses the work in progress. edit: Added note about starting BOINC with a delay to allow time for ramdisk to finish initializing.  [Edit 8 times, last edit by goben_2003 at May 4, 2021 3:52:59 PM] |
||
|
|
gibbcorp
Advanced Cruncher Joined: Nov 29, 2005 Post Count: 77 Status: Offline Project Badges: 
|
Hi Goben. I have tried this but I get the error "no empty folder selected". Did you make another BOINC folder? c: BOINC and c: BOINC.imdisk seperately? thanks
|
||
|
|
spRocket
Senior Cruncher Joined: Mar 25, 2020 Post Count: 277 Status: Offline Project Badges:
|
Also, if you're running Linux, there's a ramdisk driver built-in. Let's say you wanted to carve out 10 GB for a ramdisk:
----------------------------------------mkdir /tmp/ramdisk Note that "playground" is a name you choose. Also, /tmp itself is a ramdisk on modern Linux systems. If you want it elsewhere, you need to create a directory as a mount point, as is standard practice on Unix-like systems. You can also add a line to /etc/fstab to create such a ramdisk at boot time: playground /tmp/ramdisk tmpfs defaults,size=10G 0 0 If you're going to put a ramdisk on /var/lib/boinc-client, be sure to set something up to save and restore its contents when you reboot. There is also a zram device driver, which allows you to create a compressed ramdisk. It's a little more complicated to use , but might be useful if RAM is tight. [Edit 2 times, last edit by spRocket at Apr 30, 2021 1:24:30 PM] |
||
|
|
goben_2003
Advanced Cruncher Joined: Jun 16, 2006 Post Count: 146 Status: Offline Project Badges:
|
Hi Goben. I have tried this but I get the error "no empty folder selected". Did you make another BOINC folder? c: BOINC and c: BOINC.imdisk seperately? thanks Hi gibbcorp. I must have missed a step when I typed it up. My apologies. A line like this should have been between what was 2. and 3. Create new directory named BOINC in the same location where the old one was. edit: I edited the post above to add that step. [Edit 1 times, last edit by goben_2003 at Apr 30, 2021 2:51:56 PM] |
||
|
|
Penguin
Cruncher Joined: Feb 17, 2014 Post Count: 8 Status: Offline Project Badges:
|
Constant checkpoint writing needs to be corrected. There should be no need to have to create RAM drives and separate directories on secondary drives. No one wants to burn out an SSD running this, myself included.
So to the programmers - REDUCE CHECKPOINTING in these GPU tasks. Thanks. |
||
|
|
|