THAT'S why this project is an OPT-IN project (along with the very long duration between checkpoints and several other reasons).

Saul,

Fair enough, an individual file can be better compressed with the choice zipper. A question could be, what's without license / open source, and what comes with BOINC and works on all platforms i.e. when it's time to compress, Windows, Linux, Mac need to follow the same program routine. Don't know if they did tests how fast a current zip versus a 7z file would transit. Transfer protocols seem to waste time on trying to compress what's already compressed. My calculation lastly was that of the total est. up/down of 880GB WCG daily transferred, for CEP2 [compressed] it's 625GB or so (fills the researcher's Jabber storage unit in 200 days). Presently an average 114.6MB are used per core, crunching 24/7 (See the WCG Dashboard chart). Maybe cleanenergy likes to chip in or one of the techs as a 40% reduction surely would have interest.

Whilst, if you never seen 27MB, you ain't seen a full 16 jobs result or the much bigger early in the project. Presently the zip goes 30-31MB on me Windows and Linux cruncher and has been gradually increasing for a long time and as per the past experience and System Requirement page, may grow to 80MB when we get further into the project.

2 Euro cents.

--//--

edit: spellcheck

That's a good idea. I think someone might have raised this before (or maybe I just read a review of zip package systems). Is it cross platform? I definitely remember reading that W7 and Vista were poor in comparison to XP and Linux when it came to zipping and unzipping. So the time taken to zip/unzip, a consideration in itself, does vary, and as pointed out network compression needs to be considered.

The more files that are zipped together, the bigger the file and the greater the chance of upload/download failure, task shortages, and frustration. So smaller sized files would be great.

I remember suggesting that tasks should be allowed to continue uploading from where they left off, if interrupted. Don't know if that was/will be implemented, even if just for some big CEP2 files. Anyone? I only have 100Kb to 130Kb uploads on this system atm or I would test it.

Resuming of transfers has been part of BOINC for a very long time, but WCG doesn't allow resuming of downloads. For uploads on the other hand I don't remember ever having any problems with WCG, and a quick test with a HPF2-upload reveals upload-resuming works as it should (even the file wasn't very large). As for CEP2, I've not got any CEP2 at the moment, but don't remember any problems last time tried upload-resuming here...

gzip is what is built into BOINC. Thus that is what we use for file transfers.

For long term storage, both us and I believe Harvard decompress and re-package the data for longer term storage/transfer/backup/analysis.