Currently the “.partial” temporary files are written in a way that usually end up in file fragmentation (block > 2MB generate between 2-10 fragments) and on big nodes with millions of files created / deleted it add ups.
On windows for example (sorry the only platform I know well enough) you can pre-alocate a file with 0 on creation. This force the file system to try to allocate the file on a contiguous space and minimise fragmentation
API call used:
if ( INVALID_HANDLE_VALUE != (handle=CreateFile(fileName,GENERIC_WRITE,0,0,CREATE_ALWAYS,FILE_FLAG_SEQUENTIAL_SCAN,NULL) )) {
//preallocate 2Gb disk file
LARGE_INTEGER size;
size.QuadPart=2048 * 0x10000;
::SetFilePointerEx(handle,size,0,FILE_BEGIN);
::SetEndOfFile(handle);
::SetFilePointer(handle,0,0,FILE_BEGIN);
}
SetFileValidData()
can also be used, not sure if better.
With the system tools:
fsutil file createnew filename filesize
Ideally he client shoud be prealocating the file to his final size and not an arbitrary one (4MB default)