CSV file generation seemingly not flushing data #256
Description
The code I use for CSV generation is randomly failing to flush/sync data to disk upon return from csvsave—stating the file immediately after returns size=0 .
It happens when processing a large number of files in quick succession. I'm wondering if anything has changed recently either in RubyVIPS or libvips that could cause this to happen? Maybe some concurrency settings, perhaps? I haven't changed any code. I have upgraded libvips from 8.6.3 to 8.10.2, and ruby vips from 2.0.16 to 2.0.17.
This is a trace from the application logs immediately after calling csvsave. You can see sometimes size is non-zero and sometimes it's 0
❯ grep "AFTER SAVE" log/development.log
AFTER SAVE: #<File::Stat dev=0x1000008, ino=10503482, mode=0100644, nlink=1, uid=501, gid=20, rdev=0x0, size=280460, blksize=4096, blocks=552, atime=2020-11-04 16:13:53.860700194 +1100, mtime=2020-11-04 16:13:53.898899419 +1100, ctime=2020-11-04 16:13:53.898899419 +1100, birthtime=2020-11-04 16:13:53.860700194 +1100>
AFTER SAVE: #<File::Stat dev=0x1000008, ino=10503727, mode=0100644, nlink=1, uid=501, gid=20, rdev=0x0, size=0, blksize=4096, blocks=0, atime=2020-11-04 16:13:54.465449389 +1100, mtime=2020-11-04 16:13:54.465449389 +1100, ctime=2020-11-04 16:13:54.465449389 +1100, birthtime=2020-11-04 16:13:54.465449389 +1100>
AFTER SAVE: #<File::Stat dev=0x1000008, ino=10523907, mode=0100644, nlink=1, uid=501, gid=20, rdev=0x0, size=0, blksize=4096, blocks=0, atime=2020-11-04 16:17:04.650600852 +1100, mtime=2020-11-04 16:17:04.650600852 +1100, ctime=2020-11-04 16:17:04.650600852 +1100, birthtime=2020-11-04 16:17:04.650600852 +1100>
AFTER SAVE: #<File::Stat dev=0x1000008, ino=10523909, mode=0100644, nlink=1, uid=501, gid=20, rdev=0x0, size=0, blksize=4096, blocks=0, atime=2020-11-04 16:17:05.003626941 +1100, mtime=2020-11-04 16:17:05.003626941 +1100, ctime=2020-11-04 16:17:05.003626941 +1100, birthtime=2020-11-04 16:17:05.003626941 +1100>
AFTER SAVE: #<File::Stat dev=0x1000008, ino=10523912, mode=0100644, nlink=1, uid=501, gid=20, rdev=0x0, size=0, blksize=4096, blocks=0, atime=2020-11-04 16:17:06.20519268 +1100, mtime=2020-11-04 16:17:06.20519268 +1100, ctime=2020-11-04 16:17:06.20519268 +1100, birthtime=2020-11-04 16:17:06.20519268 +1100>
AFTER SAVE: #<File::Stat dev=0x1000008, ino=10523914, mode=0100644, nlink=1, uid=501, gid=20, rdev=0x0, size=0, blksize=4096, blocks=0, atime=2020-11-04 16:17:06.355915569 +1100, mtime=2020-11-04 16:17:06.355915569 +1100, ctime=2020-11-04 16:17:06.355915569 +1100, birthtime=2020-11-04 16:17:06.355915569 +1100>
AFTER SAVE: #<File::Stat dev=0x1000008, ino=10523919, mode=0100644, nlink=1, uid=501, gid=20, rdev=0x0, size=0, blksize=4096, blocks=0, atime=2020-11-04 16:17:06.579407921 +1100, mtime=2020-11-04 16:17:06.579407921 +1100, ctime=2020-11-04 16:17:06.579407921 +1100, birthtime=2020-11-04 16:17:06.579407921 +1100>
AFTER SAVE: #<File::Stat dev=0x1000008, ino=10523923, mode=0100644, nlink=1, uid=501, gid=20, rdev=0x0, size=0, blksize=4096, blocks=0, atime=2020-11-04 16:17:07.693460598 +1100, mtime=2020-11-04 16:17:07.693460598 +1100, ctime=2020-11-04 16:17:07.693460598 +1100, birthtime=2020-11-04 16:17:07.693460598 +1100>
AFTER SAVE: #<File::Stat dev=0x1000008, ino=10523925, mode=0100644, nlink=1, uid=501, gid=20, rdev=0x0, size=0, blksize=4096, blocks=0, atime=2020-11-04 16:17:07.811673086 +1100, mtime=2020-11-04 16:17:07.811673086 +1100, ctime=2020-11-04 16:17:07.811673086 +1100, birthtime=2020-11-04 16:17:07.811673086 +1100>
AFTER SAVE: #<File::Stat dev=0x1000008, ino=10523927, mode=0100644, nlink=1, uid=501, gid=20, rdev=0x0, size=119000, blksize=4096, blocks=240, atime=2020-11-04 16:17:07.888796708 +1100, mtime=2020-11-04 16:17:07.919291491 +1100, ctime=2020-11-04 16:17:07.919291491 +1100, birthtime=2020-11-04 16:17:07.888796708 +1100>
AFTER SAVE: #<File::Stat dev=0x1000008, ino=10523931, mode=0100644, nlink=1, uid=501, gid=20, rdev=0x0, size=203993, blksize=4096, blocks=400, atime=2020-11-04 16:17:08.270614213 +1100, mtime=2020-11-04 16:17:08.298556074 +1100, ctime=2020-11-04 16:17:08.298556074 +1100, birthtime=2020-11-04 16:17:08.270614213 +1100>
AFTER SAVE: #<File::Stat dev=0x1000008, ino=10523933, mode=0100644, nlink=1, uid=501, gid=20, rdev=0x0, size=0, blksize=4096, blocks=0, atime=2020-11-04 16:17:08.557670658 +1100, mtime=2020-11-04 16:17:08.557670658 +1100, ctime=2020-11-04 16:17:08.557670658 +1100, birthtime=2020-11-04 16:17:08.557670658 +1100>
AFTER SAVE: #<File::Stat dev=0x1000008, ino=10523935, mode=0100644, nlink=1, uid=501, gid=20, rdev=0x0, size=0, blksize=4096, blocks=0, atime=2020-11-04 16:17:08.700195289 +1100, mtime=2020-11-04 16:17:08.700195289 +1100, ctime=2020-11-04 16:17:08.700195289 +1100, birthtime=2020-11-04 16:17:08.700195289 +1100>
AFTER SAVE: #<File::Stat dev=0x1000008, ino=10523937, mode=0100644, nlink=1, uid=501, gid=20, rdev=0x0, size=118999, blksize=4096, blocks=240, atime=2020-11-04 16:17:08.790993354 +1100, mtime=2020-11-04 16:17:08.816819317 +1100, ctime=2020-11-04 16:17:08.816819317 +1100, birthtime=2020-11-04 16:17:08.790993354 +1100>
AFTER SAVE: #<File::Stat dev=0x1000008, ino=10523943, mode=0100644, nlink=1, uid=501, gid=20, rdev=0x0, size=0, blksize=4096, blocks=0, atime=2020-11-04 16:17:09.593798011 +1100, mtime=2020-11-04 16:17:09.593798011 +1100, ctime=2020-11-04 16:17:09.593798011 +1100, birthtime=2020-11-04 16:17:09.593798011 +1100>