Dear MooseFS Users,

We are thrilled to announce the release of the long-awaited MooseFS 4.x Community Edition! 🎉

New Features and Enhancements

New Format of Chunk Storage: Erasure Codes (EC)

We have introduced a new format of chunk storage with Erasure Codes, which can store data in ARCHIVE and TRASH states as 4 or 8 parts with an additional parity part, allowing data restoration in case a part is lost. This method provides data restoration in case of missing parts while optimizing storage space:

• EC8+1: Uses 112.5% of space,
• EC4+1: Uses 125% of space,

while maintaining the same level of hardware failure protection as standard replication (200%).

New Options in Storage Class Definitions

• TRASH State: files in the MooseFS TRASH, so the deleted files that are retained for a configurable period of time, can now have custom storage definitions within a storage class.
• ARCHIVE State Configuration: ARCHIVE state is now fully configurable:
• Set file archive times based on last atime, mtime, or ctime.
• Choose between FAST archival (ASAP) or PER CHUNK based on mtime.
• Exclude files below a certain length from being archived.
• Logical Expressions in Label Definitions: Support for comprehensive logical expressions in label definitions is now available, allowing users to create more flexible and precise rules, providing greater control over how storage classes and states are applied to files. This includes the ability to use negations.
• State-Specific Label Overrides: Override label modes (STD, LOOSE, STRICT) for specific states: CREATE, KEEP, ARCHIVE, or TRASH.

Storage Class Based on File Patterns

Users can now assign file storage classes dynamically based on custom file name patterns, providing greater flexibility in managing data. For example, files with a .backup extension could be assigned to a specific storage class with lower redundancy, while files starting with important_ might be placed in a high-redundancy class.

Broadcast Mechanism for Server Restarts

Server restarts (both chunk server and master) no longer block I/O until the end of chunk registration process.

Improved Chunk Format

Enhanced handling of control sums for more secure chunk storage.

New Web-based Graphical User Interface with Dark Mode

Additional monitoring charts and system insights, offering improved chart systems and more detailed information. Moreover, dark mode is now available.

Multi-LAN Configuration

Support for clients across different LANs to remain within their LAN IP class while communicating with the master server. Can be used alongside the remap option known from MooseFS 3.x.

Enhanced IP Remap

Customizable IP-to-IP remapping instead of just fixed remapping of leading bits in IP addresses.

Detailed Daemon Info

MooseFS daemons instructed by an info signal can now generate detailed info files in working directory, listing configuration, state, and module data for enhanced troubleshooting.

CS-Master Consistency Checks

New mechanism to ensure consistency between master and chunk server data held in RAM.

New mfshdd.cfg Options

Flags with limits and options for disks with chunks shared with other processes, allowing better calculation of available/occupied space on disks.

New Offline Tool: mfsmetasearch

Allows offline searches for files in metadata based on various conditions such as chunk IDs, inodes, and attributes.

Updated Logging System

Clear division of log messages by severity levels, with customizable logging options and syslog reporting.

New ACL Handling Tools

Useful for operating systems without built-in ACL management tools.

General Algorithm Improvements

Numerous improvements to algorithms for faster and more efficient MooseFS performance, especially in edge-case scenarios.

Quality of Life Improvements

• Faster Metadata Loading at master server startup.
• mfssupervisor Tool for on-demand metadata saves.
• mfsfileinfo Tool can now display chunk paths on chunk servers.
• Enhanced Chunk Rebalance Algorithms.
• More Configuration Options for various operations and storage management.

Community Contributions

During the development of this version, we received valuable feedback and suggestions from our community, many of which have been implemented. A huge thank you to everyone who contributed to shaping this release of MooseFS!

MooseFS PRO

If you are looking for high availability (HA) for master servers, improved disk space management with enhanced data protection through EC configurations with higher redundancy levels, and dedicated support, we also offer MooseFS PRO version. This version is designed for enterprise environments where uptime and data integrity are critical, providing features like automatic failover for master servers, more advanced erasure coding for greater resilience against hardware failures, and professional support services to ensure smooth operation and rapid issue resolution. MooseFS PRO adds an extra layer of reliability and optimization for those managing large-scale, mission-critical storage systems.

We recommend to update your MooseFS clusters to this version.

Thank you,
Piotr / MooseFS Team

Dear MooseFS Users,

We are thrilled to announce the release of the long-awaited MooseFS 4.x Community Edition! 🎉

New Features and Enhancements

New Format of Chunk Storage: Erasure Codes (EC)

We have introduced a new format of chunk storage with Erasure Codes, which can store data in ARCHIVE and TRASH states as 4 or 8 parts with an additional parity part, allowing data restoration in case a part is lost. This method provides data restoration in case of missing parts while optimizing storage space:

• EC8+1: Uses 112.5% of space,
• EC4+1: Uses 125% of space,

while maintaining the same level of hardware failure protection as standard replication (200%).

New Options in Storage Class Definitions

• TRASH State: files in the MooseFS TRASH, so the deleted files that are retained for a configurable period of time, can now have custom storage definitions within a storage class.
• ARCHIVE State Configuration: ARCHIVE state is now fully configurable:
• Set file archive times based on last atime, mtime, or ctime.
• Choose between FAST archival (ASAP) or PER CHUNK based on mtime.
• Exclude files below a certain length from being archived.
• Logical Expressions in Label Definitions: Support for comprehensive logical expressions in label definitions is now available, allowing users to create more flexible and precise rules, providing greater control over how storage classes and states are applied to files. This includes the ability to use negations.
• State-Specific Label Overrides: Override label modes (STD, LOOSE, STRICT) for specific states: CREATE, KEEP, ARCHIVE, or TRASH.

Storage Class Based on File Patterns

Users can now assign file storage classes dynamically based on custom file name patterns, providing greater flexibility in managing data. For example, files with a .backup extension could be assigned to a specific storage class with lower redundancy, while files starting with important_ might be placed in a high-redundancy class.

Broadcast Mechanism for Server Restarts

Server restarts (both chunk server and master) no longer block I/O until the end of chunk registration process.

Improved Chunk Format

Enhanced handling of control sums for more secure chunk storage.

New Web-based Graphical User Interface with Dark Mode

Additional monitoring charts and system insights, offering improved chart systems and more detailed information. Moreover, dark mode is now available.

Multi-LAN Configuration

Support for clients across different LANs to remain within their LAN IP class while communicating with the master server. Can be used alongside the remap option known from MooseFS 3.x.

Enhanced IP Remap

Customizable IP-to-IP remapping instead of just fixed remapping of leading bits in IP addresses.

Detailed Daemon Info

MooseFS daemons instructed by an info signal can now generate detailed info files in working directory, listing configuration, state, and module data for enhanced troubleshooting.

CS-Master Consistency Checks

New mechanism to ensure consistency between master and chunk server data held in RAM.

New mfshdd.cfg Options

Flags with limits and options for disks with chunks shared with other processes, allowing better calculation of available/occupied space on disks.

New Offline Tool: mfsmetasearch

Allows offline searches for files in metadata based on various conditions such as chunk IDs, inodes, and attributes.

Updated Logging System

Clear division of log messages by severity levels, with customizable logging options and syslog reporting.

New ACL Handling Tools

Useful for operating systems without built-in ACL management tools.

General Algorithm Improvements

Numerous improvements to algorithms for faster and more efficient MooseFS performance, especially in edge-case scenarios.

Quality of Life Improvements

• Faster Metadata Loading at master server startup.
• mfssupervisor Tool for on-demand metadata saves.
• mfsfileinfo Tool can now display chunk paths on chunk servers.
• Enhanced Chunk Rebalance Algorithms.
• More Configuration Options for various operations and storage management.

Community Contributions

During the development of this version, we received valuable feedback and suggestions from our community, many of which have been implemented. A huge thank you to everyone who contributed to shaping this release of MooseFS!

MooseFS PRO

If you are looking for high availability (HA) for master servers, improved disk space management with enhanced data protection through EC configurations with higher redundancy levels, and dedicated support, we also offer MooseFS PRO version. This version is designed for enterprise environments where uptime and data integrity are critical, providing features like automatic failover for master servers, more advanced erasure coding for greater resilience against hardware failures, and professional support services to ensure smooth operation and rapid issue resolution. MooseFS PRO adds an extra layer of reliability and optimization for those managing large-scale, mission-critical storage systems.

We recommend to update your MooseFS clusters to this version.

Thank you,
Piotr / MooseFS Team

Dear MooseFS Users,

We are happy to announce that MooseFS 3.0.118 is released! 🎉

This is mostly a bugfix release. It also introduces a few improvements and new features, including:

• MooseFS is now cross-build friendly,
• New moosefs man page.

We would like to say big thanks to all the contributors who helped us making this release, especially (alphabetically): @asyslinux, @tianon, @xctan. Thank you for your valuable time, input, and help in making MooseFS better!
(If I forgot to mention somebody, I am sorry for that and please definitely let me know!)

Please find a complete list of changes included in this release below:

• (all) regenerated autoconf files (issue #525)
• (scripts) added simple class that replaces deprecated cgi and cgitb (issue #567)
• (bdev) fixed unmap protocol (issue #551)
• (cgi) added protections against javascript injection
• (all) moved build aux files to separate folder build-aux and made moosefs cross-build friendly
• (all) housekeeping changes (year, regenerated with new autotools, silenced some compiler warnings)
• (all) updated README and README.md files
• (man) added new man page moosefs

We recommend to update your MooseFS clusters to this version.

Thank you,
Piotr / MooseFS Team

Dear MooseFS Users,

We are happy to announce that MooseFS 3.0.117 is released! 🎉

This is mostly a bugfix release. It also introduces a few improvements and new features, including:

• Displaying an error message when a config file for Master Server or Chunkserver cannot be loaded,
• An option to ignore locks for MooseFS Block Device (mfsbdev / nbd),
• A change of default value for ATIME_MODE to 2 (similar to relatime).

We would like to say big thanks to all the contributors who helped us making this release, especially (alphabetically): @alexcrow1974, @alpharde, @borkd, @deltabweb, @eleaner, @footlooseboss, @inkdot7, @jkiebzak, @jSML4ThWwBID69YC, @nickb937, @njaard, @onlyjob, @struthio, @tianon, @zcalusic, @Zorlin. Thank you for your valuable time, input, and help in making MooseFS better!

(If I forgot to mention somebody, I am sorry for that and please definitely let me know!)

Please find a complete list of changes included in this release below:

• (master) fixed handling abnormal operation status (issue #456)
• (master+cs) added error message when a config file can't be loaded (issue #457)
• (client) fixed invalidating directory cache after unlink, rmdir and rename
• (cs) fixed cond variables leaking
• (all) silence silly compiler warnings
• (cgi+cli) fixed division by zero when scount is zero in arch mode (no chunkservers – issue #412)
• (client) fixed freeing atime/mtime data structure
• (client) added displaying build number
• (master) silenced more wrong compiler warnings
• (nbd) added option to ignore locks (issue #353)
• (master) fixed file mode in changelog (problem with mask when parent has default ACLs)
• (master) reduced function stack frame usage (slow down due to use -fstack-clash-protection by some OS'es)
• (master) fixed handling .. and . in fs_lookup (issue #473)
• (all) using SIGUSR1 instead of SIGINFO even on system where SIGINFO is defined
• (master) changed default ATIME_MODE to 2 (issue #478)
• (master) fixed atime/mtime/ctime preservation during snapshot (in HA environment upgrade may cause one time desync; issue #489)
• (client) fixed race condition between client and kernel in retrieving open file list (FreeBSD only)
• (master) fixed bug in sorting chunkservers during selection for chunk creation (issue #481)

We recommend to update your MooseFS clusters to this version.

Best regards,
Piotr / MooseFS Team

Dear MooseFS Users,

We are happy to announce that MooseFS 3.0.116 is released! 🎉

This is mostly a bugfix release. It also introduces a few improvements and new features, including:

• Possibility to set symlink cache timeout (-o mfssymlinkcacheto mounting parameter),
• New INODE_REUSE_DELAY config option for defining inode reuse delay in Master Server (in mfsmaster.cfg),
• Allowing servers in rebalance state to be used in replication,
• Protection against listing too big directory.

We would like to say big thanks to all the contributors who helped us making this release, especially (alphabetically): @BillKen, @Eronana, @jSML4ThWwBID69YC, @onlyjob, @stevew-Purdue, @struthio, @tianon. Thank you for your valuable time, input, and help in making MooseFS better!

(If I forgot to mention somebody, I am sorry for that and please definitely let me know!)

Please find a complete list of changes included in this release below:

• (check) making clock test less prone to a single failure,
• (master) added protection against listing too big directory,
• (master) servers in rebalance state are now allowed to be used in replication,
• (cs) changed handling load+hlstatus packet,
• (mount) added symlink cache timeout,
• (master) added option to define inode reuse delay,
• (cgi+cli) fixed division by zero when scount is zero (no chunkservers – issue #412),
• (master) added inode virtual length for append mode,
• (mount) moved rwlock from descriptor to inode (fixes append issues),
• (cgi) changed method of charts selection in charts comparison,
• (cgi) added message to function that handles exceptions,
• (cgi) added using htmlentities function for various strings,
• (mount) removed old osxfuse patch (doesn't work with current osxfuse and is no longer needed),
• (master) fixed default paths for mfsexports.cfg and mfstopology.cfg,
• (master) fixed restore mode (missing cache initialization – issue #432),
• (mount) added parents of CWDs (current working directories) to sustained inodes (workaround for FreeBSD problem – issue #362),
• (all) fixed spelling,
• (master) fixed handling copy sgid for directories on follower and during restore from changelog,
• (master) fixed changelog order between unlock and close file or session,
• (cgi) fixed using & in javascript URLs (issue #439),
• (cgiserv) fixed error/redirect responses (Pull Request #443 by @Eronana),
• (master+cs) fixed overflows in cgi charts,
• (cs) fixed crc data in info packets (*_CHUNK_CHECKSUM, *_CHUNK_CHECKSUM_TAB).

We recommend to update your MooseFS clusters to this version.

Best regards,
Piotr / MooseFS Team

Dear MooseFS Users,

We are happy to announce that MooseFS 3.0.115 is now stable!

This is a bugfix release. We would like to say big thanks to all the contributors who helped us making this release, especially (alphabetically): @borkd, @eleaner, @richarson. Thank you for your valuable time, input, and help in making MooseFS better!

(If I forgot to mention somebody, I am sorry and please let me know!)

Please find a complete list of fixes and new features that this release includes, below:

• (cs) fixed chunk locking issues in chunkserver (lock timeout, missing release etc.)
• (all) changed to hex all chunkid logged to syslog (only in changelog it is left in decimal format)
• (cs) changed port number from hex to dec in replicator module
• (mount) added printing open/create mode to the .oplog
• (mount) fixed opening file after using lookup cache
• (master) added ignoring empty filenames

We recommend to update your MooseFS cluster(s) to this version.

Best regards,
Piotr / MooseFS Team

Dear MooseFS Users,

We are happy to announce that MooseFS 3.0.114 is now stable!

This is mostly a bugfix release, however it also introduces a few improvements, including:

• Improved internal rebalance algorithm in Chunkserver,
• More information reported in .params special file on MooseFS Client side,
• Changes in job queue policy in Chunkserver,
• Changed time format in MooseFS CGI Monitoring Interface,
• Improvements related to chunks database file (.chunkdb) on Chunkserver (logging, write error detection).

We would like to say big thanks to all the contributors who helped us making this release, especially (alphabetically):
@BillKen, @jSML4ThWwBID69YC, @kopaczew, @onlyjob, @uu.
Thank you for your valuable time, input, and help with making MooseFS better!

(If I forgot to mention somebody, I am sorry and please let me know!)

Please find a complete list of fixes and new features that this release includes, below:

• (mfsio/nbd) fixed compile issue on systems without LOCK_* constants (issue #366)
• (mount) fixed truncating files open with O_APPEND flag (bug introduced in 3.0.113, issue #368)
• (cs) added disk rebalance when all designated source disks are already empty (issue #364)
• (mount) added additional info to .params file (versions of MooseFS, libfuse etc.)
• (master+cs) added check against EACCES after lockf (issue #369)
• (master) fixed access checks in snapshot
• (cgi+cli) removed using supervisor (only available in Pro version)
• (cs) fixed deadlock condition on mutexes folderlock and hashlock (exists since 3.0.110)
• (mount) fixed handling .params file
• (tools) fixed overflow protection in number parsing function
• (cs) fixed reporting to master duplicates with newer version
• (cs) changed job queue policy (some tasks are not limited now)
• (cli) fixed help and manpage
• (master) fixed reaction to status NOTDONE received from chunkservers
• (master) fixed syncing and closing negative file descriptor in bgsaver
• (mount+master) fixed handling keepcache and direct flags (related to issue #374)
• (mount) fixed compiling on systems with old fuse library
• (cgi) changed time format (issue #197)
• (cs) added logging info when .chunkdb is not written to disk
• (master) fixed alphabetical order of commands detected in changelog
• (cs) changed queue limit to max workers (limited dynamically)
• (cs) fixed automatic chunkserver removal in master
• (master) added meta version increment in chunks_set_version
• (cs) added error detection during writing .chunkdb
• (all) added build id to what strings
• (all) added missing idstr.h and rearranging source lists in Makefiles
• (nbd+cgiserv) added chdir("/")
• (freebsd) updated rc.d master script
• (man) updated mfsmetadump manpage
• (man) fixed formatting in mfsmount manpage

Best regards,
Piotr / MooseFS Team

Dear MooseFS Users,

We are happy to announce that MooseFS 3.0.113 is now stable!

This is both bugfix and feature release. Apart from fixing the issues that were recently found, it introduces:

• Support for new extra attributes: immutable, appendonly, undeletable.
  Extra attributes are MooseFS-specific feature that allows you to add / manage a small portion of metadata to files on MooseFS. There are several extra attributes and they change file system behaviour (apart from new ones mentioned above there are several more – please refer to man mfseattr for more information).
• Topology grouping when new chunks are about to be created,
• Readonly mode and locking in MooseFS Block Device,
• Changing subfolder during internal rebalance,
• Official labels format defined in chunkserver configuration,

and several more features.

This release improves Chunkserver a lot (registering chunks to Master Server, reporting damaged chunks, internal rebalance, reloading, calculating size limits, handling inode limits on underlying file systems). It also adds new and improves existing workarounds to issues related to OS's kernels and improves directory entry invalidator introduced in MooseFS 3.0.112.

We would like to thank all the contributors who helped us making this release, especially (alphabetically):
@borkd, @eleaner, @jSML4ThWwBID69YC, @onlyjob, @rafamiga, @tamas-ac.
Thank you for your valuable time, input, and help with making MooseFS better!

(If I forgot to mention somebody, I am sorry and please let me know!)

Please find a complete list of fixes and new features that this release includes, below:

• (master) removed unnecessary debug syslog messages
• (check) increased usleep tolerance due to some operating systems (issue #351)
• (master) changed condition that decides if master should wait for more chunks during I/O
• (mount) delayed setting channel for fuse notifications (very rare segfault in libfuse2 during init)
• (client+master) changed conditions for ancestor test in getattr (issue #350)
• (cs) added parameter for number of chunks to be send in single register packet
• (cs) fixed reporting damaged chunks in testing function (issue #352)
• (cs) removed some unnecessary damaged chunk notifications
• (client) changed open test in setattr (related to issue #350)
• (master) fixed handling truncate for open files (related to issue #350)
• (master) added uid mapping in setfacl function
• (master+client) added support for atomic truncate with open
• (master+client) fixed keep cache conditions
• (cs) introduced official label format defined in chunkserver configuration
• (nbd) added readonly mode and locking (issue #353)
• (cs) fixed condition in choosing disks for internal rebalance
• (mount) added workaround in access for a bug in FreeBSD kernel (issue #354)
• (cs) fixed master reconnection conditions (reload usually shouldn't cause reconnection)
• (freebsd) fixed FreeBSD port (makeports.sh)
• (master) changed disk removal detection algorithm (issue #356)
• (cs) fixed calculating size limits
• (cs) added handling inode limits on local filesystems on chunkservers (issue #358)
• (master) added topology grouping when new chunks are about to be created (prefer closer servers)
• (master) added more error messages to bgsaver
• (cs) added changing subfolder during internal rebalance (related to issue #326)
• (master+cs) added optional logging of long function execution times
• (all) fixes of small errors found by static code analysers
• (mount) turn off dentry invalidator for Linux kernels >= 4.19 (related to issue #357)
• (master) changed condition that checks timestamp in changelogs (less prone to small clock differences)
• (daemons) added time refresh function (for future use)
• (cgi+cli) changed mark for removal state name UNKNOWN –> PENDING (related to issue #359)
• (client) decreased max IDLE time in writer module from 1s to 0.1s
• (client) update inode in dentry invalidator
• (tools) fixed mfscopyeattr, mfsseteattr and mfs*archive tools
• (master+client) added support for new eattrs: immutable, appendonly, undeletable (setting new undeletable extra attribute appropriately can help with issues like #357)
• (client) silenced kern.proc.filedesc syslog messages on FreeBSD (issue #360)

Best regards,
Piotr / MooseFS Team

Dear MooseFS Users,

MooseFS 3.0.112 is now stable!

This release is a bugfix and feature release. It introduces support for disabling individual filesystem commands in Master Server (configurable in mfsexports.cfg) and respecting them on the Client side (issue #344). It also adds support for STRICT and LOOSE mode in KEEP and ARCHIVE (issue #249). This release fixes various other issues too, including a workaround for Linux kernel bug related to EBUSY issue by adding directory entry invalidator (issue #98).

We would like to thank all the contributors who helped us making this release, especially (alphabetically):
@borkd, @Damien9527, @linux-ops, @MonkeyFang, @onlyjob, @pault28, @tnktls.
Thank you for all your valuable input, help and making MooseFS better!

(If I forgot to mention somebody, I am very sorry and please let me know!)

Please find a complete list of fixes and new features that this release includes below:

• (cs) silence stupid compiler warning
• (client) fixed handling LOCKED and EAGAIN status in readdata
• (client) added handle info to .oplog messages
• (all) added support for disabling individual filesystem commands in mfsexports.cfg
• (client) added handling read/write/readdir disables on client side (better error messages)
• (client) added session paramaters to .params file
• (tools) fixed packet reallocation error
• (master) changed syslog message for locked chunks
• (client) added dentry invalidator (needed in Linux with kernel < 4.19 – EBUSY issue)
• (master) fixed memory leaks in xattr and posixacl modules
• (cs) when chunk can't be located always send to master info about lost chunk
• (supervisor, master, cs) fixed buffer overrun in mastersupervisor code (HA only, intr. in 3.0.107)
• (master) changed algorithm of reusing csid in chunk module (adding released csid to the end of free list)
• (all) added support for STRICT and LOOSE mode in KEEP and ARCHIVE
• (master) fixed segfault during appending chunks of file with positive length and no chunks
• (metadump) fixed bugs introduced in version 3.0.106
• (master) added protection between never-ending desyncing between newer LEADER and older FOLLOWER (HA only)
• (master) fixed slices with to set to 0 in mfsappendchunks
• (master) removed starting protection time from client communication module
• (man) fixed typo in mfsappendchunks man page

Best regards,
Piotr / MooseFS Team

Dear MooseFS Users,

MooseFS 3.0.111 is now stable! This release adds protections against segfaults in MooseFS Chunkserver module (issue #342).

We would like to thank @onlyjob and @uu for reporting the issue. Thank you for all your valuable input, help and making MooseFS better!

Please find a complete list of fixes and new features that this release includes below:

• (cs) added protections against segfaults (issue #342)
• (cs) removed unnecessary error messages

Best regards,
Piotr / MooseFS Team