Scribd
Upload
87 views

Parallel File System

Parallel file systems: - Provide shared access to extremely large datasets across multiple clients concurrently - Distribute data and metadata across multiple storage nodes for high-performance parallel access - Coordinate access to ensure consistency and follow access rules They are designed for high-performance computing applications with massive data and bandwidth needs, distributing data and coordinating parallel access.

Uploaded by

Sachin Anchal
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
87 views

Parallel File System

Parallel file systems: - Provide shared access to extremely large datasets across multiple clients concurrently - Distribute data and metadata across multiple storage nodes for high-performance parallel access - Coordinate access to ensure consistency and follow access rules They are designed for high-performance computing applications with massive data and bandwidth needs, distributing data and coordinating parallel access.

Uploaded by

Sachin Anchal
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 7

PARALLEL FILE SYSTEM

What is parallel file system?

 Store application data persistently


Usually extremely large datasets that can’t fit in memory
 Provide global shared-namespace (files, directories)
 Designed for parallelism
Concurrent (often coordinated) access from many clients
 Designed for high-performance
Operate over high speed networks (IB, Murine, Portals)
Optimized I/O path for maximum bandwidth

Parallel vs. Distributed

How are Parallel File Systems different from Distributed File Systems?

 Data distribution
Distributed file systems often store entire objects (files) on a single storage node
Parallel file systems distribute data of a single object across multiple storage nodes

 Symmetry
Distributed file systems often run on architectures where the storage is co-located with the
application
Parallel file systems are often run on architectures storage is physically separate from the
compute system
(not always true here either)

 Fault Tolerance
Distributed file systems take on fault tolerance responsibilities
Parallel file systems run on enterprise shared storage

 Workloads
Distributed file systems are geared for loosely coupled, distributed applications (think data
intensive)
Parallel file systems target HPC applications, which tend to perform highly coordinated I/O
accesses, and have massive bandwidth requirements

 Overloaded terms!
GlusterFS, Ceph claim to be both
PVFS is often run in symmetric environments
Parallel File Systems

 Provide a directory tree all nodes can see (the global name space)
 Map data across many servers and drives (parallelism of access)
 Coordinate access to data so certain access rules are followed (useful semantics)

Who uses Parallel File Systems?

 Computational Science
o Use of computer simulation as a tool for greater understanding of the real world
– Complements experimentation and theory

o Problems are increasingly computationally challenging


– Large parallel machines needed to perform calculations
– Critical to leverage parallelism in all phases

o Data access is a huge challenge


– Using parallelism to obtain performance
– Finding usable, efficient, portable interfaces
– Understanding and tuning I/O

 Large-Scale Data Sets


o Application teams are beginning to generate 10s of Tbytes of data in a single
simulation. For example, a recent run on 29K processors on the XT4 generated
over 54 Tbytes of data in a 24 hour period.

Application and Storage Data Models

o Applications have data models appropriate to domain


– Multidimensional typed arrays, images composed of scan lines, variable
length records
– Headers, attributes on data
o I/O systems have very simple data models
– Tree based hierarchy of containers
– Some containers have streams of bytes (files)
– Others hold collections of other containers (directories or folders)
o High-level I/O libraries help map between these data models

Shared-file vs. File-per-process

 Scientific applications perform I/O to parallel file system in primarily one of two ways:

 Shared-file(N-to--1): A single file is created, and all application tasks write to that
file (usually to completely disjoint regions)
• Increases usability: only one file to keep of by application
• Can create lock contention and hinder performance on some systems

 File-per-process (N-to-N): Each application task creates a separate file, and


writes to that only that file.
• Avoids lock contention on file systems that use locks to maintain POSIX
consistency
• Applications running today create as many as 100,000 tasks
• Impossible to restart application with different number of tasks
Data distribution in parallel file systems

Data Distribution

 Round round is a reasonable default solution


– Works consistently for a variety of workloads
– Works well on most systems
– Who uses it? GPFS, Lustre, PVFS...
– Can you think of a system where this might not work so well?
– What other distributions could be used?
 Clients perform writes/reads of file at various regions
– Usually depends on application workload and number of tasks

Classes of Parallel File Systems: Blocks vs. Objects

 Block-Based Parallel File Systems (AKA “Shared-disk”)


– Blocks are fixed-width
– File growth requires more blocks
– Blocks distributed over storage nodes
– Suffer from block allocation issues, lock managers
– Example: GPFS

 Object-based Parallel File Systems


– Variable-length regions of the file
– A file has a constant number of objects
– Objects are given global identifiers (object-ids, handles, etc.)
– File growth increases the size of object(s)
– Objects are easier to manage and distribute
– Space allocation is managed locally on a per-object basis
– Examples: Lustre, PVFS

Blocks vs. Objects

 Metadata for a file includes distribution information

 Block-based file systems (Shared-disk) require dynamic metadata for distribution


Information

 Object based file systems only need sta6c-metadata for distribution information

Metadata in Parallel File Systems

 A single metadata server creates a single point of contention (hotspot)


 Many clients try to open the same file at the same time: Creates an
N-to-1 pattern of lookup requests
 Many clients try to create new files at once: Creates an N-to-1
pattern of create requests (requires disk access too!)
 How can metadata be distributed across metadata servers?
 Depends on underlying design (blocks vs. objects)

You might also like