Skip to content

Commit a682e00

Browse files
torvaldstorvalds
committed
Merge branch 'for-next' of git://git.kernel.org/pub/scm/linux/kernel/git/shli/md
Pull md updates from Shaohua Li: "Mainly fixes bugs and improves performance: - Improve scalability for raid1 from Coly - Improve raid5-cache read performance, disk efficiency and IO pattern from Song and me - Fix a race condition of disk hotplug for linear from Coly - A few cleanup patches from Ming and Byungchul - Fix a memory leak from Neil - Fix WRITE SAME IO failure from me - Add doc for raid5-cache from me" * 'for-next' of git://git.kernel.org/pub/scm/linux/kernel/git/shli/md: (23 commits) md/raid1: fix write behind issues introduced by bio_clone_bioset_partial md/raid1: handle flush request correctly md/linear: shutup lockdep warnning md/raid1: fix a use-after-free bug RAID1: avoid unnecessary spin locks in I/O barrier code RAID1: a new I/O barrier implementation to remove resync window md/raid5: Don't reinvent the wheel but use existing llist API md: fast clone bio in bio_clone_mddev() md: remove unnecessary check on mddev md/raid1: use bio_clone_bioset_partial() in case of write behind md: fail if mddev->bio_set can't be created block: introduce bio_clone_bioset_partial() md: disable WRITE SAME if it fails in underlayer disks md/raid5-cache: exclude reclaiming stripes in reclaim check md/raid5-cache: stripe reclaim only counts valid stripes MD: add doc for raid5-cache Documentation: move MD related doc into a separate dir md: ensure md devices are freed before module is unloaded. md/r5cache: improve journal device efficiency md/r5cache: enable chunk_aligned_read with write back cache ...
2 parents 1802979 + 1ec4922 commit a682e00

File tree

20 files changed

+942
-352
lines changed

20 files changed

+942
-352
lines changed

‎Documentation/00-INDEX

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -270,8 +270,8 @@ m68k/
270270
- directory with info about Linux on Motorola 68k architecture.
271271
mailbox.txt
272272
- How to write drivers for the common mailbox framework (IPC).
273-
md-cluster.txt
274-
- info on shared-device RAID MD cluster.
273+
md/
274+
- directory with info about Linux Software RAID
275275
media/
276276
- info on media drivers: uAPI, kAPI and driver documentation.
277277
memory-barriers.txt

‎Documentation/admin-guide/md.rst

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -725,3 +725,8 @@ These currently include:
725725
to 1. Setting this to 0 disables bypass accounting and
726726
requires preread stripes to wait until all full-width stripe-
727727
writes are complete. Valid values are 0 to stripe_cache_size.
728+
729+
journal_mode (currently raid5 only)
730+
The cache mode for raid5. raid5 could include an extra disk for
731+
caching. The mode can be "write-throuth" and "write-back". The
732+
default is "write-through".

‎Documentation/md-cluster.txt renamed to ‎Documentation/md/md-cluster.txt

File renamed without changes.

‎Documentation/md/raid5-cache.txt

Lines changed: 109 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,109 @@
1+
RAID5 cache
2+
3+
Raid 4/5/6 could include an extra disk for data cache besides normal RAID
4+
disks. The role of RAID disks isn't changed with the cache disk. The cache disk
5+
caches data to the RAID disks. The cache can be in write-through (supported
6+
since 4.4) or write-back mode (supported since 4.10). mdadm (supported since
7+
3.4) has a new option '--write-journal' to create array with cache. Please
8+
refer to mdadm manual for details. By default (RAID array starts), the cache is
9+
in write-through mode. A user can switch it to write-back mode by:
10+
11+
echo "write-back" > /sys/block/md0/md/journal_mode
12+
13+
And switch it back to write-through mode by:
14+
15+
echo "write-through" > /sys/block/md0/md/journal_mode
16+
17+
In both modes, all writes to the array will hit cache disk first. This means
18+
the cache disk must be fast and sustainable.
19+
20+
-------------------------------------
21+
write-through mode:
22+
23+
This mode mainly fixes the 'write hole' issue. For RAID 4/5/6 array, an unclean
24+
shutdown can cause data in some stripes to not be in consistent state, eg, data
25+
and parity don't match. The reason is that a stripe write involves several RAID
26+
disks and it's possible the writes don't hit all RAID disks yet before the
27+
unclean shutdown. We call an array degraded if it has inconsistent data. MD
28+
tries to resync the array to bring it back to normal state. But before the
29+
resync completes, any system crash will expose the chance of real data
30+
corruption in the RAID array. This problem is called 'write hole'.
31+
32+
The write-through cache will cache all data on cache disk first. After the data
33+
is safe on the cache disk, the data will be flushed onto RAID disks. The
34+
two-step write will guarantee MD can recover correct data after unclean
35+
shutdown even the array is degraded. Thus the cache can close the 'write hole'.
36+
37+
In write-through mode, MD reports IO completion to upper layer (usually
38+
filesystems) after the data is safe on RAID disks, so cache disk failure
39+
doesn't cause data loss. Of course cache disk failure means the array is
40+
exposed to 'write hole' again.
41+
42+
In write-through mode, the cache disk isn't required to be big. Several
43+
hundreds megabytes are enough.
44+
45+
--------------------------------------
46+
write-back mode:
47+
48+
write-back mode fixes the 'write hole' issue too, since all write data is
49+
cached on cache disk. But the main goal of 'write-back' cache is to speed up
50+
write. If a write crosses all RAID disks of a stripe, we call it full-stripe
51+
write. For non-full-stripe writes, MD must read old data before the new parity
52+
can be calculated. These synchronous reads hurt write throughput. Some writes
53+
which are sequential but not dispatched in the same time will suffer from this
54+
overhead too. Write-back cache will aggregate the data and flush the data to
55+
RAID disks only after the data becomes a full stripe write. This will
56+
completely avoid the overhead, so it's very helpful for some workloads. A
57+
typical workload which does sequential write followed by fsync is an example.
58+
59+
In write-back mode, MD reports IO completion to upper layer (usually
60+
filesystems) right after the data hits cache disk. The data is flushed to raid
61+
disks later after specific conditions met. So cache disk failure will cause
62+
data loss.
63+
64+
In write-back mode, MD also caches data in memory. The memory cache includes
65+
the same data stored on cache disk, so a power loss doesn't cause data loss.
66+
The memory cache size has performance impact for the array. It's recommended
67+
the size is big. A user can configure the size by:
68+
69+
echo "2048" > /sys/block/md0/md/stripe_cache_size
70+
71+
Too small cache disk will make the write aggregation less efficient in this
72+
mode depending on the workloads. It's recommended to use a cache disk with at
73+
least several gigabytes size in write-back mode.
74+
75+
--------------------------------------
76+
The implementation:
77+
78+
The write-through and write-back cache use the same disk format. The cache disk
79+
is organized as a simple write log. The log consists of 'meta data' and 'data'
80+
pairs. The meta data describes the data. It also includes checksum and sequence
81+
ID for recovery identification. Data can be IO data and parity data. Data is
82+
checksumed too. The checksum is stored in the meta data ahead of the data. The
83+
checksum is an optimization because MD can write meta and data freely without
84+
worry about the order. MD superblock has a field pointed to the valid meta data
85+
of log head.
86+
87+
The log implementation is pretty straightforward. The difficult part is the
88+
order in which MD writes data to cache disk and RAID disks. Specifically, in
89+
write-through mode, MD calculates parity for IO data, writes both IO data and
90+
parity to the log, writes the data and parity to RAID disks after the data and
91+
parity is settled down in log and finally the IO is finished. Read just reads
92+
from raid disks as usual.
93+
94+
In write-back mode, MD writes IO data to the log and reports IO completion. The
95+
data is also fully cached in memory at that time, which means read must query
96+
memory cache. If some conditions are met, MD will flush the data to RAID disks.
97+
MD will calculate parity for the data and write parity into the log. After this
98+
is finished, MD will write both data and parity into RAID disks, then MD can
99+
release the memory cache. The flush conditions could be stripe becomes a full
100+
stripe write, free cache disk space is low or free in-kernel memory cache space
101+
is low.
102+
103+
After an unclean shutdown, MD does recovery. MD reads all meta data and data
104+
from the log. The sequence ID and checksum will help us detect corrupted meta
105+
data and data. If MD finds a stripe with data and valid parities (1 parity for
106+
raid4/5 and 2 for raid6), MD will write the data and parities to RAID disks. If
107+
parities are incompleted, they are discarded. If part of data is corrupted,
108+
they are discarded too. MD then loads valid data and writes them to RAID disks
109+
in normal way.

‎block/bio.c

Lines changed: 48 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -625,21 +625,20 @@ struct bio *bio_clone_fast(struct bio *bio, gfp_t gfp_mask, struct bio_set *bs)
625625
}
626626
EXPORT_SYMBOL(bio_clone_fast);
627627

628-
/**
629-
* bio_clone_bioset - clone a bio
630-
* @bio_src: bio to clone
631-
* @gfp_mask: allocation priority
632-
* @bs: bio_set to allocate from
633-
*
634-
* Clone bio. Caller will own the returned bio, but not the actual data it
635-
* points to. Reference count of returned bio will be one.
636-
*/
637-
struct bio *bio_clone_bioset(struct bio *bio_src, gfp_t gfp_mask,
638-
struct bio_set *bs)
628+
static struct bio *__bio_clone_bioset(struct bio *bio_src, gfp_t gfp_mask,
629+
struct bio_set *bs, int offset,
630+
int size)
639631
{
640632
struct bvec_iter iter;
641633
struct bio_vec bv;
642634
struct bio *bio;
635+
struct bvec_iter iter_src = bio_src->bi_iter;
636+
637+
/* for supporting partial clone */
638+
if (offset || size != bio_src->bi_iter.bi_size) {
639+
bio_advance_iter(bio_src, &iter_src, offset);
640+
iter_src.bi_size = size;
641+
}
643642

644643
/*
645644
* Pre immutable biovecs, __bio_clone() used to just do a memcpy from
@@ -663,7 +662,8 @@ struct bio *bio_clone_bioset(struct bio *bio_src, gfp_t gfp_mask,
663662
* __bio_clone_fast() anyways.
664663
*/
665664

666-
bio = bio_alloc_bioset(gfp_mask, bio_segments(bio_src), bs);
665+
bio = bio_alloc_bioset(gfp_mask, __bio_segments(bio_src,
666+
&iter_src), bs);
667667
if (!bio)
668668
return NULL;
669669
bio->bi_bdev = bio_src->bi_bdev;
@@ -680,7 +680,7 @@ struct bio *bio_clone_bioset(struct bio *bio_src, gfp_t gfp_mask,
680680
bio->bi_io_vec[bio->bi_vcnt++] = bio_src->bi_io_vec[0];
681681
break;
682682
default:
683-
bio_for_each_segment(bv, bio_src, iter)
683+
__bio_for_each_segment(bv, bio_src, iter, iter_src)
684684
bio->bi_io_vec[bio->bi_vcnt++] = bv;
685685
break;
686686
}
@@ -699,8 +699,43 @@ struct bio *bio_clone_bioset(struct bio *bio_src, gfp_t gfp_mask,
699699

700700
return bio;
701701
}
702+
703+
/**
704+
* bio_clone_bioset - clone a bio
705+
* @bio_src: bio to clone
706+
* @gfp_mask: allocation priority
707+
* @bs: bio_set to allocate from
708+
*
709+
* Clone bio. Caller will own the returned bio, but not the actual data it
710+
* points to. Reference count of returned bio will be one.
711+
*/
712+
struct bio *bio_clone_bioset(struct bio *bio_src, gfp_t gfp_mask,
713+
struct bio_set *bs)
714+
{
715+
return __bio_clone_bioset(bio_src, gfp_mask, bs, 0,
716+
bio_src->bi_iter.bi_size);
717+
}
702718
EXPORT_SYMBOL(bio_clone_bioset);
703719

720+
/**
721+
* bio_clone_bioset_partial - clone a partial bio
722+
* @bio_src: bio to clone
723+
* @gfp_mask: allocation priority
724+
* @bs: bio_set to allocate from
725+
* @offset: cloned starting from the offset
726+
* @size: size for the cloned bio
727+
*
728+
* Clone bio. Caller will own the returned bio, but not the actual data it
729+
* points to. Reference count of returned bio will be one.
730+
*/
731+
struct bio *bio_clone_bioset_partial(struct bio *bio_src, gfp_t gfp_mask,
732+
struct bio_set *bs, int offset,
733+
int size)
734+
{
735+
return __bio_clone_bioset(bio_src, gfp_mask, bs, offset, size);
736+
}
737+
EXPORT_SYMBOL(bio_clone_bioset_partial);
738+
704739
/**
705740
* bio_add_pc_page - attempt to add page to bio
706741
* @q: the target queue

‎drivers/md/faulty.c

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -214,7 +214,7 @@ static void faulty_make_request(struct mddev *mddev, struct bio *bio)
214214
}
215215
}
216216
if (failit) {
217-
struct bio *b = bio_clone_mddev(bio, GFP_NOIO, mddev);
217+
struct bio *b = bio_clone_fast(bio, GFP_NOIO, mddev->bio_set);
218218

219219
b->bi_bdev = conf->rdev->bdev;
220220
b->bi_private = bio;

‎drivers/md/linear.c

Lines changed: 36 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -53,18 +53,26 @@ static inline struct dev_info *which_dev(struct mddev *mddev, sector_t sector)
5353
return conf->disks + lo;
5454
}
5555

56+
/*
57+
* In linear_congested() conf->raid_disks is used as a copy of
58+
* mddev->raid_disks to iterate conf->disks[], because conf->raid_disks
59+
* and conf->disks[] are created in linear_conf(), they are always
60+
* consitent with each other, but mddev->raid_disks does not.
61+
*/
5662
static int linear_congested(struct mddev *mddev, int bits)
5763
{
5864
struct linear_conf *conf;
5965
int i, ret = 0;
6066

61-
conf = mddev->private;
67+
rcu_read_lock();
68+
conf = rcu_dereference(mddev->private);
6269

63-
for (i = 0; i < mddev->raid_disks && !ret ; i++) {
70+
for (i = 0; i < conf->raid_disks && !ret ; i++) {
6471
struct request_queue *q = bdev_get_queue(conf->disks[i].rdev->bdev);
6572
ret |= bdi_congested(q->backing_dev_info, bits);
6673
}
6774

75+
rcu_read_unlock();
6876
return ret;
6977
}
7078

@@ -144,6 +152,19 @@ static struct linear_conf *linear_conf(struct mddev *mddev, int raid_disks)
144152
conf->disks[i-1].end_sector +
145153
conf->disks[i].rdev->sectors;
146154

155+
/*
156+
* conf->raid_disks is copy of mddev->raid_disks. The reason to
157+
* keep a copy of mddev->raid_disks in struct linear_conf is,
158+
* mddev->raid_disks may not be consistent with pointers number of
159+
* conf->disks[] when it is updated in linear_add() and used to
160+
* iterate old conf->disks[] earray in linear_congested().
161+
* Here conf->raid_disks is always consitent with number of
162+
* pointers in conf->disks[] array, and mddev->private is updated
163+
* with rcu_assign_pointer() in linear_addr(), such race can be
164+
* avoided.
165+
*/
166+
conf->raid_disks = raid_disks;
167+
147168
return conf;
148169

149170
out:
@@ -196,15 +217,24 @@ static int linear_add(struct mddev *mddev, struct md_rdev *rdev)
196217
if (!newconf)
197218
return -ENOMEM;
198219

220+
/* newconf->raid_disks already keeps a copy of * the increased
221+
* value of mddev->raid_disks, WARN_ONCE() is just used to make
222+
* sure of this. It is possible that oldconf is still referenced
223+
* in linear_congested(), therefore kfree_rcu() is used to free
224+
* oldconf until no one uses it anymore.
225+
*/
199226
mddev_suspend(mddev);
200-
oldconf = mddev->private;
227+
oldconf = rcu_dereference_protected(mddev->private,
228+
lockdep_is_held(&mddev->reconfig_mutex));
201229
mddev->raid_disks++;
202-
mddev->private = newconf;
230+
WARN_ONCE(mddev->raid_disks != newconf->raid_disks,
231+
"copied raid_disks doesn't match mddev->raid_disks");
232+
rcu_assign_pointer(mddev->private, newconf);
203233
md_set_array_sectors(mddev, linear_size(mddev, 0, 0));
204234
set_capacity(mddev->gendisk, mddev->array_sectors);
205235
mddev_resume(mddev);
206236
revalidate_disk(mddev->gendisk);
207-
kfree(oldconf);
237+
kfree_rcu(oldconf, rcu);
208238
return 0;
209239
}
210240

@@ -262,6 +292,7 @@ static void linear_make_request(struct mddev *mddev, struct bio *bio)
262292
trace_block_bio_remap(bdev_get_queue(split->bi_bdev),
263293
split, disk_devt(mddev->gendisk),
264294
bio_sector);
295+
mddev_check_writesame(mddev, split);
265296
generic_make_request(split);
266297
}
267298
} while (split != bio);

‎drivers/md/linear.h

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@ struct linear_conf
1010
{
1111
struct rcu_head rcu;
1212
sector_t array_sectors;
13+
int raid_disks; /* a copy of mddev->raid_disks */
1314
struct dev_info disks[0];
1415
};
1516
#endif

‎drivers/md/md.c

Lines changed: 11 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -190,16 +190,6 @@ struct bio *bio_alloc_mddev(gfp_t gfp_mask, int nr_iovecs,
190190
}
191191
EXPORT_SYMBOL_GPL(bio_alloc_mddev);
192192

193-
struct bio *bio_clone_mddev(struct bio *bio, gfp_t gfp_mask,
194-
struct mddev *mddev)
195-
{
196-
if (!mddev || !mddev->bio_set)
197-
return bio_clone(bio, gfp_mask);
198-
199-
return bio_clone_bioset(bio, gfp_mask, mddev->bio_set);
200-
}
201-
EXPORT_SYMBOL_GPL(bio_clone_mddev);
202-
203193
/*
204194
* We have a system wide 'event count' that is incremented
205195
* on any 'interesting' event, and readers of /proc/mdstat
@@ -5228,8 +5218,11 @@ int md_run(struct mddev *mddev)
52285218
sysfs_notify_dirent_safe(rdev->sysfs_state);
52295219
}
52305220

5231-
if (mddev->bio_set == NULL)
5221+
if (mddev->bio_set == NULL) {
52325222
mddev->bio_set = bioset_create(BIO_POOL_SIZE, 0);
5223+
if (!mddev->bio_set)
5224+
return -ENOMEM;
5225+
}
52335226

52345227
spin_lock(&pers_lock);
52355228
pers = find_pers(mddev->level, mddev->clevel);
@@ -8980,7 +8973,14 @@ static __exit void md_exit(void)
89808973

89818974
for_each_mddev(mddev, tmp) {
89828975
export_array(mddev);
8976+
mddev->ctime = 0;
89838977
mddev->hold_active = 0;
8978+
/*
8979+
* for_each_mddev() will call mddev_put() at the end of each
8980+
* iteration. As the mddev is now fully clear, this will
8981+
* schedule the mddev for destruction by a workqueue, and the
8982+
* destroy_workqueue() below will wait for that to complete.
8983+
*/
89848984
}
89858985
destroy_workqueue(md_misc_wq);
89868986
destroy_workqueue(md_wq);

‎drivers/md/md.h

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -673,8 +673,6 @@ extern void md_rdev_clear(struct md_rdev *rdev);
673673

674674
extern void mddev_suspend(struct mddev *mddev);
675675
extern void mddev_resume(struct mddev *mddev);
676-
extern struct bio *bio_clone_mddev(struct bio *bio, gfp_t gfp_mask,
677-
struct mddev *mddev);
678676
extern struct bio *bio_alloc_mddev(gfp_t gfp_mask, int nr_iovecs,
679677
struct mddev *mddev);
680678

@@ -710,4 +708,11 @@ static inline void mddev_clear_unsupported_flags(struct mddev *mddev,
710708
{
711709
mddev->flags &= ~unsupported_flags;
712710
}
711+
712+
static inline void mddev_check_writesame(struct mddev *mddev, struct bio *bio)
713+
{
714+
if (bio_op(bio) == REQ_OP_WRITE_SAME &&
715+
!bdev_get_queue(bio->bi_bdev)->limits.max_write_same_sectors)
716+
mddev->queue->limits.max_write_same_sectors = 0;
717+
}
713718
#endif /* _MD_MD_H */

0 commit comments

Comments
 (0)