Skip to content

Commit 33e247c

Browse files
torvaldstorvalds
committed
Merge branch 'akpm' (patches from Andrew)
Merge third patch-bomb from Andrew Morton: - even more of the rest of MM - lib/ updates - checkpatch updates - small changes to a few scruffy filesystems - kmod fixes/cleanups - kexec updates - a dma-mapping cleanup series from hch * emailed patches from Andrew Morton <akpm@linux-foundation.org>: (81 commits) dma-mapping: consolidate dma_set_mask dma-mapping: consolidate dma_supported dma-mapping: cosolidate dma_mapping_error dma-mapping: consolidate dma_{alloc,free}_noncoherent dma-mapping: consolidate dma_{alloc,free}_{attrs,coherent} mm: use vma_is_anonymous() in create_huge_pmd() and wp_huge_pmd() mm: make sure all file VMAs have ->vm_ops set mm, mpx: add "vm_flags_t vm_flags" arg to do_mmap_pgoff() mm: mark most vm_operations_struct const namei: fix warning while make xmldocs caused by namei.c ipc: convert invalid scenarios to use WARN_ON zlib_deflate/deftree: remove bi_reverse() lib/decompress_unlzma: Do a NULL check for pointer lib/decompressors: use real out buf size for gunzip with kernel fs/affs: make root lookup from blkdev logical size sysctl: fix int -> unsigned long assignments in INT_MIN case kexec: export KERNEL_IMAGE_SIZE to vmcoreinfo kexec: align crash_notes allocation to make it be inside one physical page kexec: remove unnecessary test in kimage_alloc_crash_control_pages() kexec: split kexec_load syscall from kexec core code ...
2 parents d71fc23 + 452e06a commit 33e247c

File tree

165 files changed

+4886
-4471
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

165 files changed

+4886
-4471
lines changed

‎CREDITS

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2992,6 +2992,10 @@ S: 2200 Mission College Blvd
29922992
S: Santa Clara, CA 95052
29932993
S: USA
29942994

2995+
N: Anil Ravindranath
2996+
E: anil_ravindranath@pmc-sierra.com
2997+
D: PMC-Sierra MaxRAID driver
2998+
29952999
N: Eric S. Raymond
29963000
E: esr@thyrsus.com
29973001
W: http://www.tuxedo.org/~esr/

‎Documentation/vm/00-INDEX

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,8 @@ hugetlbpage.txt
1414
- a brief summary of hugetlbpage support in the Linux kernel.
1515
hwpoison.txt
1616
- explains what hwpoison is
17+
idle_page_tracking.txt
18+
- description of the idle page tracking feature.
1719
ksm.txt
1820
- how to use the Kernel Samepage Merging feature.
1921
numa

‎Documentation/vm/idle_page_tracking.txt

Lines changed: 98 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,98 @@
1+
MOTIVATION
2+
3+
The idle page tracking feature allows to track which memory pages are being
4+
accessed by a workload and which are idle. This information can be useful for
5+
estimating the workload's working set size, which, in turn, can be taken into
6+
account when configuring the workload parameters, setting memory cgroup limits,
7+
or deciding where to place the workload within a compute cluster.
8+
9+
It is enabled by CONFIG_IDLE_PAGE_TRACKING=y.
10+
11+
USER API
12+
13+
The idle page tracking API is located at /sys/kernel/mm/page_idle. Currently,
14+
it consists of the only read-write file, /sys/kernel/mm/page_idle/bitmap.
15+
16+
The file implements a bitmap where each bit corresponds to a memory page. The
17+
bitmap is represented by an array of 8-byte integers, and the page at PFN #i is
18+
mapped to bit #i%64 of array element #i/64, byte order is native. When a bit is
19+
set, the corresponding page is idle.
20+
21+
A page is considered idle if it has not been accessed since it was marked idle
22+
(for more details on what "accessed" actually means see the IMPLEMENTATION
23+
DETAILS section). To mark a page idle one has to set the bit corresponding to
24+
the page by writing to the file. A value written to the file is OR-ed with the
25+
current bitmap value.
26+
27+
Only accesses to user memory pages are tracked. These are pages mapped to a
28+
process address space, page cache and buffer pages, swap cache pages. For other
29+
page types (e.g. SLAB pages) an attempt to mark a page idle is silently ignored,
30+
and hence such pages are never reported idle.
31+
32+
For huge pages the idle flag is set only on the head page, so one has to read
33+
/proc/kpageflags in order to correctly count idle huge pages.
34+
35+
Reading from or writing to /sys/kernel/mm/page_idle/bitmap will return
36+
-EINVAL if you are not starting the read/write on an 8-byte boundary, or
37+
if the size of the read/write is not a multiple of 8 bytes. Writing to
38+
this file beyond max PFN will return -ENXIO.
39+
40+
That said, in order to estimate the amount of pages that are not used by a
41+
workload one should:
42+
43+
1. Mark all the workload's pages as idle by setting corresponding bits in
44+
/sys/kernel/mm/page_idle/bitmap. The pages can be found by reading
45+
/proc/pid/pagemap if the workload is represented by a process, or by
46+
filtering out alien pages using /proc/kpagecgroup in case the workload is
47+
placed in a memory cgroup.
48+
49+
2. Wait until the workload accesses its working set.
50+
51+
3. Read /sys/kernel/mm/page_idle/bitmap and count the number of bits set. If
52+
one wants to ignore certain types of pages, e.g. mlocked pages since they
53+
are not reclaimable, he or she can filter them out using /proc/kpageflags.
54+
55+
See Documentation/vm/pagemap.txt for more information about /proc/pid/pagemap,
56+
/proc/kpageflags, and /proc/kpagecgroup.
57+
58+
IMPLEMENTATION DETAILS
59+
60+
The kernel internally keeps track of accesses to user memory pages in order to
61+
reclaim unreferenced pages first on memory shortage conditions. A page is
62+
considered referenced if it has been recently accessed via a process address
63+
space, in which case one or more PTEs it is mapped to will have the Accessed bit
64+
set, or marked accessed explicitly by the kernel (see mark_page_accessed()). The
65+
latter happens when:
66+
67+
- a userspace process reads or writes a page using a system call (e.g. read(2)
68+
or write(2))
69+
70+
- a page that is used for storing filesystem buffers is read or written,
71+
because a process needs filesystem metadata stored in it (e.g. lists a
72+
directory tree)
73+
74+
- a page is accessed by a device driver using get_user_pages()
75+
76+
When a dirty page is written to swap or disk as a result of memory reclaim or
77+
exceeding the dirty memory limit, it is not marked referenced.
78+
79+
The idle memory tracking feature adds a new page flag, the Idle flag. This flag
80+
is set manually, by writing to /sys/kernel/mm/page_idle/bitmap (see the USER API
81+
section), and cleared automatically whenever a page is referenced as defined
82+
above.
83+
84+
When a page is marked idle, the Accessed bit must be cleared in all PTEs it is
85+
mapped to, otherwise we will not be able to detect accesses to the page coming
86+
from a process address space. To avoid interference with the reclaimer, which,
87+
as noted above, uses the Accessed bit to promote actively referenced pages, one
88+
more page flag is introduced, the Young flag. When the PTE Accessed bit is
89+
cleared as a result of setting or updating a page's Idle flag, the Young flag
90+
is set on the page. The reclaimer treats the Young flag as an extra PTE
91+
Accessed bit and therefore will consider such a page as referenced.
92+
93+
Since the idle memory tracking feature is based on the memory reclaimer logic,
94+
it only works with pages that are on an LRU list, other pages are silently
95+
ignored. That means it will ignore a user memory page if it is isolated, but
96+
since there are usually not many of them, it should not affect the overall
97+
result noticeably. In order not to stall scanning of the idle page bitmap,
98+
locked pages may be skipped too.

‎Documentation/vm/pagemap.txt

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ pagemap is a new (as of 2.6.25) set of interfaces in the kernel that allow
55
userspace programs to examine the page tables and related information by
66
reading files in /proc.
77

8-
There are three components to pagemap:
8+
There are four components to pagemap:
99

1010
* /proc/pid/pagemap. This file lets a userspace process find out which
1111
physical frame each virtual page is mapped to. It contains one 64-bit
@@ -70,6 +70,11 @@ There are three components to pagemap:
7070
22. THP
7171
23. BALLOON
7272
24. ZERO_PAGE
73+
25. IDLE
74+
75+
* /proc/kpagecgroup. This file contains a 64-bit inode number of the
76+
memory cgroup each page is charged to, indexed by PFN. Only available when
77+
CONFIG_MEMCG is set.
7378

7479
Short descriptions to the page flags:
7580

@@ -116,6 +121,12 @@ Short descriptions to the page flags:
116121
24. ZERO_PAGE
117122
zero page for pfn_zero or huge_zero page
118123

124+
25. IDLE
125+
page has not been accessed since it was marked idle (see
126+
Documentation/vm/idle_page_tracking.txt). Note that this flag may be
127+
stale in case the page was accessed via a PTE. To make sure the flag
128+
is up-to-date one has to read /sys/kernel/mm/page_idle/bitmap first.
129+
119130
[IO related page flags]
120131
1. ERROR IO error occurred
121132
3. UPTODATE page has up-to-date data

‎Documentation/vm/zswap.txt

Lines changed: 28 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,7 @@ can also be enabled and disabled at runtime using the sysfs interface.
3232
An example command to enable zswap at runtime, assuming sysfs is mounted
3333
at /sys, is:
3434

35-
echo 1 > /sys/modules/zswap/parameters/enabled
35+
echo 1 > /sys/module/zswap/parameters/enabled
3636

3737
When zswap is disabled at runtime it will stop storing pages that are
3838
being swapped out. However, it will _not_ immediately write out or fault
@@ -49,14 +49,26 @@ Zswap receives pages for compression through the Frontswap API and is able to
4949
evict pages from its own compressed pool on an LRU basis and write them back to
5050
the backing swap device in the case that the compressed pool is full.
5151

52-
Zswap makes use of zbud for the managing the compressed memory pool. Each
53-
allocation in zbud is not directly accessible by address. Rather, a handle is
52+
Zswap makes use of zpool for the managing the compressed memory pool. Each
53+
allocation in zpool is not directly accessible by address. Rather, a handle is
5454
returned by the allocation routine and that handle must be mapped before being
5555
accessed. The compressed memory pool grows on demand and shrinks as compressed
56-
pages are freed. The pool is not preallocated.
56+
pages are freed. The pool is not preallocated. By default, a zpool of type
57+
zbud is created, but it can be selected at boot time by setting the "zpool"
58+
attribute, e.g. zswap.zpool=zbud. It can also be changed at runtime using the
59+
sysfs "zpool" attribute, e.g.
60+
61+
echo zbud > /sys/module/zswap/parameters/zpool
62+
63+
The zbud type zpool allocates exactly 1 page to store 2 compressed pages, which
64+
means the compression ratio will always be 2:1 or worse (because of half-full
65+
zbud pages). The zsmalloc type zpool has a more complex compressed page
66+
storage method, and it can achieve greater storage densities. However,
67+
zsmalloc does not implement compressed page eviction, so once zswap fills it
68+
cannot evict the oldest page, it can only reject new pages.
5769

5870
When a swap page is passed from frontswap to zswap, zswap maintains a mapping
59-
of the swap entry, a combination of the swap type and swap offset, to the zbud
71+
of the swap entry, a combination of the swap type and swap offset, to the zpool
6072
handle that references that compressed swap page. This mapping is achieved
6173
with a red-black tree per swap type. The swap offset is the search key for the
6274
tree nodes.
@@ -74,9 +86,17 @@ controlled policy:
7486
* max_pool_percent - The maximum percentage of memory that the compressed
7587
pool can occupy.
7688

77-
Zswap allows the compressor to be selected at kernel boot time by setting the
78-
“compressor” attribute. The default compressor is lzo. e.g.
79-
zswap.compressor=deflate
89+
The default compressor is lzo, but it can be selected at boot time by setting
90+
the “compressor” attribute, e.g. zswap.compressor=lzo. It can also be changed
91+
at runtime using the sysfs "compressor" attribute, e.g.
92+
93+
echo lzo > /sys/module/zswap/parameters/compressor
94+
95+
When the zpool and/or compressor parameter is changed at runtime, any existing
96+
compressed pages are not modified; they are left in their own zpool. When a
97+
request is made for a page in an old zpool, it is uncompressed using its
98+
original compressor. Once all pages are removed from an old zpool, the zpool
99+
and its compressor are freed.
80100

81101
A debugfs interface is provided for various statistic about pool size, number
82102
of pages stored, and various counters for the reasons pages are rejected.

‎MAINTAINERS

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -8199,10 +8199,9 @@ F: drivers/hwmon/pmbus/
81998199
F: include/linux/i2c/pmbus.h
82008200

82018201
PMC SIERRA MaxRAID DRIVER
8202-
M: Anil Ravindranath <anil_ravindranath@pmc-sierra.com>
82038202
L: linux-scsi@vger.kernel.org
82048203
W: http://www.pmc-sierra.com/
8205-
S: Supported
8204+
S: Orphan
82068205
F: drivers/scsi/pmcraid.*
82078206

82088207
PMC SIERRA PM8001 DRIVER

‎arch/Kconfig

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,9 @@
22
# General architecture dependent options
33
#
44

5+
config KEXEC_CORE
6+
bool
7+
58
config OPROFILE
69
tristate "OProfile system profiling"
710
depends on PROFILING

‎arch/alpha/include/asm/dma-mapping.h

Lines changed: 0 additions & 36 deletions
Original file line numberDiff line numberDiff line change
@@ -12,42 +12,6 @@ static inline struct dma_map_ops *get_dma_ops(struct device *dev)
1212

1313
#include <asm-generic/dma-mapping-common.h>
1414

15-
#define dma_alloc_coherent(d,s,h,f) dma_alloc_attrs(d,s,h,f,NULL)
16-
17-
static inline void *dma_alloc_attrs(struct device *dev, size_t size,
18-
dma_addr_t *dma_handle, gfp_t gfp,
19-
struct dma_attrs *attrs)
20-
{
21-
return get_dma_ops(dev)->alloc(dev, size, dma_handle, gfp, attrs);
22-
}
23-
24-
#define dma_free_coherent(d,s,c,h) dma_free_attrs(d,s,c,h,NULL)
25-
26-
static inline void dma_free_attrs(struct device *dev, size_t size,
27-
void *vaddr, dma_addr_t dma_handle,
28-
struct dma_attrs *attrs)
29-
{
30-
get_dma_ops(dev)->free(dev, size, vaddr, dma_handle, attrs);
31-
}
32-
33-
static inline int dma_mapping_error(struct device *dev, dma_addr_t dma_addr)
34-
{
35-
return get_dma_ops(dev)->mapping_error(dev, dma_addr);
36-
}
37-
38-
static inline int dma_supported(struct device *dev, u64 mask)
39-
{
40-
return get_dma_ops(dev)->dma_supported(dev, mask);
41-
}
42-
43-
static inline int dma_set_mask(struct device *dev, u64 mask)
44-
{
45-
return get_dma_ops(dev)->set_dma_mask(dev, mask);
46-
}
47-
48-
#define dma_alloc_noncoherent(d, s, h, f) dma_alloc_coherent(d, s, h, f)
49-
#define dma_free_noncoherent(d, s, v, h) dma_free_coherent(d, s, v, h)
50-
5115
#define dma_cache_sync(dev, va, size, dir) ((void)0)
5216

5317
#endif /* _ALPHA_DMA_MAPPING_H */

‎arch/alpha/kernel/pci-noop.c

Lines changed: 0 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -166,23 +166,13 @@ static int alpha_noop_supported(struct device *dev, u64 mask)
166166
return mask < 0x00ffffffUL ? 0 : 1;
167167
}
168168

169-
static int alpha_noop_set_mask(struct device *dev, u64 mask)
170-
{
171-
if (!dev->dma_mask || !dma_supported(dev, mask))
172-
return -EIO;
173-
174-
*dev->dma_mask = mask;
175-
return 0;
176-
}
177-
178169
struct dma_map_ops alpha_noop_ops = {
179170
.alloc = alpha_noop_alloc_coherent,
180171
.free = alpha_noop_free_coherent,
181172
.map_page = alpha_noop_map_page,
182173
.map_sg = alpha_noop_map_sg,
183174
.mapping_error = alpha_noop_mapping_error,
184175
.dma_supported = alpha_noop_supported,
185-
.set_dma_mask = alpha_noop_set_mask,
186176
};
187177

188178
struct dma_map_ops *dma_ops = &alpha_noop_ops;

‎arch/alpha/kernel/pci_iommu.c

Lines changed: 0 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -939,16 +939,6 @@ static int alpha_pci_mapping_error(struct device *dev, dma_addr_t dma_addr)
939939
return dma_addr == 0;
940940
}
941941

942-
static int alpha_pci_set_mask(struct device *dev, u64 mask)
943-
{
944-
if (!dev->dma_mask ||
945-
!pci_dma_supported(alpha_gendev_to_pci(dev), mask))
946-
return -EIO;
947-
948-
*dev->dma_mask = mask;
949-
return 0;
950-
}
951-
952942
struct dma_map_ops alpha_pci_ops = {
953943
.alloc = alpha_pci_alloc_coherent,
954944
.free = alpha_pci_free_coherent,
@@ -958,7 +948,6 @@ struct dma_map_ops alpha_pci_ops = {
958948
.unmap_sg = alpha_pci_unmap_sg,
959949
.mapping_error = alpha_pci_mapping_error,
960950
.dma_supported = alpha_pci_supported,
961-
.set_dma_mask = alpha_pci_set_mask,
962951
};
963952

964953
struct dma_map_ops *dma_ops = &alpha_pci_ops;

‎arch/arm/Kconfig

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2020,6 +2020,7 @@ config KEXEC
20202020
bool "Kexec system call (EXPERIMENTAL)"
20212021
depends on (!SMP || PM_SLEEP_SMP)
20222022
depends on !CPU_V7M
2023+
select KEXEC_CORE
20232024
help
20242025
kexec is a system call that implements the ability to shutdown your
20252026
current kernel, and to start another kernel. It is like a reboot

‎arch/arm/boot/compressed/decompress.c

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -57,5 +57,5 @@ extern char * strstr(const char * s1, const char *s2);
5757

5858
int do_decompress(u8 *input, int len, u8 *output, void (*error)(char *x))
5959
{
60-
return decompress(input, len, NULL, NULL, output, NULL, error);
60+
return __decompress(input, len, NULL, NULL, output, 0, NULL, error);
6161
}

0 commit comments

Comments
 (0)