Skip to content

Commit d6ec9d9

Browse files
torvaldstorvalds
committed
Merge branch 'x86-asm-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull x86 core updates from Ingo Molnar: "Note that in this cycle most of the x86 topics interacted at a level that caused them to be merged into tip:x86/asm - but this should be a temporary phenomenon, hopefully we'll back to the usual patterns in the next merge window. The main changes in this cycle were: Hardware enablement: - Add support for the Intel UMIP (User Mode Instruction Prevention) CPU feature. This is a security feature that disables certain instructions such as SGDT, SLDT, SIDT, SMSW and STR. (Ricardo Neri) [ Note that this is disabled by default for now, there are some smaller enhancements in the pipeline that I'll follow up with in the next 1-2 days, which allows this to be enabled by default.] - Add support for the AMD SEV (Secure Encrypted Virtualization) CPU feature, on top of SME (Secure Memory Encryption) support that was added in v4.14. (Tom Lendacky, Brijesh Singh) - Enable new SSE/AVX/AVX512 CPU features: AVX512_VBMI2, GFNI, VAES, VPCLMULQDQ, AVX512_VNNI, AVX512_BITALG. (Gayatri Kammela) Other changes: - A big series of entry code simplifications and enhancements (Andy Lutomirski) - Make the ORC unwinder default on x86 and various objtool enhancements. (Josh Poimboeuf) - 5-level paging enhancements (Kirill A. Shutemov) - Micro-optimize the entry code a bit (Borislav Petkov) - Improve the handling of interdependent CPU features in the early FPU init code (Andi Kleen) - Build system enhancements (Changbin Du, Masahiro Yamada) - ... plus misc enhancements, fixes and cleanups" * 'x86-asm-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (118 commits) x86/build: Make the boot image generation less verbose selftests/x86: Add tests for the STR and SLDT instructions selftests/x86: Add tests for User-Mode Instruction Prevention x86/traps: Fix up general protection faults caused by UMIP x86/umip: Enable User-Mode Instruction Prevention at runtime x86/umip: Force a page fault when unable to copy emulated result to user x86/umip: Add emulation code for UMIP instructions x86/cpufeature: Add User-Mode Instruction Prevention definitions x86/insn-eval: Add support to resolve 16-bit address encodings x86/insn-eval: Handle 32-bit address encodings in virtual-8086 mode x86/insn-eval: Add wrapper function for 32 and 64-bit addresses x86/insn-eval: Add support to resolve 32-bit address encodings x86/insn-eval: Compute linear address in several utility functions resource: Fix resource_size.cocci warnings X86/KVM: Clear encryption attribute when SEV is active X86/KVM: Decrypt shared per-cpu variables when SEV is active percpu: Introduce DEFINE_PER_CPU_DECRYPTED x86: Add support for changing memory encryption attribute in early boot x86/io: Unroll string I/O when SEV is active x86/boot: Add early boot support when running with SEV active ...
2 parents 3e20146 + 91a6a6c commit d6ec9d9

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

110 files changed

+4067
-1145
lines changed

‎Documentation/x86/amd-memory-encryption.txt

Lines changed: 26 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,29 +1,44 @@
1-
Secure Memory Encryption (SME) is a feature found on AMD processors.
1+
Secure Memory Encryption (SME) and Secure Encrypted Virtualization (SEV) are
2+
features found on AMD processors.
23

34
SME provides the ability to mark individual pages of memory as encrypted using
45
the standard x86 page tables. A page that is marked encrypted will be
56
automatically decrypted when read from DRAM and encrypted when written to
67
DRAM. SME can therefore be used to protect the contents of DRAM from physical
78
attacks on the system.
89

10+
SEV enables running encrypted virtual machines (VMs) in which the code and data
11+
of the guest VM are secured so that a decrypted version is available only
12+
within the VM itself. SEV guest VMs have the concept of private and shared
13+
memory. Private memory is encrypted with the guest-specific key, while shared
14+
memory may be encrypted with hypervisor key. When SME is enabled, the hypervisor
15+
key is the same key which is used in SME.
16+
917
A page is encrypted when a page table entry has the encryption bit set (see
1018
below on how to determine its position). The encryption bit can also be
1119
specified in the cr3 register, allowing the PGD table to be encrypted. Each
1220
successive level of page tables can also be encrypted by setting the encryption
1321
bit in the page table entry that points to the next table. This allows the full
1422
page table hierarchy to be encrypted. Note, this means that just because the
15-
encryption bit is set in cr3, doesn't imply the full hierarchy is encyrpted.
23+
encryption bit is set in cr3, doesn't imply the full hierarchy is encrypted.
1624
Each page table entry in the hierarchy needs to have the encryption bit set to
1725
achieve that. So, theoretically, you could have the encryption bit set in cr3
1826
so that the PGD is encrypted, but not set the encryption bit in the PGD entry
1927
for a PUD which results in the PUD pointed to by that entry to not be
2028
encrypted.
2129

22-
Support for SME can be determined through the CPUID instruction. The CPUID
23-
function 0x8000001f reports information related to SME:
30+
When SEV is enabled, instruction pages and guest page tables are always treated
31+
as private. All the DMA operations inside the guest must be performed on shared
32+
memory. Since the memory encryption bit is controlled by the guest OS when it
33+
is operating in 64-bit or 32-bit PAE mode, in all other modes the SEV hardware
34+
forces the memory encryption bit to 1.
35+
36+
Support for SME and SEV can be determined through the CPUID instruction. The
37+
CPUID function 0x8000001f reports information related to SME:
2438

2539
0x8000001f[eax]:
2640
Bit[0] indicates support for SME
41+
Bit[1] indicates support for SEV
2742
0x8000001f[ebx]:
2843
Bits[5:0] pagetable bit number used to activate memory
2944
encryption
@@ -39,6 +54,13 @@ determine if SME is enabled and/or to enable memory encryption:
3954
Bit[23] 0 = memory encryption features are disabled
4055
1 = memory encryption features are enabled
4156

57+
If SEV is supported, MSR 0xc0010131 (MSR_AMD64_SEV) can be used to determine if
58+
SEV is active:
59+
60+
0xc0010131:
61+
Bit[0] 0 = memory encryption is not active
62+
1 = memory encryption is active
63+
4264
Linux relies on BIOS to set this bit if BIOS has determined that the reduction
4365
in the physical address space as a result of enabling memory encryption (see
4466
CPUID information above) will not conflict with the address space resource

‎Documentation/x86/orc-unwinder.txt

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ ORC unwinder
44
Overview
55
--------
66

7-
The kernel CONFIG_ORC_UNWINDER option enables the ORC unwinder, which is
7+
The kernel CONFIG_UNWINDER_ORC option enables the ORC unwinder, which is
88
similar in concept to a DWARF unwinder. The difference is that the
99
format of the ORC data is much simpler than DWARF, which in turn allows
1010
the ORC unwinder to be much simpler and faster.

‎Documentation/x86/x86_64/mm.txt

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,7 @@ ff92000000000000 - ffd1ffffffffffff (=54 bits) vmalloc/ioremap space
3434
ffd2000000000000 - ffd3ffffffffffff (=49 bits) hole
3535
ffd4000000000000 - ffd5ffffffffffff (=49 bits) virtual memory map (512TB)
3636
... unused hole ...
37-
ffd8000000000000 - fff7ffffffffffff (=53 bits) kasan shadow memory (8PB)
37+
ffdf000000000000 - fffffc0000000000 (=53 bits) kasan shadow memory (8PB)
3838
... unused hole ...
3939
ffffff0000000000 - ffffff7fffffffff (=39 bits) %esp fixup stacks
4040
... unused hole ...

‎Makefile

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -934,8 +934,8 @@ ifdef CONFIG_STACK_VALIDATION
934934
ifeq ($(has_libelf),1)
935935
objtool_target := tools/objtool FORCE
936936
else
937-
ifdef CONFIG_ORC_UNWINDER
938-
$(error "Cannot generate ORC metadata for CONFIG_ORC_UNWINDER=y, please install libelf-dev, libelf-devel or elfutils-libelf-devel")
937+
ifdef CONFIG_UNWINDER_ORC
938+
$(error "Cannot generate ORC metadata for CONFIG_UNWINDER_ORC=y, please install libelf-dev, libelf-devel or elfutils-libelf-devel")
939939
else
940940
$(warning "Cannot use CONFIG_STACK_VALIDATION=y, please install libelf-dev, libelf-devel or elfutils-libelf-devel")
941941
endif

‎arch/powerpc/kernel/machine_kexec_file_64.c

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -91,11 +91,13 @@ int arch_kimage_file_post_load_cleanup(struct kimage *image)
9191
* and that value will be returned. If all free regions are visited without
9292
* func returning non-zero, then zero will be returned.
9393
*/
94-
int arch_kexec_walk_mem(struct kexec_buf *kbuf, int (*func)(u64, u64, void *))
94+
int arch_kexec_walk_mem(struct kexec_buf *kbuf,
95+
int (*func)(struct resource *, void *))
9596
{
9697
int ret = 0;
9798
u64 i;
9899
phys_addr_t mstart, mend;
100+
struct resource res = { };
99101

100102
if (kbuf->top_down) {
101103
for_each_free_mem_range_reverse(i, NUMA_NO_NODE, 0,
@@ -105,7 +107,9 @@ int arch_kexec_walk_mem(struct kexec_buf *kbuf, int (*func)(u64, u64, void *))
105107
* range while in kexec, end points to the last byte
106108
* in the range.
107109
*/
108-
ret = func(mstart, mend - 1, kbuf);
110+
res.start = mstart;
111+
res.end = mend - 1;
112+
ret = func(&res, kbuf);
109113
if (ret)
110114
break;
111115
}
@@ -117,7 +121,9 @@ int arch_kexec_walk_mem(struct kexec_buf *kbuf, int (*func)(u64, u64, void *))
117121
* range while in kexec, end points to the last byte
118122
* in the range.
119123
*/
120-
ret = func(mstart, mend - 1, kbuf);
124+
res.start = mstart;
125+
res.end = mend - 1;
126+
ret = func(&res, kbuf);
121127
if (ret)
122128
break;
123129
}

‎arch/x86/Kconfig

Lines changed: 11 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -171,7 +171,7 @@ config X86
171171
select HAVE_PERF_USER_STACK_DUMP
172172
select HAVE_RCU_TABLE_FREE
173173
select HAVE_REGS_AND_STACK_ACCESS_API
174-
select HAVE_RELIABLE_STACKTRACE if X86_64 && FRAME_POINTER_UNWINDER && STACK_VALIDATION
174+
select HAVE_RELIABLE_STACKTRACE if X86_64 && UNWINDER_FRAME_POINTER && STACK_VALIDATION
175175
select HAVE_STACK_VALIDATION if X86_64
176176
select HAVE_SYSCALL_TRACEPOINTS
177177
select HAVE_UNSTABLE_SCHED_CLOCK
@@ -303,7 +303,6 @@ config ARCH_SUPPORTS_DEBUG_PAGEALLOC
303303
config KASAN_SHADOW_OFFSET
304304
hex
305305
depends on KASAN
306-
default 0xdff8000000000000 if X86_5LEVEL
307306
default 0xdffffc0000000000
308307

309308
config HAVE_INTEL_TXT
@@ -1803,6 +1802,16 @@ config X86_SMAP
18031802

18041803
If unsure, say Y.
18051804

1805+
config X86_INTEL_UMIP
1806+
def_bool n
1807+
depends on CPU_SUP_INTEL
1808+
prompt "Intel User Mode Instruction Prevention" if EXPERT
1809+
---help---
1810+
The User Mode Instruction Prevention (UMIP) is a security
1811+
feature in newer Intel processors. If enabled, a general
1812+
protection fault is issued if the instructions SGDT, SLDT,
1813+
SIDT, SMSW and STR are executed in user mode.
1814+
18061815
config X86_INTEL_MPX
18071816
prompt "Intel MPX (Memory Protection Extensions)"
18081817
def_bool n

‎arch/x86/Kconfig.debug

Lines changed: 20 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -359,28 +359,14 @@ config PUNIT_ATOM_DEBUG
359359

360360
choice
361361
prompt "Choose kernel unwinder"
362-
default FRAME_POINTER_UNWINDER
362+
default UNWINDER_ORC if X86_64
363+
default UNWINDER_FRAME_POINTER if X86_32
363364
---help---
364365
This determines which method will be used for unwinding kernel stack
365366
traces for panics, oopses, bugs, warnings, perf, /proc/<pid>/stack,
366367
livepatch, lockdep, and more.
367368

368-
config FRAME_POINTER_UNWINDER
369-
bool "Frame pointer unwinder"
370-
select FRAME_POINTER
371-
---help---
372-
This option enables the frame pointer unwinder for unwinding kernel
373-
stack traces.
374-
375-
The unwinder itself is fast and it uses less RAM than the ORC
376-
unwinder, but the kernel text size will grow by ~3% and the kernel's
377-
overall performance will degrade by roughly 5-10%.
378-
379-
This option is recommended if you want to use the livepatch
380-
consistency model, as this is currently the only way to get a
381-
reliable stack trace (CONFIG_HAVE_RELIABLE_STACKTRACE).
382-
383-
config ORC_UNWINDER
369+
config UNWINDER_ORC
384370
bool "ORC unwinder"
385371
depends on X86_64
386372
select STACK_VALIDATION
@@ -396,7 +382,22 @@ config ORC_UNWINDER
396382
Enabling this option will increase the kernel's runtime memory usage
397383
by roughly 2-4MB, depending on your kernel config.
398384

399-
config GUESS_UNWINDER
385+
config UNWINDER_FRAME_POINTER
386+
bool "Frame pointer unwinder"
387+
select FRAME_POINTER
388+
---help---
389+
This option enables the frame pointer unwinder for unwinding kernel
390+
stack traces.
391+
392+
The unwinder itself is fast and it uses less RAM than the ORC
393+
unwinder, but the kernel text size will grow by ~3% and the kernel's
394+
overall performance will degrade by roughly 5-10%.
395+
396+
This option is recommended if you want to use the livepatch
397+
consistency model, as this is currently the only way to get a
398+
reliable stack trace (CONFIG_HAVE_RELIABLE_STACKTRACE).
399+
400+
config UNWINDER_GUESS
400401
bool "Guess unwinder"
401402
depends on EXPERT
402403
---help---
@@ -411,7 +412,7 @@ config GUESS_UNWINDER
411412
endchoice
412413

413414
config FRAME_POINTER
414-
depends on !ORC_UNWINDER && !GUESS_UNWINDER
415+
depends on !UNWINDER_ORC && !UNWINDER_GUESS
415416
bool
416417

417418
endmenu

‎arch/x86/boot/.gitignore

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,3 +7,6 @@ zoffset.h
77
setup
88
setup.bin
99
setup.elf
10+
fdimage
11+
mtools.conf
12+
image.iso

‎arch/x86/boot/Makefile

Lines changed: 11 additions & 48 deletions
Original file line numberDiff line numberDiff line change
@@ -123,63 +123,26 @@ image_cmdline = default linux $(FDARGS) $(if $(FDINITRD),initrd=initrd.img,)
123123
$(obj)/mtools.conf: $(src)/mtools.conf.in
124124
sed -e 's|@OBJ@|$(obj)|g' < $< > $@
125125

126+
quiet_cmd_genimage = GENIMAGE $3
127+
cmd_genimage = sh $(srctree)/$(src)/genimage.sh $2 $3 $(obj)/bzImage \
128+
$(obj)/mtools.conf '$(image_cmdline)' $(FDINITRD)
129+
126130
# This requires write access to /dev/fd0
127131
bzdisk: $(obj)/bzImage $(obj)/mtools.conf
128-
MTOOLSRC=$(obj)/mtools.conf mformat a: ; sync
129-
syslinux /dev/fd0 ; sync
130-
echo '$(image_cmdline)' | \
131-
MTOOLSRC=$(src)/mtools.conf mcopy - a:syslinux.cfg
132-
if [ -f '$(FDINITRD)' ] ; then \
133-
MTOOLSRC=$(obj)/mtools.conf mcopy '$(FDINITRD)' a:initrd.img ; \
134-
fi
135-
MTOOLSRC=$(obj)/mtools.conf mcopy $(obj)/bzImage a:linux ; sync
132+
$(call cmd,genimage,bzdisk,/dev/fd0)
136133

137134
# These require being root or having syslinux 2.02 or higher installed
138135
fdimage fdimage144: $(obj)/bzImage $(obj)/mtools.conf
139-
dd if=/dev/zero of=$(obj)/fdimage bs=1024 count=1440
140-
MTOOLSRC=$(obj)/mtools.conf mformat v: ; sync
141-
syslinux $(obj)/fdimage ; sync
142-
echo '$(image_cmdline)' | \
143-
MTOOLSRC=$(obj)/mtools.conf mcopy - v:syslinux.cfg
144-
if [ -f '$(FDINITRD)' ] ; then \
145-
MTOOLSRC=$(obj)/mtools.conf mcopy '$(FDINITRD)' v:initrd.img ; \
146-
fi
147-
MTOOLSRC=$(obj)/mtools.conf mcopy $(obj)/bzImage v:linux ; sync
136+
$(call cmd,genimage,fdimage144,$(obj)/fdimage)
137+
@$(kecho) 'Kernel: $(obj)/fdimage is ready'
148138

149139
fdimage288: $(obj)/bzImage $(obj)/mtools.conf
150-
dd if=/dev/zero of=$(obj)/fdimage bs=1024 count=2880
151-
MTOOLSRC=$(obj)/mtools.conf mformat w: ; sync
152-
syslinux $(obj)/fdimage ; sync
153-
echo '$(image_cmdline)' | \
154-
MTOOLSRC=$(obj)/mtools.conf mcopy - w:syslinux.cfg
155-
if [ -f '$(FDINITRD)' ] ; then \
156-
MTOOLSRC=$(obj)/mtools.conf mcopy '$(FDINITRD)' w:initrd.img ; \
157-
fi
158-
MTOOLSRC=$(obj)/mtools.conf mcopy $(obj)/bzImage w:linux ; sync
140+
$(call cmd,genimage,fdimage288,$(obj)/fdimage)
141+
@$(kecho) 'Kernel: $(obj)/fdimage is ready'
159142

160143
isoimage: $(obj)/bzImage
161-
-rm -rf $(obj)/isoimage
162-
mkdir $(obj)/isoimage
163-
for i in lib lib64 share end ; do \
164-
if [ -f /usr/$$i/syslinux/isolinux.bin ] ; then \
165-
cp /usr/$$i/syslinux/isolinux.bin $(obj)/isoimage ; \
166-
if [ -f /usr/$$i/syslinux/ldlinux.c32 ]; then \
167-
cp /usr/$$i/syslinux/ldlinux.c32 $(obj)/isoimage ; \
168-
fi ; \
169-
break ; \
170-
fi ; \
171-
if [ $$i = end ] ; then exit 1 ; fi ; \
172-
done
173-
cp $(obj)/bzImage $(obj)/isoimage/linux
174-
echo '$(image_cmdline)' > $(obj)/isoimage/isolinux.cfg
175-
if [ -f '$(FDINITRD)' ] ; then \
176-
cp '$(FDINITRD)' $(obj)/isoimage/initrd.img ; \
177-
fi
178-
mkisofs -J -r -o $(obj)/image.iso -b isolinux.bin -c boot.cat \
179-
-no-emul-boot -boot-load-size 4 -boot-info-table \
180-
$(obj)/isoimage
181-
isohybrid $(obj)/image.iso 2>/dev/null || true
182-
rm -rf $(obj)/isoimage
144+
$(call cmd,genimage,isoimage,$(obj)/image.iso)
145+
@$(kecho) 'Kernel: $(obj)/image.iso is ready'
183146

184147
bzlilo: $(obj)/bzImage
185148
if [ -f $(INSTALL_PATH)/vmlinuz ]; then mv $(INSTALL_PATH)/vmlinuz $(INSTALL_PATH)/vmlinuz.old; fi

‎arch/x86/boot/compressed/Makefile

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -78,6 +78,7 @@ vmlinux-objs-$(CONFIG_EARLY_PRINTK) += $(obj)/early_serial_console.o
7878
vmlinux-objs-$(CONFIG_RANDOMIZE_BASE) += $(obj)/kaslr.o
7979
ifdef CONFIG_X86_64
8080
vmlinux-objs-$(CONFIG_RANDOMIZE_BASE) += $(obj)/pagetable.o
81+
vmlinux-objs-y += $(obj)/mem_encrypt.o
8182
endif
8283

8384
$(obj)/eboot.o: KBUILD_CFLAGS += -fshort-wchar -mno-red-zone

‎arch/x86/boot/compressed/head_64.S

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -131,6 +131,19 @@ ENTRY(startup_32)
131131
/*
132132
* Build early 4G boot pagetable
133133
*/
134+
/*
135+
* If SEV is active then set the encryption mask in the page tables.
136+
* This will insure that when the kernel is copied and decompressed
137+
* it will be done so encrypted.
138+
*/
139+
call get_sev_encryption_bit
140+
xorl %edx, %edx
141+
testl %eax, %eax
142+
jz 1f
143+
subl $32, %eax /* Encryption bit is always above bit 31 */
144+
bts %eax, %edx /* Set encryption mask for page tables */
145+
1:
146+
134147
/* Initialize Page tables to 0 */
135148
leal pgtable(%ebx), %edi
136149
xorl %eax, %eax
@@ -141,12 +154,14 @@ ENTRY(startup_32)
141154
leal pgtable + 0(%ebx), %edi
142155
leal 0x1007 (%edi), %eax
143156
movl %eax, 0(%edi)
157+
addl %edx, 4(%edi)
144158

145159
/* Build Level 3 */
146160
leal pgtable + 0x1000(%ebx), %edi
147161
leal 0x1007(%edi), %eax
148162
movl $4, %ecx
149163
1: movl %eax, 0x00(%edi)
164+
addl %edx, 0x04(%edi)
150165
addl $0x00001000, %eax
151166
addl $8, %edi
152167
decl %ecx
@@ -157,6 +172,7 @@ ENTRY(startup_32)
157172
movl $0x00000183, %eax
158173
movl $2048, %ecx
159174
1: movl %eax, 0(%edi)
175+
addl %edx, 4(%edi)
160176
addl $0x00200000, %eax
161177
addl $8, %edi
162178
decl %ecx

0 commit comments

Comments
 (0)