bisecting cause commit starting from 8b787da7ba8cd2383988480d282ec1abc3917892 building syzkaller on 1880b4a9f394370a7d1fcb5c1cfca0fa1127b463 testing commit 8b787da7ba8cd2383988480d282ec1abc3917892 with gcc (GCC) 8.1.0 kernel signature: 979e1113943b79d1c5df29e188babb9979744145c20875cde77733b0bb11ef16 all runs: crashed: BUG: unable to handle kernel NULL pointer dereference in __percpu_ref_exit testing release v5.8 testing commit bcf876870b95592b52519ed4aafcf9d95999bc9c with gcc (GCC) 8.1.0 kernel signature: 85019bf285dbc70a425335c73cf329ba84cbc50c735b26bac210c077327bd10e all runs: OK # git bisect start 8b787da7ba8cd2383988480d282ec1abc3917892 bcf876870b95592b52519ed4aafcf9d95999bc9c Bisecting: 14989 revisions left to test after this (roughly 14 steps) [99faa39ec56f33591ed3cc4d3ef62ac2878fad7e] Merge branch 'for-5.10/block' into for-next testing commit 99faa39ec56f33591ed3cc4d3ef62ac2878fad7e with gcc (GCC) 8.1.0 kernel signature: dbcc34fc9c905988d88eab054ad912ca34711f935dac1997ccc5cdcb02c25759 all runs: OK # git bisect good 99faa39ec56f33591ed3cc4d3ef62ac2878fad7e Bisecting: 6895 revisions left to test after this (roughly 13 steps) [18d581144f2e261f7be5622b4e12982e4bda48a7] Merge remote-tracking branch 'net-next/master' into master testing commit 18d581144f2e261f7be5622b4e12982e4bda48a7 with gcc (GCC) 8.1.0 net/xdp/xsk_buff_pool.c:7:10: fatal error: linux/dma-noncoherent.h: No such file or directory # git bisect skip 18d581144f2e261f7be5622b4e12982e4bda48a7 Bisecting: 6895 revisions left to test after this (roughly 13 steps) [530df3c031a65a57dae8f1ade0e8266b33530b70] drm/i915: Reworkd DFP max bpc handling testing commit 530df3c031a65a57dae8f1ade0e8266b33530b70 with gcc (GCC) 8.1.0 kernel signature: a2436c7b8875fa5cad409b4ab89db7ad6ce6b7df86c691796ff87fb6af57a284 all runs: OK # git bisect good 530df3c031a65a57dae8f1ade0e8266b33530b70 Bisecting: 6030 revisions left to test after this (roughly 13 steps) [42db1d5c2d090f76125bc2cf3e4ab2b33d61d215] fs/binfmt_elf: use PT_LOAD p_align values for suitable start address testing commit 42db1d5c2d090f76125bc2cf3e4ab2b33d61d215 with gcc (GCC) 8.1.0 kernel signature: 197ec2bd2b3fcd1258408d899787b3d29ceaa8853915df17fdee6c9e05164e10 all runs: OK # git bisect good 42db1d5c2d090f76125bc2cf3e4ab2b33d61d215 Bisecting: 5665 revisions left to test after this (roughly 13 steps) [d557ea39a5f894630c403b78703ac92b08b7dd62] bpf: selftests: Add test for different inner map size testing commit d557ea39a5f894630c403b78703ac92b08b7dd62 with gcc (GCC) 8.1.0 kernel signature: 423b2e90caedd621714b5ac478c187a2d34b7dcbf6eee3f9aa7147fb2e6f5d2f all runs: OK # git bisect good d557ea39a5f894630c403b78703ac92b08b7dd62 Bisecting: 5665 revisions left to test after this (roughly 13 steps) [1f2bd2271a105767046e02ecabd39a608270a4c6] pwm: Allow store 64-bit duty cycle from sysfs interface testing commit 1f2bd2271a105767046e02ecabd39a608270a4c6 with gcc (GCC) 8.1.0 kernel signature: b95eb22fb6eb98bfc09cdd81ac0102ff3b5d1bfec34230c793267e7502f64f8a all runs: OK # git bisect good 1f2bd2271a105767046e02ecabd39a608270a4c6 Bisecting: 5651 revisions left to test after this (roughly 13 steps) [1bee263dfda57e45ad39c59a663c123a357ce38b] ALSA: hda - controller is in GPU on the DG1 testing commit 1bee263dfda57e45ad39c59a663c123a357ce38b with gcc (GCC) 8.1.0 kernel signature: ed7a599ddcfcd6c7136c8e663d4d737e7af59fe105a60da5ae7c2ff171756857 all runs: OK # git bisect good 1bee263dfda57e45ad39c59a663c123a357ce38b Bisecting: 5590 revisions left to test after this (roughly 13 steps) [0313c7c2e45c40c8f2e4156672125a9756e6d8a1] mlx4: make sure to always set the port type testing commit 0313c7c2e45c40c8f2e4156672125a9756e6d8a1 with gcc (GCC) 8.1.0 kernel signature: b5daec85831b4a0aef2f3c6791f9dc22a0d17fead35a5282689413b6f9d10966 all runs: OK # git bisect good 0313c7c2e45c40c8f2e4156672125a9756e6d8a1 Bisecting: 5590 revisions left to test after this (roughly 13 steps) [f91a35b46af04f5a1fbbe559613df3bd68ca45d8] ath9k: convert tasklets to use new tasklet_setup() API testing commit f91a35b46af04f5a1fbbe559613df3bd68ca45d8 with gcc (GCC) 8.1.0 kernel signature: 76a18f09f56390a9737b30e552ea17bbc4f501c93c3852c54c8f434a9adc8aca all runs: OK # git bisect good f91a35b46af04f5a1fbbe559613df3bd68ca45d8 Bisecting: 5590 revisions left to test after this (roughly 13 steps) [c6f0a816d9cb8135f0517622ba4f2416a8ed8cce] Merge branch 'for-5.6/hidraw' into for-next testing commit c6f0a816d9cb8135f0517622ba4f2416a8ed8cce with gcc (GCC) 8.1.0 kernel signature: ad449c630ecfe2d0f37399a39b10bc1da515c30218261c8609ec3542828bd1b4 run #0: basic kernel testing failed: failed to copy test binary to VM: failed to run ["scp" "-P" "22" "-F" "/dev/null" "-o" "UserKnownHostsFile=/dev/null" "-o" "BatchMode=yes" "-o" "IdentitiesOnly=yes" "-o" "StrictHostKeyChecking=no" "-o" "ConnectTimeout=10" "-i" "/syzkaller/jobs/linux/workdir/image/key" "/syzkaller/jobs/linux/workdir/repro.prog" "root@10.128.10.12:./repro.prog"]: exit status 1 Warning: Permanently added '10.128.10.12' (ECDSA) to the list of known hosts. /syzkaller/jobs/linux/workdir/repro.prog: Broken pipe run #1: OK run #2: OK run #3: OK run #4: OK run #5: OK run #6: OK run #7: OK run #8: OK run #9: OK # git bisect good c6f0a816d9cb8135f0517622ba4f2416a8ed8cce Bisecting: 5590 revisions left to test after this (roughly 13 steps) [c277e1f0dc3c7d7b5b028e20dd414df241642036] Input: twl4030_keypad - fix handling of platform_get_irq() error testing commit c277e1f0dc3c7d7b5b028e20dd414df241642036 with gcc (GCC) 8.1.0 kernel signature: c69e3a1ca7341d9cf1257161c934f3cbb45689c6ae032626224af3f5203b528c all runs: OK # git bisect good c277e1f0dc3c7d7b5b028e20dd414df241642036 Bisecting: 5573 revisions left to test after this (roughly 13 steps) [60db5e408e432d57f93e1eaec8fc9d7a05caa1f9] net: atlantic: implement media detect feature via phy tunables testing commit 60db5e408e432d57f93e1eaec8fc9d7a05caa1f9 with gcc (GCC) 8.1.0 kernel signature: 50eb6d116d173b2524cd09c1c2dd6a0aea6b71880cf0787032544c07f6c5808d all runs: OK # git bisect good 60db5e408e432d57f93e1eaec8fc9d7a05caa1f9 Bisecting: 4723 revisions left to test after this (roughly 12 steps) [60aa1e5420e246a2a4d8c82bb8c4ebf648d46099] Merge remote-tracking branch 'drm/drm-next' into master testing commit 60aa1e5420e246a2a4d8c82bb8c4ebf648d46099 with gcc (GCC) 8.1.0 kernel signature: 0dcc512e421e843dcd2f8c603887331d88860c42507c123b7a24dcca4bc76e31 all runs: OK # git bisect good 60aa1e5420e246a2a4d8c82bb8c4ebf648d46099 Bisecting: 2379 revisions left to test after this (roughly 11 steps) [9271da55f0572de7bd7eaccebf3f9b21ca6786f8] Merge remote-tracking branch 'rcu/rcu/next' into master testing commit 9271da55f0572de7bd7eaccebf3f9b21ca6786f8 with gcc (GCC) 8.1.0 kernel signature: 33314193c388d6699b6050cae3c5edbff8dce35aed2b5139549942889b19b14d all runs: crashed: BUG: unable to handle kernel NULL pointer dereference in __percpu_ref_exit # git bisect bad 9271da55f0572de7bd7eaccebf3f9b21ca6786f8 Bisecting: 1170 revisions left to test after this (roughly 10 steps) [172afde3279a6c5aec712cb42b6b0c0e350f88a6] Merge remote-tracking branch 'tpmdd/next' into master testing commit 172afde3279a6c5aec712cb42b6b0c0e350f88a6 with gcc (GCC) 8.1.0 kernel signature: f38650057233dd6fc76563431adcefd93be0604a6a6c92adb81e51a1df161c74 all runs: crashed: BUG: unable to handle kernel NULL pointer dereference in __percpu_ref_exit # git bisect bad 172afde3279a6c5aec712cb42b6b0c0e350f88a6 Bisecting: 534 revisions left to test after this (roughly 9 steps) [6c43a4d2e7299c4c22cfc5ea3e1193a4625fd02e] Merge remote-tracking branch 'sound-asoc/for-next' into master testing commit 6c43a4d2e7299c4c22cfc5ea3e1193a4625fd02e with gcc (GCC) 8.1.0 kernel signature: 1351534d0e7c48d40ccca88d31232bf867d4b70a6e7bd6064f9e5865e96943eb all runs: OK # git bisect good 6c43a4d2e7299c4c22cfc5ea3e1193a4625fd02e Bisecting: 268 revisions left to test after this (roughly 8 steps) [87b39e9c2abdb06fa22812dbf4f021c9bcfaf03d] Merge remote-tracking branch 'mmc/next' into master testing commit 87b39e9c2abdb06fa22812dbf4f021c9bcfaf03d with gcc (GCC) 8.1.0 kernel signature: e379fc01b3a7f36ac46d9a0188ca0e2f1b86c697e5848b6acc5b676541e8e8b8 all runs: crashed: BUG: unable to handle kernel NULL pointer dereference in __percpu_ref_exit # git bisect bad 87b39e9c2abdb06fa22812dbf4f021c9bcfaf03d Bisecting: 134 revisions left to test after this (roughly 7 steps) [df89a181eb354f6617489da7e729e5e3bed9a2d6] Merge branch 'for-5.10/block' into for-next testing commit df89a181eb354f6617489da7e729e5e3bed9a2d6 with gcc (GCC) 8.1.0 kernel signature: 8b58e159b0e142c02bc8fa950b6bfafe65aeeb198fc34b7590cfbb200d136c40 all runs: crashed: BUG: unable to handle kernel NULL pointer dereference in __percpu_ref_exit # git bisect bad df89a181eb354f6617489da7e729e5e3bed9a2d6 Bisecting: 65 revisions left to test after this (roughly 6 steps) [17e4aed8309ff28670271546c2c3263eb12f5eb6] bcache: remove 'int n' from parameter list of bch_bucket_alloc_set() testing commit 17e4aed8309ff28670271546c2c3263eb12f5eb6 with gcc (GCC) 8.1.0 kernel signature: 849f830cca13a2d7221ea52470a54fcbb7cf300d7c97c1b84e0912935ef36374 all runs: OK # git bisect good 17e4aed8309ff28670271546c2c3263eb12f5eb6 Bisecting: 34 revisions left to test after this (roughly 5 steps) [d296bc819b61fa403e0e8170135d95c19bbc8d43] Merge branch 'for-5.10/block' into for-next testing commit d296bc819b61fa403e0e8170135d95c19bbc8d43 with gcc (GCC) 8.1.0 kernel signature: 4b41cfd98965b308f793c121601dc127777d41beb99135036055dc24e39b1e95 all runs: OK # git bisect good d296bc819b61fa403e0e8170135d95c19bbc8d43 Bisecting: 16 revisions left to test after this (roughly 4 steps) [9bcb6114077f709564e2207bb06fb5042684ae4e] Merge branch 'for-5.10/libata' into for-next testing commit 9bcb6114077f709564e2207bb06fb5042684ae4e with gcc (GCC) 8.1.0 kernel signature: 63a6c96a8d6f8f95aa64d1a9b5a9188703a9d6cca3938d74a0396b5dc57b6093 all runs: OK # git bisect good 9bcb6114077f709564e2207bb06fb5042684ae4e Bisecting: 8 revisions left to test after this (roughly 3 steps) [92cf2fd156b273879198bb1d7e58851f822c481f] block: remove the unused blk_integrity_merge_rq export testing commit 92cf2fd156b273879198bb1d7e58851f822c481f with gcc (GCC) 8.1.0 kernel signature: 951a5cf1fa0b5a879f6fe91277dd17bd800d2586a22690d06c315bbbe4f908d8 all runs: crashed: BUG: unable to handle kernel NULL pointer dereference in __percpu_ref_exit # git bisect bad 92cf2fd156b273879198bb1d7e58851f822c481f Bisecting: 3 revisions left to test after this (roughly 2 steps) [93f221ae08381e994ac9f9ff6aa743e612e49718] block: make blk_crypto_rq_bio_prep() able to fail testing commit 93f221ae08381e994ac9f9ff6aa743e612e49718 with gcc (GCC) 8.1.0 kernel signature: 5739633cc007765536ef2a319a0c78413ec0ab5502ee49b38d41e2cc42ed12c5 all runs: OK # git bisect good 93f221ae08381e994ac9f9ff6aa743e612e49718 Bisecting: 1 revision left to test after this (roughly 1 step) [2b0d3d3e4fcfb19d10f9a82910b8f0f05c56ee3e] percpu_ref: reduce memory footprint of percpu_ref in fast path testing commit 2b0d3d3e4fcfb19d10f9a82910b8f0f05c56ee3e with gcc (GCC) 8.1.0 kernel signature: 70f2f5104b983bc6c71f817d0eccdcd54aa47c05c547c27c315e79742f496985 all runs: crashed: BUG: unable to handle kernel NULL pointer dereference in __percpu_ref_exit # git bisect bad 2b0d3d3e4fcfb19d10f9a82910b8f0f05c56ee3e Bisecting: 0 revisions left to test after this (roughly 0 steps) [cf785af19319f9fc0fc0c0604d259f1ef31b502f] block: warn if !__GFP_DIRECT_RECLAIM in bio_crypt_set_ctx() testing commit cf785af19319f9fc0fc0c0604d259f1ef31b502f with gcc (GCC) 8.1.0 kernel signature: 3f49b85ee49b65b02873a3ac40a704f4eb8da6fe45d2e87a03ca950d65411e3b all runs: OK # git bisect good cf785af19319f9fc0fc0c0604d259f1ef31b502f 2b0d3d3e4fcfb19d10f9a82910b8f0f05c56ee3e is the first bad commit commit 2b0d3d3e4fcfb19d10f9a82910b8f0f05c56ee3e Author: Ming Lei Date: Thu Oct 1 23:48:41 2020 +0800 percpu_ref: reduce memory footprint of percpu_ref in fast path 'struct percpu_ref' is often embedded into one user structure, and the instance is usually referenced in fast path, however actually only 'percpu_count_ptr' is needed in fast path. So move other fields into one new structure of 'percpu_ref_data', and allocate it dynamically via kzalloc(), then memory footprint of 'percpu_ref' in fast path is reduced a lot and becomes suitable to put into hot cacheline of user structure. Signed-off-by: Ming Lei Tested-by: Veronika Kabatova Reviewed-by: Christoph Hellwig Acked-by: Tejun Heo Cc: Sagi Grimberg Cc: Tejun Heo Cc: Christoph Hellwig Cc: Jens Axboe Cc: Bart Van Assche Signed-off-by: Jens Axboe drivers/infiniband/sw/rdmavt/mr.c | 2 +- include/linux/percpu-refcount.h | 52 +++++++-------- lib/percpu-refcount.c | 131 ++++++++++++++++++++++++++++---------- 3 files changed, 123 insertions(+), 62 deletions(-) culprit signature: 70f2f5104b983bc6c71f817d0eccdcd54aa47c05c547c27c315e79742f496985 parent signature: 3f49b85ee49b65b02873a3ac40a704f4eb8da6fe45d2e87a03ca950d65411e3b revisions tested: 27, total time: 6h23m37.680566368s (build: 2h25m42.258536126s, test: 3h54m10.61431736s) first bad commit: 2b0d3d3e4fcfb19d10f9a82910b8f0f05c56ee3e percpu_ref: reduce memory footprint of percpu_ref in fast path recipients (to): ["axboe@kernel.dk" "hch@lst.de" "ming.lei@redhat.com" "tj@kernel.org" "vkabatov@redhat.com"] recipients (cc): [] crash: BUG: unable to handle kernel NULL pointer dereference in __percpu_ref_exit RDX: 0000000000000000 RSI: 0000000020000140 RDI: 000000000000f501 RBP: 00007f85f1d00ca0 R08: 0000000000000000 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000006 R13: 00007fff624ab2cf R14: 00007f85f1d019c0 R15: 000000000118bf2c BUG: kernel NULL pointer dereference, address: 0000000000000010 #PF: supervisor read access in kernel mode #PF: error_code(0x0000) - not-present page PGD 10e0f8067 P4D 10e0f8067 PUD 10e0f9067 PMD 0 Oops: 0000 [#1] PREEMPT SMP CPU: 1 PID: 8310 Comm: syz-executor.1 Not tainted 5.9.0-rc3-syzkaller #0 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 RIP: 0010:__percpu_ref_exit+0xe/0x30 lib/percpu-refcount.c:112 Code: f2 ff ff ff eb d9 cc cc cc cc cc cc c3 0f 1f 44 00 00 66 2e 0f 1f 84 00 00 00 00 00 48 8b 07 48 83 e0 fc 74 20 53 48 8b 57 08 <48> 83 7a 10 00 75 15 48 89 fb 48 89 c7 e8 50 b4 5e ff 48 c7 03 03 RSP: 0018:ffffc900029cfe78 EFLAGS: 00010202 RAX: 0000607ed3c957e0 RBX: ffff88810e139b80 RCX: 0000000000000001 RDX: 0000000000000000 RSI: ffff88810e139da8 RDI: ffff88810e139b80 RBP: 0000000000000000 R08: 0000000000000001 R09: 0000000000000001 R10: 0000000000000000 R11: 35f462ff3b85af40 R12: 000000000001ea02 R13: ffff88810e139b98 R14: fffffffffffffff4 R15: ffff88810e5506c0 FS: 00007f85f1d01700(0000) GS:ffff88812c100000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000010 CR3: 000000010e0f7000 CR4: 00000000001506e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace: percpu_ref_exit+0xe/0x60 lib/percpu-refcount.c:133 ioctx_alloc+0x197/0x970 fs/aio.c:805 __do_sys_io_setup fs/aio.c:1329 [inline] __se_sys_io_setup fs/aio.c:1312 [inline] __x64_sys_io_setup+0x4f/0x1c0 fs/aio.c:1312 do_syscall_64+0x2d/0x70 arch/x86/entry/common.c:46 entry_SYSCALL_64_after_hwframe+0x44/0xa9 RIP: 0033:0x45de29 Code: 0d b4 fb ff c3 66 2e 0f 1f 84 00 00 00 00 00 66 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 0f 83 db b3 fb ff c3 66 2e 0f 1f 84 00 00 00 00 RSP: 002b:00007f85f1d00c78 EFLAGS: 00000246 ORIG_RAX: 00000000000000ce RAX: ffffffffffffffda RBX: 0000000000008240 RCX: 000000000045de29 RDX: 0000000000000000 RSI: 0000000020000140 RDI: 000000000000f501 RBP: 00007f85f1d00ca0 R08: 0000000000000000 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000006 R13: 00007fff624ab2cf R14: 00007f85f1d019c0 R15: 000000000118bf2c Modules linked in: CR2: 0000000000000010 ---[ end trace 49410691a60fdbc7 ]--- RIP: 0010:__percpu_ref_exit+0xe/0x30 lib/percpu-refcount.c:112 Code: f2 ff ff ff eb d9 cc cc cc cc cc cc c3 0f 1f 44 00 00 66 2e 0f 1f 84 00 00 00 00 00 48 8b 07 48 83 e0 fc 74 20 53 48 8b 57 08 <48> 83 7a 10 00 75 15 48 89 fb 48 89 c7 e8 50 b4 5e ff 48 c7 03 03 RSP: 0018:ffffc900029cfe78 EFLAGS: 00010202 RAX: 0000607ed3c957e0 RBX: ffff88810e139b80 RCX: 0000000000000001 RDX: 0000000000000000 RSI: ffff88810e139da8 RDI: ffff88810e139b80 RBP: 0000000000000000 R08: 0000000000000001 R09: 0000000000000001 R10: 0000000000000000 R11: 35f462ff3b85af40 R12: 000000000001ea02 R13: ffff88810e139b98 R14: fffffffffffffff4 R15: ffff88810e5506c0 FS: 00007f85f1d01700(0000) GS:ffff88812c100000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000010 CR3: 000000010e0f7000 CR4: 00000000001506e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400