EXT4-fs (loop7): Number of reserved GDT blocks insanely large: 17286 I/O error, dev loop5, sector 0 op 0x0:(READ) flags 0x80700 phys_seg 1 prio class 2 ======================================================== WARNING: possible irq lock inversion dependency detected 6.4.0-rc7-next-20230622 #1 Not tainted -------------------------------------------------------- systemd-udevd/102 just changed the state of lock: ffffffff85814c18 (blkg_stat_lock){+.-.}-{2:2}, at: __blkcg_rstat_flush.isra.0+0x11f/0x4e0 but this lock was taken by another, HARDIRQ-safe lock in the past: (per_cpu_ptr(&cgroup_rstat_cpu_lock, cpu)){-.-.}-{2:2} and interrupts could create inverse lock ordering between them. other info that might help us debug this: Possible interrupt unsafe locking scenario: CPU0 CPU1 ---- ---- lock(blkg_stat_lock); local_irq_disable(); lock(per_cpu_ptr(&cgroup_rstat_cpu_lock, cpu)); lock(blkg_stat_lock); lock(per_cpu_ptr(&cgroup_rstat_cpu_lock, cpu)); *** DEADLOCK *** 2 locks held by systemd-udevd/102: #0: ffffffff8560b5a0 (rcu_callback){....}-{0:0}, at: rcu_core+0x83e/0x2860 #1: ffffffff8560b6c0 (rcu_read_lock){....}-{1:2}, at: __blkcg_rstat_flush.isra.0+0x93/0x4e0 the shortest dependencies between 2nd lock and 1st lock: -> (per_cpu_ptr(&cgroup_rstat_cpu_lock, cpu)){-.-.}-{2:2} { IN-HARDIRQ-W at: lock_acquire+0x19a/0x4c0 _raw_spin_lock_irqsave+0x3a/0x60 cgroup_rstat_updated+0xcb/0x2e0 __cgroup_account_cputime_field+0xa0/0x120 account_system_index_time+0x199/0x2c0 update_process_times+0x26/0x150 tick_sched_handle+0x8e/0x170 tick_sched_timer+0xe6/0x110 __hrtimer_run_queues+0x17f/0xb60 hrtimer_interrupt+0x2ef/0x750 __sysvec_apic_timer_interrupt+0xff/0x380 sysvec_apic_timer_interrupt+0x69/0x90 asm_sysvec_apic_timer_interrupt+0x1a/0x20 lock_acquire+0x1c7/0x4c0 __mutex_lock+0x12a/0x1a30 __flush_workqueue+0x137/0x1110 cgroup_procs_write_finish+0xeb/0x170 __cgroup_procs_write+0x371/0x6b0 cgroup_procs_write+0x26/0x60 cgroup_file_write+0x1e5/0x7c0 kernfs_fop_write_iter+0x3f7/0x610 vfs_write+0x97d/0xda0 ksys_write+0x122/0x250 do_syscall_64+0x3f/0x90 entry_SYSCALL_64_after_hwframe+0x6e/0xd8 IN-SOFTIRQ-W at: lock_acquire+0x19a/0x4c0 _raw_spin_lock_irqsave+0x3a/0x60 cgroup_rstat_updated+0xcb/0x2e0 __cgroup_account_cputime+0x75/0xc0 update_curr+0x350/0x6d0 dequeue_task_fair+0x20e/0x14a0 load_balance+0xcb4/0x2790 rebalance_domains+0x66c/0xc00 __do_softirq+0x1b7/0x7d4 irq_exit_rcu+0x93/0xc0 sysvec_apic_timer_interrupt+0x6e/0x90 asm_sysvec_apic_timer_interrupt+0x1a/0x20 __read_once_word_nocheck+0x3/0x10 unwind_next_frame+0xc66/0x2490 __unwind_start+0x513/0x7c0 arch_stack_walk+0x63/0xf0 stack_trace_save+0x90/0xd0 kasan_save_stack+0x22/0x50 kasan_set_track+0x25/0x30 __kasan_slab_alloc+0x59/0x70 kmem_cache_alloc+0x16b/0x370 __create_object+0x3c/0xc90 kmem_cache_alloc+0x20b/0x370 anon_vma_fork+0x1f4/0x630 dup_mmap+0xe49/0x1770 copy_process+0x3e68/0x7320 kernel_clone+0xeb/0x7d0 __do_sys_clone+0xba/0x100 do_syscall_64+0x3f/0x90 entry_SYSCALL_64_after_hwframe+0x6e/0xd8 INITIAL USE at: lock_acquire+0x19a/0x4c0 _raw_spin_lock_irqsave+0x3a/0x60 cgroup_rstat_flush_locked+0x131/0xd80 cgroup_rstat_flush+0x37/0x50 do_flush_stats+0x97/0xf0 flush_memcg_stats_dwork+0x9/0x50 process_one_work+0xabf/0x1770 worker_thread+0x64f/0x12a0 kthread+0x33f/0x440 ret_from_fork+0x2c/0x50 } ... key at: [] __key.0+0x0/0x40 ... acquired at: _raw_spin_lock+0x2b/0x40 __blkcg_rstat_flush.isra.0+0x11f/0x4e0 blkcg_rstat_flush+0x87/0xb0 cgroup_rstat_flush_locked+0x706/0xd80 cgroup_rstat_flush+0x37/0x50 do_flush_stats+0x97/0xf0 flush_memcg_stats_dwork+0x9/0x50 process_one_work+0xabf/0x1770 worker_thread+0x64f/0x12a0 kthread+0x33f/0x440 ret_from_fork+0x2c/0x50 -> (blkg_stat_lock){+.-.}-{2:2} { HARDIRQ-ON-W at: lock_acquire+0x19a/0x4c0 _raw_spin_lock+0x2b/0x40 __blkcg_rstat_flush.isra.0+0x11f/0x4e0 __blkg_release+0xfa/0x3b0 rcu_core+0x8c8/0x2860 __do_softirq+0x1b7/0x7d4 irq_exit_rcu+0x93/0xc0 sysvec_apic_timer_interrupt+0x6e/0x90 asm_sysvec_apic_timer_interrupt+0x1a/0x20 kasan_check_range+0x59/0x1c0 __up_read+0x11a/0x750 kernfs_iop_permission+0xe4/0x120 inode_permission.part.0+0x29a/0x520 link_path_walk.part.0+0x85e/0xd90 path_lookupat+0xb7/0x850 filename_lookup+0x1d2/0x590 user_path_at_empty+0x46/0x70 do_readlinkat+0xcd/0x2f0 __x64_sys_readlinkat+0x97/0x100 do_syscall_64+0x3f/0x90 entry_SYSCALL_64_after_hwframe+0x6e/0xd8 IN-SOFTIRQ-W at: lock_acquire+0x19a/0x4c0 _raw_spin_lock+0x2b/0x40 __blkcg_rstat_flush.isra.0+0x11f/0x4e0 __blkg_release+0xfa/0x3b0 rcu_core+0x8c8/0x2860 __do_softirq+0x1b7/0x7d4 irq_exit_rcu+0x93/0xc0 sysvec_apic_timer_interrupt+0x6e/0x90 asm_sysvec_apic_timer_interrupt+0x1a/0x20 kasan_check_range+0x59/0x1c0 __up_read+0x11a/0x750 kernfs_iop_permission+0xe4/0x120 inode_permission.part.0+0x29a/0x520 link_path_walk.part.0+0x85e/0xd90 path_lookupat+0xb7/0x850 filename_lookup+0x1d2/0x590 user_path_at_empty+0x46/0x70 do_readlinkat+0xcd/0x2f0 __x64_sys_readlinkat+0x97/0x100 do_syscall_64+0x3f/0x90 entry_SYSCALL_64_after_hwframe+0x6e/0xd8 INITIAL USE at: lock_acquire+0x19a/0x4c0 _raw_spin_lock+0x2b/0x40 __blkcg_rstat_flush.isra.0+0x11f/0x4e0 blkcg_rstat_flush+0x87/0xb0 cgroup_rstat_flush_locked+0x706/0xd80 cgroup_rstat_flush+0x37/0x50 do_flush_stats+0x97/0xf0 flush_memcg_stats_dwork+0x9/0x50 process_one_work+0xabf/0x1770 worker_thread+0x64f/0x12a0 kthread+0x33f/0x440 ret_from_fork+0x2c/0x50 } ... key at: [] blkg_stat_lock+0x18/0x60 ... acquired at: __lock_acquire+0x8b8/0x6340 lock_acquire+0x19a/0x4c0 _raw_spin_lock+0x2b/0x40 __blkcg_rstat_flush.isra.0+0x11f/0x4e0 __blkg_release+0xfa/0x3b0 rcu_core+0x8c8/0x2860 __do_softirq+0x1b7/0x7d4 irq_exit_rcu+0x93/0xc0 sysvec_apic_timer_interrupt+0x6e/0x90 asm_sysvec_apic_timer_interrupt+0x1a/0x20 kasan_check_range+0x59/0x1c0 __up_read+0x11a/0x750 kernfs_iop_permission+0xe4/0x120 inode_permission.part.0+0x29a/0x520 link_path_walk.part.0+0x85e/0xd90 path_lookupat+0xb7/0x850 filename_lookup+0x1d2/0x590 user_path_at_empty+0x46/0x70 do_readlinkat+0xcd/0x2f0 __x64_sys_readlinkat+0x97/0x100 do_syscall_64+0x3f/0x90 entry_SYSCALL_64_after_hwframe+0x6e/0xd8 stack backtrace: CPU: 0 PID: 102 Comm: systemd-udevd Not tainted 6.4.0-rc7-next-20230622 #1 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014 Call Trace: dump_stack_lvl+0x91/0xf0 print_irq_inversion_bug.part.0+0x3d5/0x570 mark_lock.part.0+0x900/0x2f50 __lock_acquire+0x8b8/0x6340 lock_acquire+0x19a/0x4c0 _raw_spin_lock+0x2b/0x40 __blkcg_rstat_flush.isra.0+0x11f/0x4e0 __blkg_release+0xfa/0x3b0 rcu_core+0x8c8/0x2860 __do_softirq+0x1b7/0x7d4 irq_exit_rcu+0x93/0xc0 sysvec_apic_timer_interrupt+0x6e/0x90 asm_sysvec_apic_timer_interrupt+0x1a/0x20 RIP: 0010:kasan_check_range+0x59/0x1c0 Code: 77 23 44 89 c2 e8 97 ea ff ff 83 f0 01 5b 5d 41 5c e9 4b bb d2 02 48 b8 ff ff ff ff ff 7f ff ff 48 39 c7 76 dd 4c 8d 54 37 ff <48> 89 fd 48 b8 00 00 00 00 00 fc ff df 4d 89 d1 48 c1 ed 03 49 c1 RSP: 0018:ffff8880146df9b0 EFLAGS: 00000212 RAX: ffff7fffffffffff RBX: ffffffff85d3f20c RCX: ffffffff812bfb4a RDX: 0000000000000000 RSI: 0000000000000008 RDI: ffff888008784978 RBP: ffff888008784970 R08: 0000000000000000 R09: ffffed10010f092f R10: ffff88800878497f R11: 0000000000000001 R12: 1ffff110028dbf3d R13: ffff8880087849d8 R14: ffff88800c8fdcb0 R15: ffff888008784978 __up_read+0x11a/0x750 kernfs_iop_permission+0xe4/0x120 inode_permission.part.0+0x29a/0x520 link_path_walk.part.0+0x85e/0xd90 path_lookupat+0xb7/0x850 filename_lookup+0x1d2/0x590 user_path_at_empty+0x46/0x70 do_readlinkat+0xcd/0x2f0 __x64_sys_readlinkat+0x97/0x100 do_syscall_64+0x3f/0x90 entry_SYSCALL_64_after_hwframe+0x6e/0xd8 RIP: 0033:0x7f59b57aeb2a Code: 48 8b 0d 69 d3 0c 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 49 89 ca b8 0b 01 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 36 d3 0c 00 f7 d8 64 89 01 48 RSP: 002b:00007ffeed43bd28 EFLAGS: 00000206 ORIG_RAX: 000000000000010b RAX: ffffffffffffffda RBX: 0000000000001001 RCX: 00007f59b57aeb2a RDX: 000055a7a6dc3300 RSI: 00007ffeed43bdb0 RDI: 00000000ffffff9c RBP: 000055a7a6dc3300 R08: 000055a7a6dc3300 R09: 00007f59b587cbe0 R10: 0000000000001000 R11: 0000000000000206 R12: 00007ffeed43bdb0 R13: 00000000ffffff9c R14: 0000000000000001 R15: 0000000000001000 netlink: 8 bytes leftover after parsing attributes in process `syz-executor.3'. loop7: detected capacity change from 0 to 16 loop6: detected capacity change from 0 to 512 EXT4-fs (loop7): Number of reserved GDT blocks insanely large: 17286 loop5: detected capacity change from 0 to 48 loop6: detected capacity change from 0 to 512 ---------------- Code disassembly (best guess): 0: 77 23 ja 0x25 2: 44 89 c2 mov %r8d,%edx 5: e8 97 ea ff ff callq 0xffffeaa1 a: 83 f0 01 xor $0x1,%eax d: 5b pop %rbx e: 5d pop %rbp f: 41 5c pop %r12 11: e9 4b bb d2 02 jmpq 0x2d2bb61 16: 48 b8 ff ff ff ff ff movabs $0xffff7fffffffffff,%rax 1d: 7f ff ff 20: 48 39 c7 cmp %rax,%rdi 23: 76 dd jbe 0x2 25: 4c 8d 54 37 ff lea -0x1(%rdi,%rsi,1),%r10 * 2a: 48 89 fd mov %rdi,%rbp <-- trapping instruction 2d: 48 b8 00 00 00 00 00 movabs $0xdffffc0000000000,%rax 34: fc ff df 37: 4d 89 d1 mov %r10,%r9 3a: 48 c1 ed 03 shr $0x3,%rbp 3e: 49 rex.WB 3f: c1 .byte 0xc1