======================================================== WARNING: possible irq lock inversion dependency detected 6.4.0-rc7-next-20230623 #1 Not tainted -------------------------------------------------------- syz-executor.2/12221 just changed the state of lock: ffffffff85815818 (blkg_stat_lock){+.-.}-{2:2}, at: __blkcg_rstat_flush.isra.0+0x11f/0x4e0 but this lock was taken by another, HARDIRQ-safe lock in the past: (per_cpu_ptr(&cgroup_rstat_cpu_lock, cpu)){-.-.}-{2:2} and interrupts could create inverse lock ordering between them. other info that might help us debug this: Possible interrupt unsafe locking scenario: CPU0 CPU1 ---- ---- lock(blkg_stat_lock); local_irq_disable(); lock(per_cpu_ptr(&cgroup_rstat_cpu_lock, cpu)); lock(blkg_stat_lock); lock(per_cpu_ptr(&cgroup_rstat_cpu_lock, cpu)); *** DEADLOCK *** 4 locks held by syz-executor.2/12221: #0: ffff888009b723e8 (sb_writers#3){.+.+}-{0:0}, at: __x64_sys_sendfile64+0x1d5/0x210 #1: ffff88801b6f0f88 (mapping.invalidate_lock){++++}-{3:3}, at: page_cache_ra_unbounded+0x157/0x5e0 #2: ffffffff8560b5a0 (rcu_callback){....}-{0:0}, at: rcu_core+0x83e/0x28b0 #3: ffffffff8560b6c0 (rcu_read_lock){....}-{1:2}, at: __blkcg_rstat_flush.isra.0+0x93/0x4e0 the shortest dependencies between 2nd lock and 1st lock: -> (per_cpu_ptr(&cgroup_rstat_cpu_lock, cpu)){-.-.}-{2:2} { IN-HARDIRQ-W at: lock_acquire+0x19a/0x4c0 _raw_spin_lock_irqsave+0x3a/0x60 cgroup_rstat_updated+0xcb/0x2e0 __cgroup_account_cputime_field+0xa0/0x120 account_system_index_time+0x199/0x2c0 update_process_times+0x26/0x150 tick_sched_handle+0x8e/0x170 tick_sched_timer+0xe6/0x110 __hrtimer_run_queues+0x17f/0xb60 hrtimer_interrupt+0x2ef/0x750 __sysvec_apic_timer_interrupt+0xff/0x380 sysvec_apic_timer_interrupt+0x69/0x90 asm_sysvec_apic_timer_interrupt+0x1a/0x20 lock_is_held_type+0xe4/0x120 __d_lookup_rcu+0x395/0x4c0 d_alloc_parallel+0x3b8/0x1640 lookup_open.isra.0+0x909/0x1400 path_openat+0x96c/0x2710 do_filp_open+0x1ba/0x410 do_sys_openat2+0x164/0x1d0 __x64_sys_openat+0x143/0x200 do_syscall_64+0x3f/0x90 entry_SYSCALL_64_after_hwframe+0x6e/0xd8 IN-SOFTIRQ-W at: lock_acquire+0x19a/0x4c0 _raw_spin_lock_irqsave+0x3a/0x60 cgroup_rstat_updated+0xcb/0x2e0 __cgroup_account_cputime+0x75/0xc0 update_curr+0x350/0x6d0 dequeue_task_fair+0x20e/0x14a0 load_balance+0xcb4/0x2790 rebalance_domains+0x66c/0xc00 __do_softirq+0x1b7/0x7d4 irq_exit_rcu+0x93/0xc0 sysvec_apic_timer_interrupt+0x6e/0x90 asm_sysvec_apic_timer_interrupt+0x1a/0x20 lock_is_held_type+0x10/0x120 xas_reload+0x275/0x440 next_uptodate_page+0x14a/0x470 filemap_map_pages+0x6da/0xdd0 __handle_mm_fault+0x1c9d/0x2d20 handle_mm_fault+0x1af/0xba0 do_user_addr_fault+0x58e/0x12f0 exc_page_fault+0x9c/0x1a0 asm_exc_page_fault+0x26/0x30 INITIAL USE at: lock_acquire+0x19a/0x4c0 _raw_spin_lock_irqsave+0x3a/0x60 cgroup_rstat_flush_locked+0x131/0xd80 cgroup_rstat_flush+0x37/0x50 do_flush_stats+0x97/0xf0 flush_memcg_stats_dwork+0x9/0x50 process_one_work+0xabf/0x1770 worker_thread+0x64f/0x12a0 kthread+0x33f/0x440 ret_from_fork+0x2c/0x50 } ... key at: [] __key.0+0x0/0x40 ... acquired at: _raw_spin_lock+0x2b/0x40 __blkcg_rstat_flush.isra.0+0x11f/0x4e0 blkcg_rstat_flush+0x87/0xb0 cgroup_rstat_flush_locked+0x706/0xd80 cgroup_rstat_flush+0x37/0x50 do_flush_stats+0x97/0xf0 mem_cgroup_wb_stats+0x3d8/0x4b0 wb_over_bg_thresh+0x30e/0x6a0 wb_workfn+0x486/0xed0 process_one_work+0xabf/0x1770 worker_thread+0x64f/0x12a0 kthread+0x33f/0x440 ret_from_fork+0x2c/0x50 -> (blkg_stat_lock){+.-.}-{2:2} { HARDIRQ-ON-W at: lock_acquire+0x19a/0x4c0 _raw_spin_lock+0x2b/0x40 __blkcg_rstat_flush.isra.0+0x11f/0x4e0 __blkg_release+0xfa/0x3b0 rcu_core+0x8c8/0x28b0 __do_softirq+0x1b7/0x7d4 irq_exit_rcu+0x93/0xc0 sysvec_apic_timer_interrupt+0x6e/0x90 asm_sysvec_apic_timer_interrupt+0x1a/0x20 xa_load+0x16d/0x2d0 page_cache_ra_unbounded+0x2bf/0x5e0 ondemand_readahead+0x8d7/0x1110 page_cache_async_ra+0xfc/0x130 filemap_get_pages+0x6b1/0x1560 filemap_splice_read+0x3ca/0x990 ext4_file_splice_read+0x101/0x160 vfs_splice_read+0x238/0x2d0 splice_direct_to_actor+0x2aa/0x9d0 do_splice_direct+0x1bb/0x290 do_sendfile+0xb67/0x1300 __x64_sys_sendfile64+0x1d5/0x210 do_syscall_64+0x3f/0x90 entry_SYSCALL_64_after_hwframe+0x6e/0xd8 IN-SOFTIRQ-W at: lock_acquire+0x19a/0x4c0 _raw_spin_lock+0x2b/0x40 __blkcg_rstat_flush.isra.0+0x11f/0x4e0 __blkg_release+0xfa/0x3b0 rcu_core+0x8c8/0x28b0 __do_softirq+0x1b7/0x7d4 irq_exit_rcu+0x93/0xc0 sysvec_apic_timer_interrupt+0x6e/0x90 asm_sysvec_apic_timer_interrupt+0x1a/0x20 xa_load+0x16d/0x2d0 page_cache_ra_unbounded+0x2bf/0x5e0 ondemand_readahead+0x8d7/0x1110 page_cache_async_ra+0xfc/0x130 filemap_get_pages+0x6b1/0x1560 filemap_splice_read+0x3ca/0x990 ext4_file_splice_read+0x101/0x160 vfs_splice_read+0x238/0x2d0 splice_direct_to_actor+0x2aa/0x9d0 do_splice_direct+0x1bb/0x290 do_sendfile+0xb67/0x1300 __x64_sys_sendfile64+0x1d5/0x210 do_syscall_64+0x3f/0x90 entry_SYSCALL_64_after_hwframe+0x6e/0xd8 INITIAL USE at: lock_acquire+0x19a/0x4c0 _raw_spin_lock+0x2b/0x40 __blkcg_rstat_flush.isra.0+0x11f/0x4e0 blkcg_rstat_flush+0x87/0xb0 cgroup_rstat_flush_locked+0x706/0xd80 cgroup_rstat_flush+0x37/0x50 do_flush_stats+0x97/0xf0 mem_cgroup_wb_stats+0x3d8/0x4b0 wb_over_bg_thresh+0x30e/0x6a0 wb_workfn+0x486/0xed0 process_one_work+0xabf/0x1770 worker_thread+0x64f/0x12a0 kthread+0x33f/0x440 ret_from_fork+0x2c/0x50 } ... key at: [] blkg_stat_lock+0x18/0x60 ... acquired at: __lock_acquire+0x8b8/0x6340 lock_acquire+0x19a/0x4c0 _raw_spin_lock+0x2b/0x40 __blkcg_rstat_flush.isra.0+0x11f/0x4e0 __blkg_release+0xfa/0x3b0 rcu_core+0x8c8/0x28b0 __do_softirq+0x1b7/0x7d4 irq_exit_rcu+0x93/0xc0 sysvec_apic_timer_interrupt+0x6e/0x90 asm_sysvec_apic_timer_interrupt+0x1a/0x20 xa_load+0x16d/0x2d0 page_cache_ra_unbounded+0x2bf/0x5e0 ondemand_readahead+0x8d7/0x1110 page_cache_async_ra+0xfc/0x130 filemap_get_pages+0x6b1/0x1560 filemap_splice_read+0x3ca/0x990 ext4_file_splice_read+0x101/0x160 vfs_splice_read+0x238/0x2d0 splice_direct_to_actor+0x2aa/0x9d0 do_splice_direct+0x1bb/0x290 do_sendfile+0xb67/0x1300 __x64_sys_sendfile64+0x1d5/0x210 do_syscall_64+0x3f/0x90 entry_SYSCALL_64_after_hwframe+0x6e/0xd8 stack backtrace: CPU: 1 PID: 12221 Comm: syz-executor.2 Not tainted 6.4.0-rc7-next-20230623 #1 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014 Call Trace: dump_stack_lvl+0x91/0xf0 print_irq_inversion_bug.part.0+0x3d5/0x570 mark_lock.part.0+0x900/0x2f50 __lock_acquire+0x8b8/0x6340 lock_acquire+0x19a/0x4c0 _raw_spin_lock+0x2b/0x40 __blkcg_rstat_flush.isra.0+0x11f/0x4e0 __blkg_release+0xfa/0x3b0 rcu_core+0x8c8/0x28b0 __do_softirq+0x1b7/0x7d4 irq_exit_rcu+0x93/0xc0 sysvec_apic_timer_interrupt+0x6e/0x90 asm_sysvec_apic_timer_interrupt+0x1a/0x20 RIP: 0010:xa_load+0x16d/0x2d0 Code: e9 f0 fa fc 48 c7 c6 48 6e 51 84 48 c7 c7 c0 b6 60 85 e8 d6 53 dc fc 48 b8 00 00 00 00 00 fc ff df 48 01 c3 c7 03 00 00 00 00 <48> c7 43 08 00 00 00 00 48 8b 84 24 98 00 00 00 65 48 2b 04 25 28 RSP: 0018:ffff88803eee75f8 EFLAGS: 00000282 RAX: dffffc0000000000 RBX: ffffed1007ddcebf RCX: ffff88803eee7580 RDX: 1ffff11007911b74 RSI: 0000000000000000 RDI: ffff88803c88dba0 RBP: 0000000000000001 R08: 0000000000000001 R09: ffff88803c88dba8 R10: ffffffff85d40917 R11: 0000000000000001 R12: 0000000000000000 R13: 0000000000000020 R14: 0000000000000038 R15: 0000000000112cca page_cache_ra_unbounded+0x2bf/0x5e0 ondemand_readahead+0x8d7/0x1110 page_cache_async_ra+0xfc/0x130 filemap_get_pages+0x6b1/0x1560 filemap_splice_read+0x3ca/0x990 ext4_file_splice_read+0x101/0x160 vfs_splice_read+0x238/0x2d0 splice_direct_to_actor+0x2aa/0x9d0 do_splice_direct+0x1bb/0x290 do_sendfile+0xb67/0x1300 __x64_sys_sendfile64+0x1d5/0x210 do_syscall_64+0x3f/0x90 entry_SYSCALL_64_after_hwframe+0x6e/0xd8 RIP: 0033:0x7fdbcc578b19 Code: ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 bc ff ff ff f7 d8 64 89 01 48 RSP: 002b:00007fdbc9aee188 EFLAGS: 00000246 ORIG_RAX: 0000000000000028 RAX: ffffffffffffffda RBX: 00007fdbcc68bf60 RCX: 00007fdbcc578b19 RDX: 0000000000000000 RSI: 0000000000000003 RDI: 0000000000000005 RBP: 00007fdbcc5d2f6d R08: 0000000000000000 R09: 0000000000000000 R10: 00000000fffffdab R11: 0000000000000246 R12: 0000000000000000 R13: 00007ffe299c3aff R14: 00007fdbc9aee300 R15: 0000000000022000 program syz-executor.2 is using a deprecated SCSI ioctl, please convert it to SG_IO program syz-executor.2 is using a deprecated SCSI ioctl, please convert it to SG_IO program syz-executor.2 is using a deprecated SCSI ioctl, please convert it to SG_IO program syz-executor.7 is using a deprecated SCSI ioctl, please convert it to SG_IO program syz-executor.2 is using a deprecated SCSI ioctl, please convert it to SG_IO program syz-executor.7 is using a deprecated SCSI ioctl, please convert it to SG_IO program syz-executor.7 is using a deprecated SCSI ioctl, please convert it to SG_IO loop6: detected capacity change from 0 to 40 Process accounting resumed Process accounting resumed Process accounting resumed loop6: detected capacity change from 0 to 40 Process accounting resumed Process accounting resumed loop6: detected capacity change from 0 to 40 Process accounting resumed Process accounting resumed loop6: detected capacity change from 0 to 40 Process accounting resumed Process accounting resumed Process accounting resumed Process accounting resumed Process accounting resumed Process accounting resumed Process accounting resumed Process accounting resumed Process accounting resumed loop6: detected capacity change from 0 to 40 Process accounting resumed Process accounting resumed Process accounting resumed Process accounting resumed loop6: detected capacity change from 0 to 40 Process accounting resumed loop6: detected capacity change from 0 to 40 Process accounting resumed Process accounting resumed ---------------- Code disassembly (best guess): 0: e9 f0 fa fc 48 jmpq 0x48fcfaf5 5: c7 c6 48 6e 51 84 mov $0x84516e48,%esi b: 48 c7 c7 c0 b6 60 85 mov $0xffffffff8560b6c0,%rdi 12: e8 d6 53 dc fc callq 0xfcdc53ed 17: 48 b8 00 00 00 00 00 movabs $0xdffffc0000000000,%rax 1e: fc ff df 21: 48 01 c3 add %rax,%rbx 24: c7 03 00 00 00 00 movl $0x0,(%rbx) * 2a: 48 c7 43 08 00 00 00 movq $0x0,0x8(%rbx) <-- trapping instruction 31: 00 32: 48 8b 84 24 98 00 00 mov 0x98(%rsp),%rax 39: 00 3a: 65 gs 3b: 48 rex.W 3c: 2b .byte 0x2b 3d: 04 25 add $0x25,%al 3f: 28 .byte 0x28