A lot of talk around Async Compute is very misleading. To developers, it's not even about fully utilizing shaders at all, because such a focus will result in no gains on GPUs which have less shaders (as individually, the shaders are better utilized).
Think of DX12 & Vulkan as a Multi-Engine API that fully access GPUs sub-units. GPUs have various units, you may know these from reading reviews with diagrams, Shaders, Rasterizers (ROPs) and DMAs (Direct Memory Access units to stream data). In serial API like DX11 and older, if you run work on Shaders, you cannot run work on Rasterizers or DMA at the same time. Likewise if you stream assets from memory to prepare a frame, you can't run Shaders or Rasterizer. Yes, it's that stupid if you think about it, but it was the way things were done, one step at a time.
There's a much better explanation that what I can come up with here in this video:
Asynchronous Compute in DX12 & Vulkan: Dispelling Myths & Misconceptions Concurrently - YouTube
^ The sources in this video come from various IHVs (AMD ofc, Sony and Microsoft) and developers. It should explain what Async Compute is all about, it certainly helped me understand it better.
We often see devs on consoles hype up Async Compute, seeing major gains for them, but you all know console hardware is very low shader count. Console API also are fully multi-threaded and so shader utilization isn't an issue.
Likewise, with Vulkan in Doom on the RX 480 and even older GCN like Tahiti 7970/280X seeing massive gains, these lower shader-count GCN would get little gains if Async Compute was only about shader utilization.
Ultimately, think of it as a Multi-Engine API, tapping the Shaders, Rasterizers and DMAs simultaneously in parallel.
