Mobile usability moves rankings when it breaks access to the content, and barely registers when it does not. Calling mobile usability a huge ranking factor across the board overstates it; the effect is sharp at one end and negligible at the other. What decides it is whether the mobile experience breaks access or merely annoys, and the reason that line is so decisive comes down to which version Google actually indexes.
Under mobile-first indexing, the mobile version is the one Google crawls and ranks. So anything that strips or hides content on mobile costs rankings directly, because Google is reading the reduced version. Content tucked behind taps that never renders, key text or descriptions dropped on small screens, resources blocked so the layout cannot render, a crawler that stops reading before it reaches the main body; these genuinely cost position, because they shrink what Google sees of the page. The damage is not cosmetic, it is missing content and missing signals.
At the other end, minor imperfections barely move anything. Touch targets a little small, a slightly awkward layout, spacing that could be tighter; these are user-experience friction that people tolerate, not access failures that change what Google can read. A page that looks imperfect on a phone but serves its full content and signals is in a very different situation from a page that withholds half of itself on mobile. The standalone idea of a mobile usability score doing heavy ranking work is weaker than the simple fact that the mobile version has to carry the whole page.
So the reader checks whether mobile users, and the mobile crawler, can actually reach the content, rather than whether the page looks perfect, and treats broken access as the thing that costs rankings while cosmetic friction stays a user concern.