Poor UX costs rankings when it degrades a signal search engines actually use or drives behavior that feeds back into ranking, and it costs only conversions when it is friction that annoys users without breaking access or touching a measurable signal. The pivot is hits-a-ranking-signal versus annoys-the-converter. A page can be irritating to use and still rank fine, because annoyance alone is not a ranking input; the same page starts losing rankings the moment its problems show up as failed Core Web Vitals, mobile unusability, content-blocking interstitials, or a behavioral pattern of searchers bouncing back to results. Because the exact weighting of these factors shifts, treat the line as a working distinction worth verifying.
Some UX problems sit squarely on the ranking side because they map to things the system measures or rewards. A slow, unstable page that fails Core Web Vitals, a layout that is hard to use on mobile, or a pop-up that blocks the content right after a searcher arrives from search all touch page-experience signals or push users back to the results, and that can pull rankings down. These are not just bad for the visitor, they are legible to the algorithm, which is exactly why fixing them protects position and not only conversion.
Other UX problems live on the conversion side because they irritate without degrading access or signals. A confusing checkout, a weak call to action, or clutter that makes the page tiring to read can quietly kill sign-ups and sales while the page still loads fast, works on mobile, and answers the query well enough that searchers stay. Google has little visibility into your funnel friction, so this kind of UX failure shows up in your revenue, not your rankings, and chasing it as a ranking fix wastes effort on the wrong problem.
For your next audit, sort UX issues into two buckets before you fix anything. Put the items that hit a ranking signal first (Core Web Vitals failures, mobile usability, intrusive interstitials, anything that makes searchers bounce back) and fix those to protect position, then turn to the conversion friction that costs you sales but leaves rankings intact. Fixing the ranking-signal problems first prevents you from losing the traffic you are trying to convert.