Favor speed when the media is decorative or your audience and query are speed-sensitive, and favor rich media when it is essential to the content’s value, but the real move is to optimize the essential media rather than treat this as a clean either-or. The pivot is whether the media is essential to the value the page delivers. Once you decide that, the answer is rarely to cut the important media or to ignore speed, it is to keep what matters and make it as light as you can.
Speed should win when the media is not doing real work. A stock image that adds nothing, an autoplaying background video, or heavy decoration that only slows the page should give way to performance, especially when your readers arrive on mobile or your query is the kind people want answered fast. In those cases the media is a cost with little return, and the searcher is better served by a page that loads quickly and gets to the point than by visuals that delay the answer. Cutting or deferring that kind of media is an easy call.
Rich media should win when it is the point. A product demo, a key diagram, a tutorial video, a comparison visual that the content genuinely depends on, these carry the value the reader came for, and stripping them to shave load time would make the page worse at its actual job. But favoring them does not mean accepting whatever speed cost they bring. The right approach is to keep the essential media and optimize it, compress images, serve modern formats, size them correctly, and lazy-load or defer what is below the fold, so you preserve the value while limiting the damage to performance.
The practical move is to keep and optimize the media that is essential and cut or defer the media that is decorative. For each heavy element, ask whether the page would be meaningfully worse without it. If yes, it stays, and you put the work into making it fast. If no, remove it or defer it and let the page load lighter. Optimize before you sacrifice, and you usually do not have to choose between a fast page and a rich one.