Took me a while to find this thread:
But setting my hover images wrapper to pointer-events: none worked like a charm.