XLR vs RCA
Both cables have the same job: Get the signal from your receiver/preamp to your subwoofer. How they do it is a little different: RCA cables are unbalanced, while XLR cables are balanced. We could get very technical here, but in short it means that XLR cables are better at rejecting noise than RCA cables. XLR cables are more commonly used in the Pro audio scene due their ability to maintain signal integrity without adding noise even over longer distances.
Which is right for me?
In most cases, RCA is perfectly fine. However: If your receiver has XLR subwoofer outputs, there are no downsides to using them. If you have both, and experience noise when using RCA cables, you can most likely get rid of it by using XLR cables.
Can I use an RCA to XLR adapter?
You can, but there will not be any benefits compared to a normal RCA cable. In this case, you just as well stick with a regular RCA cable.
