Adaptation is one of the key constituents of the perceptual alternation process during binocular rivalry, as it has been shown that preadapting one of the rivaling pairs before rivalry onset biases perception away from the adapted stimulus during rivalry. We investigated the influence of retinotopic and spatiotopic preadaptation on binocular rivalry. We show that for grating stimuli, preadaptation only influences rivalry when adaptation and rivalry locations are retinotopically matched. With more complex house and face stimuli, effects of preadaptation are found for both retinotopic and spatiotopic preadaptation, showing the importance of spatiotopic encoding in binocular rivalry. We show, furthermore, that adaptation to phase-scrambled faces results in retinotopic effects only, demonstrating the importance of form content for spatiotopic adaptation effects, as opposed to spatial frequency content. Are the spatiotopic adaptation influences on rivalry caused by direct spatiotopic stimulus interactions, or instead are they due to altered feedback from the adapted spatiotopic representations to the retinotopic representations that are involved in rivalry? By using rivaling face and grating stimuli that minimize rivalry between spatiotopic representations while still engaging these representations in stimulus encoding, we show that at least part of the preadaptation effects with face stimuli depend on feedback information.