Mixing 32bit Outlook previews and 64bit Explorer previews is a recipe for the disaster of neither working, It may be possible (I think some may have reported eventual success in the past) but highly not recommended.
The recommendation is to ensure that if Windows is 64bit then install a 64bit Outlook.
You could try installing the 32bit preview option to only see them in outlook or just the 64bit one to see them in Explorer.
The prime difficulty you may be facing is multiple previewers of varing bitness fighting as to which one is active and to be applied to the previewer panes.
The best I can suggest is to ensure all traces of the SumatraPDF (and possibly others) are uninstalled then just pick one of the bitness you think would be more useful.
When I uninstall SumatraPDF then the Acrobat XI one takes over responsibility for Previews, I guess If that is uninstalled it then reverts to any native PDF previewer I had before.
Normally AcrobatDC will provide an adequate version for both Explorer and Outlook that has the Fit width/Whole page toggle at lower left (plus page up down) whereas the Windows native and SumatraPDF previewers don’t show any controls (SumatraPDF just shows the scrollbar when needed for multiple pages)
Thus when having the Acrobat XI one I would normally not bother using the SumatraPDF one so there is less chance of conflicts.