The “Pre-viewers” other than PDF are not active in a public release.
They work fairly well with most eBook covers but for example may not show ALL CBRar5 covers.
They can be added by installing a SumatraPDF developer version however
If only needing those CBZ (ZiP) and ePub (Zip) covers it may be just as easy to load other previewers for those similar formats. CDisplayEx / CBXshell and their grandchildren clones can usually build THUMBNAILS only (Not quick/pre-view) You could look at recent GitHub - dark-knight404/DarkThumbs: Thumbnail preview for EPUB, CBZ, CBR, RAR and ZIP image archives which ads CBR but do read about the currently known issue with multi-file.epub and I also had some cbr5 files not showing
Dare I mention potentially the best serious solution is to try Mystic Thumbs (circa $35) 14 day FREE fully functional evaluation so no commitment to give credit card first.
Like SumatraPDF it uses GhostScript but unlike SumatraPDF it uses that for (.AI .PDF .EPS .PS etc.) thumbnails and previews. It currently has better CBR support, but apparently none for FB2.
There are many SumatraPDF formats it does not support and vise versa
Personally I am happy with the default MSedge PDF & TXT previews as I use portable and they are the only 2 formats I need to quickly skim read pages without opening them.