This article is about physically destroying and rendering into tiny pieces removable computer media other than paper. If you're interested in erasing (for secure re-use) rewritable removable computer media, see the article on erasing CDs and other computer media.
"Removable media" includes "floppy disks". You will have to take care here: a true "floppy" disk or diskette is the 5 1/4" (or larger) plastic things that came with the early PC's and with minicomputers of about the same vintage. Any medium-duty shredder will look after these if the feed throat is wide enough. (But check the manual to be sure). 3 1/2" diskettes are usually called "floppies" but that's a distinct misnomer since the things won't bend at all. As well, they have a largish metal center which actually spins the diskette once in the drive. Heavy-duty shredders will certainly handle these but if you have a lower-capacity model check the manual very carefully to see what it means by "floppy"!
Modern removable media includes CDs and CD-Rs, DVD, Blu-Ray, etc., etc. Top-of-the-line medium-duty shredders and all heavy-duty units will handle all these with aplomb.
The end-product will be a volume of hard or soft plastic shards. These may be suitable for recycling. If not they'll have to be sent off to the landfill or otherwise destroyed. If the goal is recycling there is an alternative, which is to render the recording surface of the CD unusable without otherwise damaging it. There is at least one hand-held device (the Disc eraser" by Sun-zag Creative Products) available which applies an proprietory "optical strip" on the CD surface, making it completely unreadable. One such strip is sufficient, but as it leaves a visible line on the surface multiple strips can be applied to make pleasing patterns.
If artiness is not an option or volume is an issue, there exist many machines which will physically destroy the recording surface, but leave the disk otherwise intact: important if physical proof of its destruction is required. CDs etc. disabled in this fashion are generally acceptable to recycling services. An example is the Model 1200 CD Eraser by SEM Direct of Westboro, Mass.