Trying out the site without the CSS made me realize how I wasn't setting width and height for the svg icons I was using, so they became huge when the CSS was removed. I also rearranged the templates so the footer is at the end of the document (previously it was being put there by CSS). So at the very least this did lead to some fixes!
(Hopefully I don't forget to restore the CSS tomorrow!)
Screenshot of the Naked CSS version for posterity: