Depending on the theme, you can do this by going to one of these locations.
Appearance > Customize > Icons
Appearance > Customize > Social Media Icons
Appearance > Customize > Header
Appearance > Customize > Footer
Please read Social Media Icons for more information.