Downtown building’s brick losing its painted exterior