Public pushes back on proposed gym demolition