HARRISBURG — The head of Pennsylvania’s Office of Administration stressed to lawmakers in the state Senate on Wednesday that the data loss caused by human error on servers for the state police and a retirement system was not the result of a breach, hack or cybersecurity incident.
Neil Weaver, secretary of the Office of Administration (OA), testified at a joint meeting of the committees on Communications & Technology and State Government. He explained that human error occurred during routine server maintenance on Jan. 3, erasing data from 77 servers.
The result was a “limited data loss incident,” one in which impacted data has since been recovered or restored on all servers but one.
Data was lost for two systems used by the Pennsylvania State Police to log evidence and manage case information and one system for the State Employees Retirement System (SERS) for its member account login.
The systems shared a single server for which data couldn’t be recovered.
“This was not a data breach. This was not a hack. And, this was not a cybersecurity incident compromising any resident data,” Weaver said. “This was an incredible, serious human error that I do not take lightly.”
Efforts are underway to reconstruct the lost data for state police, with both the Office of Administration and state police noting that all physical evidence is secured and wasn’t compromised as a result of the incident.
Meanwhile, SERS said no member data was accessed or stolen. The 98,000 impacted members must verify their identities and resubmit new login codes to access their accounts — a common practice for reauthorization online.
The OA entered an emergency contract with consultant Layer Aleph of Seattle, Wash., at a cost of $530,000 to guide crisis engineering and aid in data recovery.
The incident is unrelated to an ongoing attack against the Pennsylvania court system, described as a denial of service attack as an unknown entity floods the system with online traffic and disrupts certain online services. Cyber attacks have also recently been committed against municipal entities in Bucks County and Aliquippa.
A bill cleared the state Senate on Tuesday unopposed that would require that state government purchases of computer hardware meet the standards of the National Institute of Standards and Technology to reduce the risk of cyber attacks. It’s now with the state House for further consideration.
On Wednesday, the hearing focused on the incident repeatedly attributed to human error.
State senators grilled Weaver as well as Chief Information Officer Amaya Capellán and Chief Information Security Officer Jim Sipe about the cause of the incident, post-incident accountability and the lack of notification for state lawmakers.
Citing security concerns and personnel privacy, the OA administrators often declined answers to questions about the system and what occurred, both before and after the deletion. And, they declined to answer questions related to the employee involved.
While Weaver wouldn’t say whether the employee was fired, citing personnel privacy, PennLive previously reported the employee was fired.
“Was that self-reported, the deletion?” Sen. Jarrett Coleman, R-Bucks/Lehigh, asked as he probed for information about the cause and the employee involved.
“Yes,” Weaver replied.
“The individual realized their error immediately and immediately notified their supervisor,” Capellán said, later explaining that server deletion isn’t uncommon as part of routine duties for a server administrator but that in this case, “the step was taken at an inappropriate time.”
The administrators did defend the concept of remote work, saying the information technology industry shifted to secure remote work years before the COVID-19 pandemic further normalized the practice in other industries.
“It’s our belief that the location of the employee didn’t have any impact in this incident,” Capellán said, adding that the work performed can be done safely and securely off-site.
Sen. Cris Dush, R-Cameron/Centre/Clinton/Elk/Jefferson/McKean, pressed Weaver about why he hadn’t been notified for four days about the deleted data. Weaver explained that recovery efforts were ongoing and successful up until the last server. At that point, he said he was notified.
“The fact that different agencies were involved, I think you should have been notified immediately,” Dush said. “I think that needs to be addressed.”
Dush was critical about lawmakers not being notified in a timely fashion while Sen. Mike Regan, R-Cumberland/York, shared the same criticism as to why the Office of Attorney General wasn’t notified in order to contact county district attorneys concerning the state police data issue; rather, word was spread through the state district attorney’s association, he said.