Council Member Kelly presses for transparency on July 1st fiber optic break
By Bob Seidenberg
Evanston’s top information security officer provided details to a city committee Monday about the severing of fiber optic cables on July 1 that left many of the city’s major public buildings without internet for three days and also affected Northwestern University.
First Ward Council Member Clare Kelly, the current chair of the city’s Administration & Public Works Committee, said she asked for the issue to be placed on the committee’s agenda “so there could be transparency as to the cause to better prevent this from happening again in the future.”
Notifications from city monitors tipped officials off
In a brief presentation, Dmitry Shub, the city’s chief information security officer, told committee members the incident occurred at 9:03 a.m. Monday, July 1, at the site of an ongoing road improvement project spearheaded by the city at Leon Place near Ridge Avenue.
The city’s Information Technology team “began seeing notifications from our network monitors of loss of communications to a number of our sites,” he said.
A third-party contractor and crew using a milling-grinding system as part of the road improvement project accidentally damaged underground telecommunications cables, Shub told the committee.
An incident response team was on site within about an hour, which he attributed “to our longstanding and very positive relationship with Northwestern University and a vendor in the area, Raptor Industries, who the city has used for many years for fiber optic cable installation and repair.”
A damage assessment was completed by 11 a.m., with an update provided to members of the city’s senior leadership team, Shub told committee members.
“We immediately began talking about business continuity – operations to keep things going because there were a number of sites affected,” he said. “Fortunately, again, thanks to the efforts of Raptor Industries and their subcontractors, they were able to begin repairs the next day.
“What’s significant about this,” Shub continued, “is that the disruption occurred leading into the July 4 holiday, a time when many folks are planning to take time off. Fortunate for us, we had a vendor that was able to get to work right away.”
System restoration
Thanks to Raptor’s efforts, according to Shub, “we were able to see 95% of our sites see their network communications restored by 1:30 a.m. on July 4, which was significant, considering that the city was hosting the July 4 parade later that day, so that was really great. It really came down to some pretty good communication units and our vendor partners.”
The city posted about the service disruption on social media after the incident, but did not release additional details for several days until responding to RoundTable questions about service being down.
‘Serious and significant risks’
Expanding on comments she made at the A&PW meeting, Kelly wrote in a statement to the RoundTable that “on July 1st, I discovered there was a City network disruption when staff at Chandler-Newberger Center were unable to process my request for a beach pass due to a broad disruption of the city’s network.
“Shortly afterwards I learned that two fiber optic data lines had been severed and that Police and Fire, our water plant and many of our parks, were experiencing network disruptions for what was expected to be for a number of days. I learned that the LEADS [Law Enforcement Agency Data System, which police use for access to criminal histories], and other systems at EPD [Evanston Police Department] were down, Fire Department radio coverage [at Station 5] and the Water Plant data operations were all down and expected to be down for at least three days.”
