Optus reveals 'routine software upgrade' caused national outage

In a statement the telco said routers weren't able to handle changes to the network and disconnected to protect themselves, leading to the nine-hour outage on November 8.

"The restoration required a large-scale effort of the team and in some cases required Optus to reconnect or reboot routers physically, requiring the dispatch of people across a number of sites in Australia.

"This is why restoration was progressive over the afternoon."

The outage began at about 4am and affected health and emergency services, government, schools, public transport and businesses.

Optus said the widespread impact of the outage meant investigations into the issue "took longer than we would have liked".

"We are committed to learning from what has occurred and continuing to work with our international vendors and partners to increase the resilience of our network," the statement said.

"We will also support and fully cooperate with the reviews being undertaken by the Government and the Senate."

RMIT University's associate professor Mark Gregory said Optus had identified the cause as "human error".

While this ruled out hardware failure or a cyber attack, he said questions still remained unanswered by the telco.

"Optus has not explained what went wrong with the test process that should have occurred before the routing software upgrade occurred," he said.

"Also, there is no explanation as to why there appears to have been a lack of redundancy of the key routers, so that if there was a problem the key routers would swap to the redundant routers, which you would expect to be running the previous iteration of software."

IN OTHER NEWS:

Research fellow at the Centre for Defence Communications and Information Networking at The University of Adelaide Mark Stewart said Optus had faced a "well-known and predictable problem... commonly associated with software updates".

"A major telco should have a disaster recovery plan which is more sophisticated than your average corporate network," he said.

"At a minimum, they should have had a plan to revert the changes or remotely reboot their systems."

Following the disruption the telco offered customers at least 200GB of extra data.

Optus users were also warned of phishing scam messages claiming to offer compensation for the outage.

The telco said it would not send any communications with links about the outage, to provide "peace of mind" for customers.