Technology setbacks seem like the least of BC Ferries’ issues when considering the problems they have faced over the course of the summer.
As the ferry service grappled with staffing shortages, high passenger demand and technical challenges with its vessels, technology issues caused additional problems for the organization, which moved more people in 2022 than did Vancouver International Airport – the second-busiest airport in the country.
Technical problems manifested in the form of IT issues during the May long weekend, website crashes in June and issues with BC Ferries’ booking service in July.
“It’s fair to say we’ve had a number of frustrating events happen over the summer,” said Nicolas Jimenez, CEO of BC Ferries, during an Aug. 2 news conference.
“It includes cancellations due to crewing and network instability. Our customers are right to expect a lot from this company, we provide a vital service connecting people and communities up and down the coast.”
BC Ferries spent $20.5 million in 2023 and $17.3 million in 2022 on information technology, according to organization’s 2022-23 annual report.
Despite these investments, their computer system crashed in the early morning hours of May 22, leading to nine hours of outages across BC Ferries’ website, app and phone system, according to records obtained by Glacier Media under freedom of information law.
James Tan, vice-president and CIO of BC Ferries, said the root cause of the outage was attributable to a server in the province’s Interior reaching capacity.
“The main storage at the server level in our Kamloops data centre reached a threshold unexpectedly and basically ran out of space, and because of that, it caused multiple systems then to come down and fail,” said Tan, who joined BC Ferries in July 2023.
“As we were investigating the root causes of what should have been done proactively, [we] made sure that we’ve got monitoring thresholds so that we’re actually catching these unexpected spikes in usage of the systems much earlier than what we had on the May long weekend.”
Glacier Media reached out to BC Ferries for comment on the reasons behind other technology issues experienced throughout the summer.
In an editorial board meeting with BIV earlier this month, Jimenez said a lot has since been done to change and improve the back-end of BC Ferries’ tech systems.
“We put in more storage and server capacity in our data centre. We introduced kind of a virtual waiting room, so when there’s a surge of traffic into the website it doesn’t crash,” he said.
Ivan Beschastnikh, a computer science researcher at the University of British Columbia, said that BC Ferries’ system may need modernization.
“My guess is that the system is probably like a legacy system, but they haven’t had the chance to really update all that much. Which is how these systems develop. You basically build it in the 90s and you end up piping internet access to it, then everyone starts using smartphones and then all of a sudden, you have way more traffic than you’d expect or that you planned for,” he said.
He proposes mixing physical servers already in use with cloud services, which have more flexibility to deal with high demand.
“You can meet the demands, pay a little bit more for that one day and then ramp down and pay a lot less the next day without needing to buy any physical servers.… That’s the way I would build the site today and it seems pretty clear that that’s not how it’s done.”
“It becomes a business case of is there enough investment into upgrading these systems to make sure that they’re up to date. Security is another whole other dimension to all this. Usually, that gets a lot of attention. I think capacity gets less attention because it’s a little bit harder to predict until you reach this kind of critical event.”
Serge Buy, CEO of the Canadian Ferry Operators Association, said that he challenges anyone to find a booking system as complex as the one used by BC Ferries that hasn’t experienced issues.
“I would say that there are very few ferry operators in Canada which have such a complex reservation system,” he said. “They’re trying to do their best to improve customer experience and to increase the visibility of the whole thing and the transparency of the process. They should be applauded for that.”
“BC Ferries is going through challenges as they’re putting in place a system that’s second to none.”
To solve the issues they have experienced over the summer, BC Ferries has implemented a waiting room to ensure that high demand does not lead to the outages seen during Victoria Day, according to Jimenez.
Other solutions involve increasing server capacity and storage at their data centres, and establishing reservation allocations in the booking system to help manage any expected high volumes of unreserved travel.
A spokesperson for the British Columbia Ferry Commission, an independent regulator of the ferry service, said that the commissioner Eva Hage does have any plans to address issues with the BC Ferries website.
“She’s been following up with BC Ferries asking them ‘What are you planning to do to make sure that this doesn’t happen in the future?,’ that kind of thing and she certainly will undertake a performance review of it if she feels it’s a good idea,” said Susan Kerschbaumer, speaking on behalf of the commissioner.
