Nazir Vellani, Fidelity Investments VP of enterprise resiliency, speaks to StrategicRISK about how the organisation built a proactive program that seamlessly detects, addresses, and rectifies risk

The challenge

Fidelity Investments wanted to create a new type of enterprise business resilience program to help it manage increasing disruption and meet looming regulatory deadlines.

cloud

To achieve this, the company knew it needed to evolve from a legacy disaster recovery mindset, and instead build a proactive program that seamlessly detects, addresses, and rectifies risk 

The investment management company also wanted to foster greater collaboration across the business and technology teams, and position itself to more easily absorb small- and medium-sized “shocks”.

The company had historically used manual processes to assess risk. This proved to be time- and labour-intensive, and ultimately produced inaccurate data, and made it difficult to work within cost measures .

Nazir Vellani, Fidelity Investments VP of enterprise resiliency explains: “We relied on manual processes to assess risk which required substantial time investment and high costs, and these processes could not efficiently scale.

“We soon realised that our manual processes could no longer effectively assess risk through our dramatic increase in data and critical applications.”

“We ran tests on weekends, but this ultimately led to inaccurate results since systems weren’t exposed to the same stress volume as a regular workday. We soon realised that our manual processes could no longer effectively assess risk through our dramatic increase in data and critical applications.”

The organisation had also planned to move some of its critical systems to the cloud, something Vellani says led additional challenges.

For instance, it created meant a four-times increase in the number of components requiring end-to-end testing in line with internal practices to measure and manage resiliency. 

He says:  ”We also saw an increase in data, as the company took on larger initiatives to capture customer behaviors for effective targeting. We also saw a 32% increase in critical applications.”

The solution

The dramatic growth in critical applications and data volumes meant it was impossible for Fidelity to keep pace, and so the investment manager looked for external help. 

Specifically, it wanted a collaborative software partner that could provide a flexible and scalable solution to effectively capture data, automate processes, and conduct testing.

After a competitive evaluation, the company selected the Fusion Framework to help it create a new enterprise business resilience program.

Historically, Fidelity had minimal proactive measures to test systems before they went live.

“We realised that we needed to design our program with a resiliency mindset at the core.”

So, the company worked with Fusion to create a resiliency index that allowed for proactive risk assessment before deploying new cloud applications.

Collectively, they established set parameters to test and promote resilience confidence to the CIO stack.

To ensure accurate results, these tests are carried out in conditions reminiscent of real-world expectations. Scenario tests are based on service composite rather than individual applications and focus on actual production traffic and shocks.

Now, before any new products deploy, the company requires mandatory testing to proactively mitigate risk.

“We were no longer dealing with a traditional disaster recovery approach where we could simply un-plug and re-plug our hardware.”

Vellani says: “We realised that we needed to design our program with a resiliency mindset at the core. The threat landscape had changed dramatically, and we were no longer dealing with a traditional disaster recovery approach where we could simply un-plug and re-plug our hardware.

“We needed to implement a system that allowed us to demonstrate resilience following the move to cloud systems. 

“We ran a competitive evaluation in the market and ultimately selected Fusion Risk Management to help us create our new enterprise business resilience program.”

The impact

By working with Fusion’s advanced automation and guided workflows, the company minimised the need for extensive man hours.

The new solution meant Fidelity could capture large amounts of data from the cloud in real time, which also reduced the need for manual data entry and the chance for human error. 

The flexibility offered by Fusion enabled the company to decide when to close out its testing cycle and fostered a heightened focus on outliers. 

“We can now deliver more value to our customers by seamlessly absorbing shocks and automating tasks, enabling us to build trust”

The resilience testing and automotion flags systems and applications that do not meet resilience requirements. This allows Fidelity to detect alerts and mitigate risk factors before they impact on the the business.

Vellani says: “Fusion’s automated systems allow us to proactively assess risks which allows us to minimise their potential impact and demonstrate to key stakeholders, including the CISO, that systems are ready for deployment.

“We can now deliver more value to our customers by seamlessly absorbing shocks and automating tasks, enabling us to build trust and a culture of resilience that ensures that our customers never experience a substantial delay in product or service delivery.”

Put to the test

The new approach was put to test during a recent interruption.

Fidelity learned about the outage with their pricing platform at AWS just 10 minutes after it occurred. With the proper tools for resilience in place, the company continued business as usual.

Typically, this sort of disruption would result in a halt in services. However, instead services were automatically rerouted to an alternative hosting location which absorbed the entire workload of the downed system.

Once the regular environment was thoroughly checked, services seamlessly flipped back to their regular host location.

What’s next on the risk agenda?

After technology disaster recovery, Fidelity set on a mission to revamp its third-party risk management program.

Vellani concludes: “We understand that third parties are critical to continuity and resilience but realise that horizontal dependencies lead to unrealised risks.

“We have created a program that allows us to horizontally map third-party dependencies and visualise unseen concentration risk, providing us with a better understanding of what shocks our organisation can absorb.”