OpenSea Site Reliability: January 2022

Originally posted here.
By: Devin Finzer @

Overview

I wanted to take a moment to address the site stability challenges that OpenSea has experienced over the last few months, including this morning’s downtime. I recognize that the impact of OpenSea downtime is significant for many of you who depend on our platform. We take accountability for the recent instabilities – and I wanted […]

The post OpenSea Site Reliability: January 2022 appeared first on OpenSea Blog .

The Post

I wanted to take a moment to address the site stability challenges that OpenSea has experienced over the last few months, including this morning’s downtime. I recognize that the impact of OpenSea downtime is significant for many of you who depend on our platform. We take accountability for the recent instabilities – and I wanted to personally apologize, explain, and outline our plans to prevent this from affecting you in the future. 

This morning, we experienced a sustained surge in API traffic that overloaded our systems and resulted in degraded performance and site outages. To address these issues throughout the day, we made changes to optimize our API access and scaled our search datastore to better manage the increased load.

Improving site reliability has been a priority for some time (in fact, it’s one of the focus areas I mentioned in our recent funding announcement ). We were a team of just seven people at the start of 2021, and as NFTs took off last year, we had to scale fast. That kind of scale comes with growing pains, which many of you have experienced firsthand. Thank you for bearing with us. You deserve better, and we are up to the challenge. 

To that end, I want to share a few of our internal goals with you to be more transparent about what we’re doing to improve site reliability and support : 

Expanding our engineering team to more than 200 people by the end of this year . In the last year, we’ve brought on incredible engineers – but we need significantly more to both improve our core architecture and ship the products and features needed to improve the consumer experience. To that end, we are hiring and would love your help with referrals. Rearchitecting for scale . Our platform team is actively executing on a plan to rearchitect core parts of our architecture to match current and future demand on our systems. I am personally overseeing this priority alongside our technical leadership team. Reducing our customer support times significantly. To make sure we’re responsive to community needs, especially around platform reliability, we’re building out a large team that is solely focused on reducing response times for your support questions. That includes an investment in hiring more than 60 individuals as of last week, and plans to double that team by the end of 2022 to meet increasing demand.

We’ve got a lot to build this year, but these are by far our most critical improvement areas. Consider this my commitment to you that we’ll improve quickly, and that site stability will remain the absolute top priority until we are performing at our best. We need to do better and we will.

Devin

The post OpenSea Site Reliability: January 2022 appeared first on OpenSea Blog .