Etsy has fully migrated the infrastructure behind its e-Commerce operations from two data centers to Google Cloud, leveraging the server’s computing power to improve its machine learning and product development capabilities.
During the two-year migration, the marketplace has increased its speed for running experiments with large datasets and sophisticated algorithms. These experiments can determine how to best sort product reviews or how to alert potential customers when an item might be snapped up by someone else. In 2019, Etsy ran several hundred experiments every quarter, 115% more than 2018.
Additionally, Etsy shifted 15% of its engineering headcount from managing system infrastructure to focusing on the customer experience, and set a goal to operate by 100% renewable electricity by the end of 2020.
With data now more accessible through Google Cloud, Etsy aims to deliver a more curated shopper experience by serving the most relevant items for shoppers at the top of its search results.
“In the cloud, we’re better leveraging Etsy’s vast trove of data to uplevel our machine learning capabilities,” said Mike Fisher, CTO of Etsy in a blog post. “Our marketplace includes more than 65 million seller-generated listings that don’t map back to a single catalog, making search a unique challenge for our engineers. We’re building more sophisticated algorithms to help buyers find exactly what they’re searching for, making our sea of listings more manageable and curated. We’re also using this detailed data to deliver enriched recommendations that better reflect our buyers’ unique styles and tastes.”
As part of the migration, Etsy moved 5.5 petabytes of data — the equivalent of approximately 40 billion photos ― from 2,000 on-premise servers to Google Cloud. Instead of running servers 24/7 in data centers, Google Cloud’s shared infrastructure is designed to allow Etsy to scale capacity as needed, based on the natural cycles of its e-Commerce business.
Since the experiments eat up a lot of computing power, moving to a cloud server was an imperative for Etsy. It wasn’t cost effective to buy extra computing power that would likely remain idle in hopes of running future tests. Before the shift to the cloud, creating and running data science experiments could take several days or weeks and now they can be done in a few hours, according to Fisher.