Tool King Puts A Hammer Down On Price-Scraping Bots With Distil Networks

Price remains a top factor for consumers, and they’re scouring the web for the best deals and offers. Retailers, in turn, are waging an all-out price war with competitors, using new tools and solutions to keep a constant pulse on prices across channels.

Multichannel tool retailer Tool King recognized consumers’ heightened desire for lower prices as more store visitors asked the retailer to match Amazon prices. 

“While tools didn’t suffer the level of showrooming a category such as electronics has, it definitely happened,” said Brandon Allhands, Director of IT at “The effect on our business has led to us shutting down our retail storefront. While it wasn’t directly showrooming and the Amazon effect, they were definitely factors.”


Focusing on e-Commerce, Tool King has seen ongoing growth and has expanded its catalog from 8,500 tools in early 2010 to more than 400,000 general products.

However, the retailer has seen a whole new set of challenges. Competitors now are using bots to scrape the Tool King e-Commerce site to collect vital product and pricing data. Traditionally, the admin team would catch bots by manually tracking logs and patterns. Now, Tool King uses Distil Networks, which detects and blocks malicious bots, malware and competitors that try to scrape web sites.

Tool King started seeing results immediately, including decreased costs and improved conversion rates, Allhands reported. “We were able to cut our monthly origin hosting bill by nearly as much as Distil’s service costs. It almost paid for itself just in hosting savings.”

Tool King also saw conversion rates increase dramatically since “bogus traffic” has been thrown out. Over the past three months, Distil has blocked more than 42 million requests from bots, Allhands added. “That’s 42 million less log entries, which has allowed us to gain more accurate insight into our traffic patterns and customer experience.”

Distil Networks runs the traffic through their network, “which means setup is pretty much only some domain name system changes and telling Distil about our origin servers,” Allhands said. “I was able to point traffic within 24 hours of signing up. They also provide configuration for how you want to respond to a type of threat. All of the configuration is done through a web interface, so it’s easy for non-technical people to implement.”

Prior to implementing Distil, the admin team would notice spikes in traffic and would take action against IP addresses, Allhands explained. “We would look at logs and see a collection of IP addresses performing nonsensical searches, like searching for different UPCs every couple seconds.”

Now, Tool King is equipped to compete more successfully in the pricing battle, which is ongoing and constantly evolving.

“As someone who has written scrapers for competitive pricing, it was almost like a war,” Allhands said. “We would take action to block, and the ‘enemy’ would adapt. We would then adapt, only to have them adapt again.”

The cost of not adapting and deflecting price-scraping bots is two-fold, Allhands noted.

“One: they are getting data/pricing. Why collect pricing unless you plan on taking action from it? That action is typically the ‘beat them by a penny’ action. Two: the traffic itself. Bandwidth is not free, and as our catalog grew, so did the non-converting traffic.”

Allhands explained: “Paying for a competitor to grab your information is like a slap in the face — we had to find a way to adapt and block the traffic at the source, as opposed to the reactionary and labor intensive process we were following.”


Featured Event

Join the retail community as we come together for three days of strategic sessions, meaningful off-site networking events and interactive learning experiences.



Access The Media Kit


Access Our Editorial Calendar

If you are downloading this on behalf of a client, please provide the company name and website information below: