Pepperdata: Carving Out a Niche in the Big Data Arena

In the data analytics and Hadoop arena, the folks at Pepperdata have an interesting story to tell. Pepperdata's cofounders ran the web search engineering team at Yahoo during the development of the first production use of Hadoop and created Pepperdata with the mission of providing a simple way of prioritizing Hadoop jobs to give resources to the ones that need them most, while ensuring that a company adheres to its SLAs.

The company's software installs in under 30 minutes on an existing Hadoop cluster without any modifications to the scheduler, workflow, or jobs, delivering visibility into Hadoop workloads at the task level.  This week, Pepperdata announced that former CTO of Yahoo, Ashfaq Munshi, is taking over as CEO. Here are more details on this company from an interview we did recently with co-founder Chad Carson (shown in the photo).

Co-founder Chad Carson previously held management positions at Microsoft, and in our interview with him, he detailed how far back the Pepperdata team goes with Hadoop:

"The web search team my Pepperdata co-founder, Sean Suchter, led at Yahoo was the first production use of Hadoop anywhere in the world. Hadoop had a huge impact on our ability to make improvements to the product very quickly and cost-effectively. I saw similar gains from using Hadoop in my sponsored search data science team — where we were able to run hundreds of live experiments each year, an increase of about 10X, leading to huge revenue gains. When we founded Pepperdata, our goal was to help enterprises get those same kinds of benefits from using Hadoop, by giving it the level of performance and reliability that enterprises need. "

 Pepperdata’s software installs quickly and monitors and controls the use of hardware resources in real time, so that Hadoop operators can ensure that high-priority production jobs get the resources they need ​so their most critical jobs complete on time. ​Carson also noted this:

"We monitor and control hardware resource usage by every job and process, in real time, once they’ve started running. Enterprises who rely on Hadoop in production should definitely be using both cluster management tools and Pepperdata’s real-time cluster supervisor software — they work great together."

 As for Munshi, he hails from the research arm of Yahoo, which is largely shuttered now, but Munshi has extensive Hadoop experience. We'll keep up with Pepperdata's moves in the Big Data space in the months to come.