Bitwise Introduces ‘ETL on Hadoop’ Utility at Strata + Hadoop World

NEW YORK, NY, September 27, 2016

Bitwise, a data management consulting and services company, today announced the introduction of a next-generation data integration tool called Hydrograph to address a need for ETL-like functionality on Hadoop in enterprises with big data workloads.

The announcement was made at Strata + Hadoop World, a deep-immersion event where data scientists, innovators, and executives get up to speed on emerging techniques and technologies. Built by practitioners that understand the pains of offloading ETL (Extract, Transform and Load) on Hadoop, Hydrograph is engineered to accelerate ETL development in the big data ecosystem.

ETL for the Hadoop Data Lake

“With its massively parallel processing, Hadoop once held promise for transforming the ETL process on large datasets – but moving ETLs to Hadoop still requires heavy MapReduce coding, which defeats the purpose,” notes Shahab Kamal, Executive Vice President – Client Management & Solution Engineering at Bitwise. “In order to take advantage of the processing power of Hadoop, Hydrograph uses Cascading to generate MapReduce jobs that are converted from XMLs developed using a desktop based GUI by connecting the components with links.”

Despite innovation in traditional ETL tools, gaps remain for organizations that are looking to cut costs and increase computation power by re-using ETL processes being used by their existing ETL applications on Hadoop, or to build new ETL processes on data in their Hadoop data lake.

“As the center of gravity of data moves toward Hadoop, tradition ETL vendors seek to adapt their tools to the new environment…but gaps remain” points out Nik Rouda, Senior Analyst – Big Data at Enterprise Strategy Group (ESG) after receiving a preview of the tool. “Traditional ETL vendors are expanding tool capabilities to Hadoop, but often carry over legacy issues to the new environment; Hydrograph is pure ETL on Hadoop built for today’s big data needs.”

Development of a Modern Data Integration Tool

Bitwise partnered with business users to design a tool that would help them pivot from a traditional ‘closed’ system for ETL development to a more modern open source environment. With the Bitwise team’s innovative developments, such as using Cascading as the foundation and building abstraction layers to simulate an ETL job, the effort proved that an ETL solution on Hadoop was possible.

“When we began development on Hydrograph over two years ago,” says Dhwanit Malani, Executive Vice President – Innovations at Bitwise, “Cascading provided the best compute option. With its swappable architecture, Hydrograph can work with the latest technologies, such as Spark and Flink, as they reach maturity.” With the addition of a User Interface that has a similar look and feel to leading data integration tools, Bitwise has developed an effective and powerful ETL tool for the big data ecosystem.

In order to foster innovation and allow other enterprises to benefit from Hydrograph, plans are underway to open source the project in the near future. Bitwise can help organizations quickly conduct a Hydrograph pilot and offers various support models to ensure optimal performance of the tool.

Bitwise is a Premier Exhibitor at Strata + Hadoop World this week in New York. Strata attendees are invited to visit Booth 215 where Bitwise will be demonstrating the Hydrograph tool, sharing customer case studies and answering questions.

For Hydrograph reference architecture and demo video, visit https://www.bitwiseglobal.com/en-us/data-tools/hydrograph/.

About Bitwise

Bitwise is a technology solutions provider that leverages data to enable business insights. Our breakthrough technology innovations help our global clients maximize their competitive advantage. We are the industry's most experienced and dedicated team of data professionals, focusing on Data Warehousing, Data Integration, Analytics, Business Intelligence, Big Data and Digital Computation. We optimize value for our clients through our global delivery model and with our proprietary technology tools that reduce the time, complexity and cost of data initiatives. Together, our people and technology provide the insights clients need to continue to lead in their fields.

Founded in 1996, Bitwise is headquartered in Chicago, IL with development centers in Pune, India and Sydney, Australia. For more information, visit Bitwise at www.bitwiseglobal.com.