Highlights:

  • Ibis provides a standardized framework to access data and inquire about multiple backends using Python.
  • Apache Arrow is an open-source project that issued a merged format for the in-memory portrayal of Vector Data, which eliminates the requirement for data to be serialized and deserialized between multiple formats.

Voltron Data, an enterprise support service provider for Apache Arrow, recently announced that it is adding an enterprise support offering for Ibis – an open-source Python framework for standardizing analytics and queries across multiple backends.

This development is the next step to building Voltron Data’s portfolio of enterprise support offerings and in line with the company’s goal to invest in open-source technologies continuously.

The announcement was made at The Data Thread, a virtual learning event hosted by Voltron Data to promote Apache Arrow’s practical applications and bring practitioners, industry leaders, and members of the Arrow community.

Establishing standards

A famous media house spoke with Voltron Data CEO Josh Patterson about the importance of creating benchmarks and communication across systems to remove pain points and reduce replication of effort.

Apache Arrow and Ibis were designed to offer standardized solutions to common pain points. Apache Arrow is an open-source project developed to facilitate a merged format for an in-memory portrayal of Vector Data, thus eliminating the requirement for data to be serialized and deserialized between multiple formats. Doing away with the duplication of work generates efficiencies. As Patterson said, it “just allows developers to focus on things they want to focus on.”

Ibis is a Python data frame Application Programming Interface (API) that provides a standardized framework to access data and inquire multiple backends using Python. The backends supported are a combination of SQL and big data analytics databases. This includes Google, BigQuery, Heav.ai (formerly OmniSci), PostgreSQL, MySQL, and others. Ibis permits developers to write one kind of code to target multiple systems and switch between backends if need be without changing the same framework. According to Patterson, “it gives users this freedom to not have to keep rewriting their code if the way that they do compute changes.”

A natural extension

Voltron Data was launched earlier this year with an enterprise support offering for Apache Arrow. Wes McKinney, Voltron Data co-founder and CTO, is also the co-creator of Apache Arrow, Ibis, and the Python pandas project. McKinney is also the committer and member of the project management committee for Apache Parquet. The firm says that the decision to introduce enterprise support for Ibis came partly before high customer demand. “The same Arrow customers are asking… well, what about Ibis? They see that the same people are maintaining these projects,” and as a result, “we just felt that it was a natural extension,” Patterson said.

Lift velocity?

The planning and perks generated by Apache Arrow and Ibis stem from their ability to offer a unified solution to a user’s pain points. The wider a standard is adopted, the more value it tends to bring, with Pandas and its data frame paradigm being a case in point.

It’s unclear if Ibis can achieve comparable adoption and value, but Voltron’s move makes sense and should assist in standardizing data-oriented software development rather than fragmenting it.