Notes from Industry

What should managers look for when growing their teams? And what tools can provide relief for their already overburdened staff?

Photo by Brooke Cagle on Unsplash

For any person who works with data, you’ve likely heard the phrase “data is the new oil.” However in today’s digital economy, if data truly is the new oil, we’re going to need a lot more people working the rigs and refineries to keep up with the demand.

Today, data-driven innovation has become a strategic imperative for just about every company, in every industry. But as organizations expand their investment in analytics, AI/ML, business intelligence, and more, data teams are struggling to keep pace with the expectations of the business.

For example, many data engineers are often overburdened by the…

The DataOps methodology offers a new way to improve both the quality and speed of data analytics.

DataOps graphic, with permission from Ascend’s

The proliferation of data and data systems — spurred by an increasing number of use cases for advanced data analytics — has catapulted DataOps into the mainstream for modern organizations. The DataOps methodology has been growing in popularity among data teams, offering a new way to improve both the quality and speed of data analytics.

Traditionally, data pipelines relied on very little automation and required intensive coding. …

Photo by Kevin Ku on Unsplash

Earlier this year, in the LinkedIn 2020 Emerging Jobs Report, the role of the data engineer was listed as number eight in a ranking of emerging jobs in the U.S. — with a 33% annual hiring growth rate. More recently, the Robert Half Technology’s Salary Guide ranked data engineering as the highest-paying IT job in 2021. According to Robert Half, data engineering talent will continue to be compensated well above average as organizations will heavily rely on “individuals who can transform large amounts of raw data into actionable information for strategy-setting, decision making and innovation.”

As the demand for data…

Sheel Choksi

Solutions Architect at

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store