Data management is hard. Don’t let any vendor tell you they have tools that will set you free to life on a beach while your data program magically runs itself. Tools can automate pieces of your data program. Tools cannot tackle many of the challenges presented by the human side of our work as data professionals. Let’s take a look at five of the most prevalent hard challenges I have faced over the years.
- Looking beyond the moment
A good data architect looks beyond the current challenge they are presented. They always look at fitting the puzzle piece at hand into the enterprise puzzle of data. Unfortunately this passion and thought process is often not embraced by other IT team members.Typically this is manifested in a tight timeline where the database development is on the critical path. The “spend $1 now and save $100 later” is very applicable to all of the application development lifecycle. I marvel at the number of leaders in IT management that do not understand or value the enterprise view, particularly the enterprise view of data.
- Staying alive
Many data professionals are questioning their role, the changes of their role and often the absence of their role in today’s application development. Change is faster today than I have seen in my 40 years in IT. Data management is not only changing at a faster pace. It is being asked to manage this ever expanding data asset with smaller resources and limited experience in new data technologies.This scenario presents opportunities and challenges to data management professionals. We must remain current in technologies, methods, processes and standards. Data management needs to assert their roles and importance in assuring the integrity of the enterprise data assets. Most likely, no one else has this on their radar. Data management must also recognize the diversity of data, embracing new types of data and finding ways to manage that data.
- Is everyone happy?
Gone are the days when the data management team had a single set of users to keep happy, application developers. Data now has a very interesting challenge. The audience for the data team’s deliverables and processes is very diverse cuts across the enterprise. The audience is often unknown.Many of our new audience members are unfamiliar with the data they must use. There is an expectation that the data management team will deliver interfaces that not only assist them in working with their data but also help them learn and understand what the data is and what it can offer them. Get out your data management super powers!
- The right stuff
Continuing the theme of an explosion of data, there is just so much data hovering around us. Our data management strategy must address what data we are and will manage now and in the future. Managing data is not free. It may be cheaper than it has been in the past. We still must assess the costs and benefits.New challenges have appeared to complicate our roles. How secure is our data from ransom and hacker attacks? How long do we need to keep it and where do we store it? How do we work with cloud data and maintain data integrity and responsiveness to the enterprise? This is truly one of the hardest parts of data management today.
- People, people who need people
Staffing data management has never been easy. It is definitely harder today. The demand for data professionals is high, particularly when you add analytics and scientist to the job. Many data management professionals grew up in the relational and data warehouse boom years. Do a little calculation, and you will understand how their age and retirement plans are impacting the data profession.Beyond maturing data professionals lies a new world of opportunities for new people and new roles in the data management space. I spoke earlier about the evolution of technologies and processes. With this evolution come opportunities for new roles and responsibilities. The challenge is finding the right person for the right job while the data management team is struggling to understand this data, technologies and the roles it requires. There will be no shortage of opportunities for the data person in today’s technology workforce.
A data guy from Ohio