Maybe it’s because I am getting older, but I pay more attention when I hear the words, “You can’t teach an old dog new tricks”. IT professionals need to be learners today if they want to survive in the current and future job market. Here are my top five lessons learned from 2014.
1. Big data needs data architects.
Data needs to be managed. A trend I have observed this past year is the desire to load large volumes of legacy and app data into big data databases such as Hadoop. Technology has made that process simpler and the retrieval faster.
As the volume of this data become larger and more diverse, organizations need to understand what data they have. It sounds like a job for a data architect. I cannot offer hands-on advice on this subject since I am not a big data practitioner. I am confident that the big data architect role will be critical in the coming years. It may not fit the traditional data modeler mold. Perhaps the process will work back from the question at hand to uncover the correct data to answer that question.
2. Lingering data problems are the same yet different.
Information technology has become increasingly complex over the years. It has made access to data and technology friendlier and easier. IT is good at solving problems related to hardware and software. We are less agile in solving problems related to soft skills.
I observed poor communication, undefined roles, and working in silos when I entered the IT world forty years ago. These problems still linger in most IT staffs. Methodologies have morphed to be more responsive and are tied closer to today’s technologies which are much more advanced. Yet IT has been unable to effectively tackle the interpersonal communication and soft skill issues with the same dexterity.
3. Bad data design causing even bigger problems today.
There is more data than big data. It may be today’s hot buzzword but the reality is that our enterprises rely on operational databases to carry the everyday workload. Data that manages manufacturing, policy underwriting, customer resource management and many other applications are crucial in keeping the enterprise running and must be properly designed.
This not-so-big data requires the same attention to detailed design as it has in years past. I would argue it requires more attention. Today’s data is web accessed by many more people; many external to our organization. It is increasingly common to give more business users a view of these databases. Sound data design principles assure data structures are secure, can handle heavy access loads, are integrated across the enterprise and meet our corporate standards.
4. Educating people about data is important.
I have mentored new data team employees over the years. My career teaching portfolio contains many variations of Data Modeling 101. For years the prime audience was IT team members.
I find myself educating a wider audience about a larger portfolio of data subjects. Business intelligence, metadata management, data governance, and industry model usage are a few that join data modeling in my course catalog. These classes are mostly informal, often 1-on-1. It is the data architect’s job to share the knowledge of data and steer our users in the right direction.
5. Data architects are scarce
I won’t sugar coat the topic. The data architect job is under appreciated and is not glamorous. Colleges ignore or under value the knowledge of database design. Young, talented IT professionals steer towards the more glamorous web app development. This results in a shrinking data talent pool.
The data architect population is aging. Organizations rely on the knowledge accumulated over the years by these seasoned data architects. This practice will result in a shortage of data professionals in the coming years as the baby boomers bow out of the workplace. It could be a case of “You don’t know how well you had it until it is gone”. Enterprises need to pay attention to the staffing of the IT data group.
Happy New Year!