The evolution of data over my career fascinates me. The volume of data continues to increase exponentially. Data is becoming more accessible to more diverse audiences. Data has moved to the forefront from being buried in the shadows of application development.
Given these facts, it seems that data architects should be elated with this positive trend. Not so much. These trends have left many a data architect uncertain of their role and the people they serve. Not so many years ago that was an easy question for data architects to answer. Our work was primarily in application development efforts having us teamed up with systems analysts, business systems and DBAS.
I challenge you that our role has not really changed. Data architects help people understand data and enable them to turn that data into information. Understanding data is more important today than it has ever been. Databases that contain terabytes of data are becoming increasingly more common. More challenging are the networks of these terabyte plus databases that may be inside our firewalls or in the cloud. More data equates to the need for more understanding.
In almost all of my data modeling jobs, the cry of the data architect has been to involve us early and more frequently. This is where discomfort arises today. Many new big data technologies and development methods do not support this mentality. This gap is in no way a show stopper for data architects. It is merely a challenge to focus our energies on areas where we can help people understand the data they have and how to best utilize it to solve their business problem.
You may be called upon as a data architect to transform physical data structures into modeled objects that help data scientists, data analysts and other business people understand how to best use the data to answer their business question. You may be called upon to uncover and deploy metadata that clarifies the nature of the data. I often think of our role as being the librarian who brings order to the vast library of data by providing our customers with maps and directions to the right data. Metadata is perhaps becoming more important than the data model today.
Our customer base is changing and evolving. It is more of an expansion rather than a fundamental shift of parties. Working in data management is a profession where our work is built on relationships we form and how well we manage and nurture those relationships. We are in a job where we must gather information from many diverse roles and be able to disseminate that information to diverse roles; many that are outside of our normal circle of peers and consumers.
There is really only one way to ease the discomfort of an evolving customer base. Data architects must understand who their customers are, their needs, and what they can do to help them succeed. That is a pretty basic rule from any customer service role. Sounds easy, huh? Not so much.
The challenge in aligning with new customers and establishing rapport, trust and partnerships is knowing who these people are. Data architects have worked with a pretty defined set of customers in traditional operational and analytical environments. Many of our new customers are less predictable. That is, they may be casual users far removed from our span of control. They may be in roles that are very complex. Roles in which data is just one of the complexities they must deal with.
Flexibility is a key characteristic that a new world data architect must possess. Expect the new role of data to continue to expand and evolve in the coming years. Technologies like No SQL and big data are in their relative infancy. Look for changes to come as they mature. There is no doubt that their evolution and growth will involve more robust data management. The savvy data architect needs to be flexible and able to adapt and grow their role to meet these new data management needs.