My Dabbling in Agile BI

Many years ago I was introduced to the term “agile”. In those early days of social networking, discussion lists ruled. I was a subscriber of a large Chicago-based data list server. Whenever the word agile was uttered, a barrage of negative responses ensured for days, often weeks. Data professionals were certain that agile development was meant to shortcut the design process, cut the data modeler from system development, and deliver stovepipe applications with no documentation.

Fast forward to today; ten years later. Agile development is still here. It has made great strides from that fringe cowboy development to a mainstream development methodology. Controversy still surrounds its use. The results it delivers cannot be diminished. In a world that is very cash, resource and time constrained, agile development presents a good option to develop applications faster and take a bite out of the never ending application development backlog.

Since the beginning of this year, I have been involved in a few agile BI initiatives and have also attended an agile BI class. I was a skeptic up to this point but have always reserved my opinion until agile actually impacted my work and role. I have to say that I believe that agile development has a place in BI and data warehousing initiatives. I still am not certain that it is a fit for all BI initiatives. I am devoting the remainder of this blog post to my observations over the past three months.

Something old …

There are concepts in agile that are reminiscent of traditional system development. The principle that work efforts are segmented and prioritized into bite-size chunks that can be completed in a shorter period of time is a good practice for all development efforts. Bringing together IT roles from many disciplines in a room with business users has always been a winner. Having a dedicated workspace that provides opportunities for frequent interaction and collaboration is certainly a more nimble and faster application development environment.

Something new…

Agile is a much more intense method. It calls on team members to focus more on the solution at hand in a way that demonstrable results can be delivered in a short time frame, often 3-4 weeks. The waterfall development methodology must give way to a more give-and-take method that is focused on solving the problem at hand as a team. The most dramatic change is that the agile team is in control of the project, deliverables and timeline. That is certainly a new role for many IT professionals.

From a data modeler’s perspective…

Traditional agile team structures refer to team members as developers. Agile tends to see all developers as interchangeable. A data modeler is not the same as a Java developer or IBM InfoStage ETL designer. That is the reality. My training and our use of agile to date views the data modeler as a visiting scholar. Visiting scholars are part of the team and may leave the team at times when their expertise is not required. They are never far away but close enough to be consulted and perform tasks on an as needed basis.

The Bottom Line…

Agile is here to stay. It is most likely going to become more prevalent in development projects. It is also most likely going to evolve just as all development methodologies have evolved to keep in sync with technology and the changing world.

The most pressing problem I see with agile development is in the area of staffing. Assigning key personnel to an agile team can adversely affect other projects and initiatives that lose that resource. IT leadership needs to staff and prioritize resource workloads to satisfy both agile and traditional development streams. I am curious if agile development can lead to developer burn-out. Can developers maintain a fast-paced development environment over a long period of time?

How has agile development impacted your job as a data professional? I would love to hear the good, the bad and the ugly about agile in the data world. Agile BI appears to be a twist on traditional agile development. Has your shop ventured in integrating agile development into data warehousing initiatives? Tell me more.

Tom Bilcze
Modeling Global User Community President

 I’ve reached 3NF in “Why Be Normal?” How normalized are you?


 Find out more about “Why Be Normal?” at Want to know what “We Be Normal!” is all about? Visit get the whole story.

Follow ERwin online through Twitter, Facebook, and LinkedIn.
Twitter: @ERwinModeling and #YBNormal  

Print Friendly, PDF & Email

Leave a Reply

Your email address will not be published. Required fields are marked *