I’m at the Data Warehouse Institute’s San Diego conference this week, and experimenting with an incremental approach to blogging for this event; I’ll try to get on a few times in the next 2 days (unfortunately that’s all the time I’ll have here) and communicate some quick thoughts, as opposed to my more typical style, which is longer and more in depth. That will no doubt follow on many of the topics later.
I begin with the keynote from Wayne Eckerson this morning, where he offered his thoughts on Agile BI. Agile is a loaded word; for developers it means a very specific set of techniques and methodologies. Data folk are not part of that culture in most cases, and they use the word as an adjective. Wayne attempted to bridge the gap in a few places, but by and large, his hints at best practices were not particularly new, or surprising, or tied closely to the Agile playbook. Wayne asked the hundreds in the room how many of them considered themselves agile. (I render it in lower case because he did not specify “as the developers define it.”) Literally fewer than 5 hands went up. His speech focused on some paradoxical ideas; it began with a focus on governance. Counterintuitive, even Orwellian: “structure is freedom;” agility happens if you prepare for it. Not a bad point by any means. Overall, Wayne’s 5 tips were:
- align with business
- slow down to speed up
- anticipate the business
- manage expectations
- cede control
In general, the discussion was well structured, illustrated with quotes from real users from TDWI’s research, and paced very effectively. Yet I found it fundamentally unsatisfying, leaving out as it did the “real” Agile story I was hoping for.
WhereScape RED Gets it Right
By contrast, WhereScape, a small but intriguing player in the data warehouse design space, introduced a new release that was much more on target. WhereScape RED 6.5 adds features to support Agile practices like continuous integration testing and integrates with the kind of source control and testing systems Agile developers use. The tools make it easier to back out changes, enabling the kind of iterative process common in development shops but not in data architect thinking. In its press release, WhereScape calls out RED including:
- Automated generation of standard code: error handling, status updates, documentation, parameters, naming conventions, indexing, etc.
- Automated development of standard BI logic: incremental data loads, slowly changing dimensions, surrogate keys, de-normalization.
- Integrated operations framework, with job scheduler, status and error reporting, dependency management, email notification.
- Complete version management for both stored procedures and database objects.
- Ability to load and back out changes from the command line.
- Multiple user definable windows to allow opening of testing or other browser based products.
- The ability to define tests and provide a standard procedure template for running tests in the integrated scheduler.
I don’t usually include content like this verbatim, but it’s highly specific and useful to the point that I’m after here: two worlds are increasingly coming together, and the analytic platform revolution is bringing key development topics to the fore in ways that are unprecedented.
Aster Data Provides Free MapReduce Developer Tool
Aster is one of the more analytic development-focused of the ADBMS vendors, and reinforced that positioning today with the release of its Eclipse-based Aster Data Developer Express for building MapReduce applications to download. It’s a mark of Aster’s focus on the issue that Developer Express includes testing capabilities for locally checking the code before one-click deployment. Developers familiar with Java programming language can complete analytic applications rapidly; The IDE reduces MapReduce coding, automating the generation of glue code that surround the programming logic without requiring MapReduce expertise from developers or analysts.
More to come as the event progresses.
Merv,
You must have been too busy tweeting to listen to what i was saying. Your comment “Orwellian structure is freedom” is the exact opposite of the point I conveyed in my “cede control” principle. Since your primary focus is consulting to vendors, I’m not really sure you understand what resonates with users and what they need to do to really succeed with BI. So, I must say, I found your blog as unsatsifying as you found my keynote. And I’m very sorry about that.
Wayne
I listened carefully, Wayne, and the comments (especially the part about “Orwellian paradoxes”) were intended to be light-hearted. The ceding of control you described came after discussions of control structures far beyond what in my experience – admittedly not as extensive as yours – is typical. (Though their usefulness is hard to dispute.)
In general, I liked the speech – what I said in full was “In general, the discussion was well structured, illustrated with quotes from real users from TDWI’s research, and paced very effectively. . But the agile part – reference to a formal approach to development that has principles of its own and methodological guidelines associated with it – was missing.”
I thought a keynote for an event with Agile as its theme ought to have used agile as more than an adjective, and made reference to the body of thought and practice that exists for the benefit of the attendees. “Agile” is a reserved word to development folks – it doesn’t just mean fast, flexible, effective and other good things. I know you know that. And I think there is an opportunity for a more formal mapping between the two communities.
As for what I understand about users, I began my own career decades ago building Information Center and EIS applications before we called it BI, and followed the market for over 15 years as an analyst at Giga and Forrester working with both users and vendors. I don’t think the last 15 months of vendor-focused work have erased what I learned.
Let’s continue the conversation. Maybe a debate at an upcoming event about whether Agile BI is real, or just a convenient marketing label to sell products (and tickets to conferences) is in order.
Guys, please kiss and make up. When you do, I’ll send you a free copy of our latest Agile BI report http://www.forrester.com/rb/Research/agile_bi_out_of_box/q/id/56722/t/2.
In a pre-conference Webcast in July, Boris, Steve Dine, Ralph Hughes, Paul Kautza, and I discussed the meaning of Agile BI. Like any commonly used (and abused) word in our industry, defining agile BI is a quixotic task: everyone has a different perspective. TDWI’s definition takes at least 10 minutes to explain (4 themes covering organization, methodology, infrastructure, and delivery.) I find users don’t care so much about definitions as how to get things done. So I’ll let you and others debate the semantics. I prefer to focus on offering prescriptive advice, like I did in my keynote.