TDWI Event Focuses on Agile BI. What’s That?

I’m at the Data Warehouse Institute’s San Diego conference this week, and experimenting with an incremental approach to blogging for this event; I’ll try to get on a few times in the next 2 days (unfortunately that’s all the time I’ll have here) and communicate some quick thoughts, as opposed to my more typical style, which is longer and more in depth. That will no doubt follow on many of the topics later.

I begin with the keynote from Wayne Eckerson this morning, where he offered his thoughts on Agile BI. Agile is a loaded word; for developers it means a very specific set of techniques and methodologies. Data folk are not part of that culture in most cases, and they use the word as an adjective. Wayne attempted to bridge the gap in a few places, but by and large, his hints at best practices were not particularly new, or surprising, or tied closely to the Agile playbook. Wayne asked the hundreds in the room how many of them considered themselves agile. (I render it in lower case because he did not specify “as the developers define it.”) Literally fewer than 5 hands went up. His speech focused on some paradoxical ideas; it began with a focus on governance. Counterintuitive, even Orwellian: “structure is freedom;” agility happens if you prepare for it. Not a bad point by any means. Overall, Wayne’s 5 tips were:

1. align with business
2. slow down to speed up
3. anticipate the business
4. manage expectations
5. cede control

In general, the discussion was well structured, illustrated with quotes from real users from TDWI’s research, and paced very effectively. Yet I found it fundamentally unsatisfying, leaving out as it did the “real” Agile story I was hoping for.

WhereScape RED Gets it Right

By contrast, WhereScape, a small but intriguing player in the data warehouse design space, introduced a new release that was much more on target. WhereScape RED 6.5 adds features to support Agile practices like continuous integration testing and integrates with the kind of source control and testing systems Agile developers use. The tools make it easier to back out changes, enabling the kind of iterative process common in development shops but not in data architect thinking. In its press release, WhereScape calls out RED including:

• Automated generation of standard code: error handling, status updates, documentation, parameters, naming conventions, indexing, etc.
• Automated development of standard BI logic: incremental data loads, slowly changing dimensions, surrogate keys, de-normalization.
• Integrated operations framework, with job scheduler, status and error reporting, dependency management, email notification.
• Complete version management for both stored procedures and database objects.
• Ability to load and back out changes from the command line.
• Multiple user definable windows to allow opening of testing or other browser based products.
• The ability to define tests and provide a standard procedure template for running tests in the integrated scheduler.

I don’t usually include content like this verbatim, but it’s highly specific and useful to the point that I’m after here: two worlds are increasingly coming together, and the analytic platform revolution is bringing key development topics to the fore in ways that are unprecedented.

Aster Data Provides Free MapReduce Developer Tool

Aster is one of the more analytic development-focused of the ADBMS vendors, and reinforced that positioning today with the release of its Eclipse-based Aster Data Developer Express for building MapReduce applications to download.  It’s a mark of Aster’s focus on the issue that Developer Express includes testing capabilities for locally checking the code before one-click deployment. Developers familiar with Java programming language can complete analytic applications rapidly; The IDE reduces MapReduce coding, automating the generation of glue code that surround the programming logic without requiring MapReduce expertise from developers or analysts.

More to come as the event progresses.