Notes on Scott Ambler's Talk
Carl Erickson, Atomic Object
Scott Ambler spoke at XPwm tonight. These are the notes I took during the meeting.
Data can be done in an agile fashion. Nothing prevents evolutionary development of data models and databases.
Requirements documents are incredibly efficient means of throwing money away.
The agile community takes flack about providing “proof” that agile practices work. Where’s the proof from the heavyweight world that their methodologies work?
Specialists and document interfaces don’t work in practice. We should worry that that’s how we’re told things should be done.
Agile comes from the object community. The object community wasn’t good about recognizing the importance of data. These biases have crept into agile. On the other hand the data community hasn’t kept up with new practices. The blame is shared. The rift is wide.
Whiteboards are the most commonly deployed CASE or modeling tool.
XP gets beat up for not modeling. User stories, acceptance tests, sketches, diagrams – these are all models. The second addition of Kent’s book on XP starts with a model: a mind map.
Agile data is at the lefthand of the adoption curve – these ideas work, but they might be too early for your organization.
Database refactoring has not “crossed the chasm”. Who can rename a column in a production database and rollout new apps in a single day (not many hands raised). This is doable. Why is a trivial change so hard? What does this say about the data community? (Remember: If you can do stuff your competitors can’t, that’s a competitive advantage.)
Agile Data or Crystal? Which is the flakiest agile method? (Scott argues this point with Alistair Cockburn)
Data isn’t the only important thing. We shouldn’t only focus on a single concern. Data is important, it’s not singular.
The data community usually has a better handle on enterprise issues and global scope.
Evolutionary data model means iterative + incremental. Evolutionary programming means incremental + highly collaborative
XP teams put a price tag on documentation. Once you do this you tend not to build as much documentation. $50k for a requirements doc? What can I get for $5k?
Something the agile community isn’t good at: explaining what we do at the start of the project (cycle 0). We start modeling. We decompose into stories. We noodle about the architecture. We sketch on the whiteboard. We do just enough modeling, just in time, to understand the problem.
In agile modeling the value of reviews goes away. Reviews is a compensatory practice. It compensates for people doing solo work, their own way. Modeling should be done together. Continuous review reduces the risk and replaces the need for formal reviews. People who gather data on the value of reviews in agile teams find the reviews don’t pay for themselves.
Standish Group study: 65% of features delivered by traditional waterfall processes were never or rarely used. This represents 2/3 wastage on “successful” projects.
Acceptance testing in a conventional methodology is a joke. If it doesn’t satisfy users it’s often too late to change. Change management is really change prevention.
“Find the people who know the least about successful software development projects and ask them how to do things. This is what CMM is all about.”
Up-front high-level modeling: think through the big issues, don’t shoot yourself in the foot, don’t waste effort.
“If you don’t have access to your stakeholders, cancel the project. If your project is so simple you don’t need access to your stakeholders, the project is a waste of money.”
Second half…
Scott’s book on DB Refactoring comes out in a few weeks.
Database refactoring: a simple change to a database schema that improves it design while retaining both its behavioral and informational semantics.
Example refactoring: take a single format for telephone numbers and apply it to phone numbers stored as a string (with characters like: -, (, ., etc)
DB refactoring is hard because you break a 100 apps with a simple change. There is very tight coupling between the apps and the database. This is bad architecture.
Rename Fname to FirstName.
- Add the new column
- Add a trigger to keep Fname and FirstName sychronized
- Remove the Fname column once no apps reference it (eventually)
This is like deprecating the Fname column.
You need a good regression test suite to do refactoring (just like code).
Quiz: how many people work in a place where they’ve got a full regression test suite? how many data groups are responsible for data quality? (almost no hands up on either; most times lots of hands up on the latter, none on the former) The point is that data quality requires testing.
Question from audience: How do you track what’s still using deprecated views, columns, etc? A: Some databases support knowing this. Regression test suites tell you the rest.
Question from audience: How do you communicate deprecations to the team? A: Mailing list. (Sun manages this for 10^6 Java programmers.) Maybe the better question is, why do data tools suck?
Question from audience: Who pays for the refactoring? A: Whichever team hits the need first.
Q: What’s the motivation for change? A: A new feature request that requires the change.
“Clue number one that you should be dealing with an app/database is that you’re afraid to touch it.”
The solution to the tight coupling above? A data access layer.
DB tools are bad now. Code refactoring tools were bad a few years ago. DB tools will get better: automated deprecation detection, finding problems. Cold harsh truth: data people aren’t toolsmiths. This is why the tools are bad. Vendors are taking this more seriously now.
“If I’m writing in Java or C# in my database, should that not be tested?” (triggers, stored procedures, functions, ...)
“Should we not be offended by the lack of software engineering in the data community?” We hear a lot about data quality, but nothing about testing.
“Coupling is your enemy, encapsulation is your ally.”
“Promote the concept of generalizing specialists.” A team of craftspeople or renaissance developers is more productive. Vendors encourage specialization.
A video of Scott's presentation is available on Google Video.