Reprise of Agile 2006 International Conference

When: September 25th, 2006, 6:00 pm – 8:00 pmWhere: Atomic Object, 941 Wealthy Street SE, Grand Rapids, MI 49506

Six members of XPwm attended the Agile International Conference in Minneapolis this summer. This meeting will be organized like last year’s successful reprise of the Agile conference. A panel of conference attendees will talk about what they found most valuable in the tutorials, workshops, talks, and open space of the conference. Questions are welcome and the discussion is always interesting.

Come to hear:

Carl Erickson – On Ken Schwaber’s observations about the dynamics of quality, economics, legacy code, and company survival

Dave Limbaugh – on “Problem Frames” as introduced in the “Skills for the Agile Designer” tutorial

Patrick Bacon – on Bob Martin’s “Clean Code” presentation

Matt Fletcher – on Bob Martin’s observations on the relationship between unit testing and dynamically typed languages

Justin Dewind – ‘value based xp’, rapid testing approaches of Michael Bolton

David Crosby – adoption of agile by huge companies (British Telecom, Google and Microsoft) and the promiscuous pairing and rapid agile adoption by the Microsoft MSD dialup team

In addition to general observations on the conference, what people are talking about, and where agile is headed.

Lightning Talks: tips, techniques, tools, and more

When: May 23rd, 2006, 6:00 pm – 8:00 pmWhere: Atomic Object, 941 Wealthy Street SE, Grand Rapids MI

The final meeting of the season will feature lightning talks. Named for their duration and intensity, lightning talks give you a bunch of good ideas, observations, tricks, tips, tools, etc in a single meeting. No talk is longer than 5 minutes. No Power Point allowed. You’ll get to hear from as many as 15 presenters in one evening.

XPwm lightning talks can be about anything related to software development. Here are some of the kinds of things we expect to have:

• a new idea
• an evaluation of a tool, process, program, etc.
• an observation
• a story
• a complaint
• an explanation
• a suggestion
• a report of success or failure
• a call to action
• a description of a technique, practice, or technology

Summary of Scott Miller's Talk

Carl Erickson, Atomic Object

Scott Miller gave a presentation on the Presenter First technique of organizing and developing large GUI applications in a test-driven fashion.

Presenter First originated with Michael Marsiglia (formerly of AO) and Brian Harleton (X-Rite) on the X-Rite Intellitrax project. It’s been extended and refined by developers at Atomic Object and Burke Porter Machinery on a variety of projects.

What we want when we build GUI apps: correctness, scalability, portability among platforms, customer-driven, test-driven.

What gets in the way: testing GUIs, commercial tools, complex libraries, mixing presentation and business logic.

“Thin GUI” minimizes the code in the GUI so it doesn’t need automated tests.

First cut: separate Model and View classes, model contains view, model making calls into the GUI. Still a problem as it couples the model to the view.

Michael Feathers Humble Dialog is a major step forward in the thin GUI approach. Model uses an interface that the dialog implements. There is still a problem though. The dialog can call the model when an event happens. But this makes the model responsible for both the business logic and the flow of the application. The model is too fat. And it’s harder to test. The coupling between model and view is still pretty tight.

New goal: complete separation of view and model, segregation of control and business logic. Model View Presenter is a design pattern that does this. Martin Fowler has a good description of MVP on his website.

Presenter knows both Model and View. Neither knows Presenter. View generates events for Model and Model generates events for Presenter. Easy to write unit tests on the Model – it’s mostly a data structure.

Presenter First uses a variant on MVP. Presenter knows both Model and View. View and Model don’t know each other, or the presenter. They publish events that the Presenter subscribes to. The Presenter represents user stories of the app. Model and View are completely de-coupled. Presenter First MVP variant uses Observer pattern. Model and View are subjects, and Presenters are observers.

Where do you start coding? (M, V, or P)

Starting with the model is the worst choice. Very infrastructure-first’ish, with the usual problems. It’s better to delay working on the model until the user stories define the need.

Starting with the view seems very logical as it is what the user sees. All users stories appear to be written in terms of the view. Unfortunately this can be a costly mistake to make. Customers generate a high rate of change requests for the view. The focus is on the interface details. Instead, most user stories can be read in terms of the logic of the application, independent of the widgets in the view. Focusing on the view first makes it very easy to put more code in the view. This means the view then requires testing, which is hard.

The last alternative is the presenter. The presenter encodes the logic of the application. User stories describe what the presenter does. It’s simple to write unit tests on the intentions of the presenter. The code is pretty simple, and the model and view can both be mocked. Developing the presenter first creates a specification for the model and view. No more code is built than is needed.

Steve P observed that all the control flow goes through the presenter.

Scott then started with Visual Studio to demonstrate building an application Presenter First in C#.

Rule #1 Start with the presenter.

Presenter constructor takes an interface for model, and an interface for view. Using interfaces for model and view has the usual advantages, but the most important for PF is to make it easy to mock the model and view when testing the presenter.

Rule #2 Makes interfaces for everything – except presenters.

Presenter doesn’t need a public interface. Everything else does for mocking and testing.

NMock generates code for the concrete mock class (you don’t have to write this). You define the expectations in a unit test, fire the event that the model or view would be emitting, and the unit test library takes care of the rest.

Errors found in unit testing the presenter are generally control flow or application logic errors, sometimes model errors.

Unit tests on the presenter test the functionality described in a user story.

This is a form of what Martin Fowler refers to as interfaction testing, in contrast to state testing.

After the presenter is done, either all related stories, or just a single story, we move to the view. Implementing the view is the eye candy reward for doing the presenter first. We usually implement a quick view and don’t spend a lot of time on the interface. This gets the feature into the clients hands more quickly for feedback. User feedback goes into the story backlog.

The view fires events as actions for the widgets (clicks, keyboard, etc). These are the events that the presenter is subsribed to.

The view is so thin that it can be manually tested. Or if the budget and risk allows, an automated GUI testing tool could be used.

Steve P suggested adding break points everywhere, running tests while removing breakpoints until green bar, then seeing what break points remain to detect untested code. We keep our models as simple as possible. Models that would need this type of analysis would be decomposed for better testability and single responsibility. Brian H recommended Clover for measuring test coverage in conjunction with automated unit testing.

If you catch yourself feeling it’s necessary to make assertions on the private state of the model it usually indicates the model is doing too much. Decomposition ala SRP is called for. Single Responsibility Principle says there should only be one reason for a class to change.

Rule #3 Swear an oath to live by SRP

A large app can’t be built as a single giant MVP triad. A recent project Atomic Object and Burke Porter completed consisted of approximately 100 presenters, 120 models and 120 views. This project was 6 developers working for approximately 9 months.

Models are decomposed according to SRP. MVP triads communicate with each other through their models. This keeps models easy to test. Models are decomposed into helpers, etc (other non-model classes).

Putting it all together

Where are the M, V, and P objects created? Some alternatives:

1. In the main function. This works ok for small apps:

IModel model = new Model(); IView view = new View(); new Presenter(model, view);

Event registration will prevent the presenters from being garbage collected, even if they are created in a method that returns.

2. Use a tool like Spring that lets you declaratively construct your triads.

The biggest advantage of this approach is testability. Missing the creation of a presenter is not immediately obvious. Interfaces offered by Spring let you test for proper composition.

Order dependency between MVP triads is easier to handle in Spring. Spring handles the ordering.

Scott finished the talk by running the puzzle app he showed examples from during the talk.

"Presenter First: Organizing Complex GUI applications for TDD"

Scott Miller, Atomic Object

When: April 25th, 2006, 6:00 pm – 8:00 pm Where: Atomic Object, 941 Wealthy Street SE, Grand Rapids, MI 49506

Applying test-driven development to the graphical user interface portion of complex applications is complicated. Testing through the GUI by inserting events and making assertions on the state of the application is expensive and increases the burden of test maintenance. Presenter First uses a variant of Model View Presenter (MVP), a particular means of composing MVP triads, and a strategy for the order in which the elements of the triad are developed and tested. Atomic Object has used Presenter First on projects ranging in size from several to several hundred MVP triads for application development in C# and Java.

This presentation will reveal the difficulties we encountered in scaling the use of MVP, discuss the approaches we tried and rejected, and describe our GUI application development TDD best practices. The talk will include a demonstration of tools, code samples, and example problems in C# using Visual Studio .NET.

Speaker Bio

Scott Miller is a developer at Atomic Object. He’s been professionally programming since 1989. He’s worked at companies ranging from startups to high growth product companies to giant multinationals. He joined Atomic Object in May 2005 and has most recently been a member of the team working on a large C# application using the Presenter First organization approach.

Summary of Ron Jeffries' Talk, with comments

Shawn Crowley, Atomic Object

The advantages of an agile approach: Traditional development methods (many requirements up front) often miss the deadline because they don’t give a good indication of development velocity.

The customer has no idea of what the real development velocity of the project is going to be until development starts.

Because of the lengthy requirements phase in traditional project schedules, the customer may not realize they are going to blow a deadline until they have already invested a significant amount of money.

Agile processes give the customer more decision making power . A project’s development velocity can be seen right away, giving the customer a better indicator if they are going to hit their date. This timely information call allow the customer to take controlled actions like:

• Reduce scope
• Extend schedule

Usually hitting the date is the most important because being on time is what most people will remember.

Importance of tests:

One of the most important things to the customer is for the development effort to be predictable. The best way to be predictable is to have running tested features. It is import to look at a burn down chart as running, tested, features. Without tests you may be putting defects (negative features) into the software. Your curve may not be where you think it is if there are defects lurking in the software. A possible remedy for a lack of tests is putting a block of time for testing into the schedule after development. The trouble with adding a testing phase is that no one has a real indicator of how long the testing phase will take.

It is important to have to layers of testing. Unit tests provide a way for the developer to know that their code is doing what they think it is doing. Units tests are not enough though because they do not assert what the customer wants as features (that is the job of system tests).

System tests (automated customer tests) need to be part of the feedback loop between the customer and the developer. These tests will tell that a feature is done. When a feature is too large, try to reduce the scope of the story to a minimal thing that the customer can recognize as progress. Bugs are easier to identify when the scope of a feature is smaller.

Refactoring:

The software’s design should follow the growth of the features. No frameworks up front. If a design is good enough for today, is good enough to add one more feature. After a feature is put into the software, see if it adds an ugly blob to the software and refactor it if it does.

A design that is inadequate for tomorrow is inadequate for today. Don’t sit with bad design when adding features (refactor as necessary).

Most important development skills:

• Small features
• Short iterations
• Testing
  • Test driven development
  • Customer acceptance testing
• Refactoring

A video of Ron Jeffries' presentation is available on Google Video.

“Making the Date” and Q&A

Ron Jeffries, 
Independent Consultant

When: March 28th, 2006, 6:00 pm – 8:00 pm Where: Atomic Object, 941 Wealthy Street SE, Grand Rapids, MI 49506

The March meeting of XPwm features Ron Jeffries. The first half will be a presentation from Ron on the confused dynamics of project dates. The second half will be an open question session. Bring your thorny problems, your myths in need of debunking, your agile adoption stories, your test-driven dilemmas or your complaints about the weather. Ron promises to tackle them all.

Never shy of making provocative statements, Ron Jeffries recently wrote this on the subject of projects and deadlines:

“In my opinion, every project should begin with high goals and a short deadline. And, with rare exceptions, every project should deliver its product by the deadline. Really. Every single project.”

But who works to make the date?

“It seems like every development project begins with the date, and we’re held responsible for ‘making the date.’ Making the date is not a development responsibility.”

In this talk Ron considers the distinct and essential roles of management, customers and developers with respect to projects dates.

Speaker Bio

Ron Jeffries is a prolific author, industry leader and practitioner in the Extreme Programming (XP) movement. A true XP pioneer, Mr. Jeffries began practicing this agile development process in 1996. He was one of the original authors of the Manifesto for Agile Software Development (http://agilemanifesto.org), and was the on-site coach for the first XP project at Chrysler in 1998 (http://www.xprogramming.com/publications/dc9810cs.pdf). Now an independent consultant, he has presented numerous talks and published several papers on the topic. He and his teams have built operating systems, compilers, relational database systems, and a wide range of applications.

In addition to his consulting work, Ron Jeffries is author or co-author of several articles and books including Extreme Programming Installed and his most recent book, Extreme Programming Adventures in C# (DV-Microsoft Professional). Ron is editor of the website XProgramming.com, and is, by far, the most prolific on-line author in Extreme Programming and related topics.

Notes on Scott Ambler's Talk

Carl Erickson, Atomic Object

Scott Ambler spoke at XPwm tonight. These are the notes I took during the meeting.

Data can be done in an agile fashion. Nothing prevents evolutionary development of data models and databases.

Requirements documents are incredibly efficient means of throwing money away.

The agile community takes flack about providing “proof” that agile practices work. Where’s the proof from the heavyweight world that their methodologies work?

Specialists and document interfaces don’t work in practice. We should worry that that’s how we’re told things should be done.

Agile comes from the object community. The object community wasn’t good about recognizing the importance of data. These biases have crept into agile. On the other hand the data community hasn’t kept up with new practices. The blame is shared. The rift is wide.

Whiteboards are the most commonly deployed CASE or modeling tool.

XP gets beat up for not modeling. User stories, acceptance tests, sketches, diagrams – these are all models. The second addition of Kent’s book on XP starts with a model: a mind map.

Agile data is at the lefthand of the adoption curve – these ideas work, but they might be too early for your organization.

Database refactoring has not “crossed the chasm”. Who can rename a column in a production database and rollout new apps in a single day (not many hands raised). This is doable. Why is a trivial change so hard? What does this say about the data community? (Remember: If you can do stuff your competitors can’t, that’s a competitive advantage.)

Agile Data or Crystal? Which is the flakiest agile method? (Scott argues this point with Alistair Cockburn)

Data isn’t the only important thing. We shouldn’t only focus on a single concern. Data is important, it’s not singular.

The data community usually has a better handle on enterprise issues and global scope.

Evolutionary data model means iterative + incremental. Evolutionary programming means incremental + highly collaborative

XP teams put a price tag on documentation. Once you do this you tend not to build as much documentation. $50k for a requirements doc? What can I get for $5k?

Something the agile community isn’t good at: explaining what we do at the start of the project (cycle 0). We start modeling. We decompose into stories. We noodle about the architecture. We sketch on the whiteboard. We do just enough modeling, just in time, to understand the problem.

In agile modeling the value of reviews goes away. Reviews is a compensatory practice. It compensates for people doing solo work, their own way. Modeling should be done together. Continuous review reduces the risk and replaces the need for formal reviews. People who gather data on the value of reviews in agile teams find the reviews don’t pay for themselves.

Standish Group study: 65% of features delivered by traditional waterfall processes were never or rarely used. This represents 2/3 wastage on “successful” projects.

Acceptance testing in a conventional methodology is a joke. If it doesn’t satisfy users it’s often too late to change. Change management is really change prevention.

“Find the people who know the least about successful software development projects and ask them how to do things. This is what CMM is all about.”

Up-front high-level modeling: think through the big issues, don’t shoot yourself in the foot, don’t waste effort.

“If you don’t have access to your stakeholders, cancel the project. If your project is so simple you don’t need access to your stakeholders, the project is a waste of money.”

Second half…

Scott’s book on DB Refactoring comes out in a few weeks.

Database refactoring: a simple change to a database schema that improves it design while retaining both its behavioral and informational semantics.

Example refactoring: take a single format for telephone numbers and apply it to phone numbers stored as a string (with characters like: -, (, ., etc)

DB refactoring is hard because you break a 100 apps with a simple change. There is very tight coupling between the apps and the database. This is bad architecture.

Rename Fname to FirstName.

1. Add the new column
2. Add a trigger to keep Fname and FirstName sychronized
3. Remove the Fname column once no apps reference it (eventually)

This is like deprecating the Fname column.

You need a good regression test suite to do refactoring (just like code).

Quiz: how many people work in a place where they’ve got a full regression test suite? how many data groups are responsible for data quality? (almost no hands up on either; most times lots of hands up on the latter, none on the former) The point is that data quality requires testing.

Question from audience: How do you track what’s still using deprecated views, columns, etc? A: Some databases support knowing this. Regression test suites tell you the rest.

Question from audience: How do you communicate deprecations to the team? A: Mailing list. (Sun manages this for 10^6 Java programmers.) Maybe the better question is, why do data tools suck?

Question from audience: Who pays for the refactoring? A: Whichever team hits the need first.

Q: What’s the motivation for change? A: A new feature request that requires the change.

“Clue number one that you should be dealing with an app/database is that you’re afraid to touch it.”

The solution to the tight coupling above? A data access layer.

DB tools are bad now. Code refactoring tools were bad a few years ago. DB tools will get better: automated deprecation detection, finding problems. Cold harsh truth: data people aren’t toolsmiths. This is why the tools are bad. Vendors are taking this more seriously now.

“If I’m writing in Java or C# in my database, should that not be tested?” (triggers, stored procedures, functions, ...)

“Should we not be offended by the lack of software engineering in the data community?” We hear a lot about data quality, but nothing about testing.

“Coupling is your enemy, encapsulation is your ally.”

“Promote the concept of generalizing specialists.” A team of craftspeople or renaissance developers is more productive. Vendors encourage specialization.

A video of Scott's presentation is available on Google Video.

"Agile Database Techniques: Data Doesn’t Have To Be A Four-Letter Word Anymore"

Scott Ambler

When: February 27th, 2006, 6:00 pm - 8:00pm Where: Atomic Object, 941 Wealthy Street SE, Grand Rapids, MI 49506

Note: this meeting is on MONDAY, not our usual Tuesday.

Data is clearly an important aspect of software-based systems, a fact that the information technology (IT) industry has understood for decades, yet many agile development teams are struggling to involve data professionals within their projects. The Agile Data (AD) methodology (www.agiledata.org) defines a philosophical framework for data-oriented activities within agile projects, defining ways that application developers and data professionals can work together effectively. However, philosophy isn't enough, you also need proven techniques which support those philosophies. In this presentation Scott Ambler discusses techniques for agile database development, including: database refactoring, Agile-Model Driven Development (AMDD), Test-Driven Design (TDD), and environment/tool strategies.

Speaker Bio

Scott W. Ambler is an industry-recognized software process improvement (SPI) expert. He is the practice leader of the Agile Modeling (AM), Agile Data (AD), Agile Unified Process (AUP), and Enterprise Unified Process (EUP) methodologies. Scott is the (co-)author of several books, including Refactoring Databases (Prentice Hall), Agile Modeling (John Wiley & Sons), Agile Database Techniques (John Wiley & Sons), The Object Primer 3rd Edition (Cambridge University Press), and The Enterprise Unified Process (Prentice Hall). Scott is a contributing editor with Software Development magazine. His personal home page is www.ambysoft.com/scottAmbler.html.

Notes on Carl Erickson's Talk

Mike Karlesky, Atomic Object

- Useful requirements, engaged customers - There appears to be an analogy between the development circle of the XP approach and customer interface circle. - Development tasks, Simple Design, Unit tests -> Customer stories, Simple features, Acceptance tests - Not so simple. The customer is an unknown and complicating factor in the equation.

Struggling through Story Driven Development Benefits:

1. Requirements

• when you need them
• in small chunks
• actionable, unambiguous, useful
• opportunity to learn as you go

2. Predictability

• stories can be estimated (small chunks)
• project velocity can be measured
• rhythm is established

3. Feature Management

• keeping it simple
• staying focused
• putting a cost on features
• letting the customer steer

4. Tools Atomic Object uses for SDD

• Project Management: Basecamp
• Iteration Planning: Explain
• Estimating & Time Tracking: PunchIt (homegrown)

Comments / Q&A

Perhaps the outer ring of the XP practices map is so difficult because that ring is simply incredibly difficult. Perhaps, we have become effective at the inner ring (development) and now there is a newly revealed need for a new “testament” in the XP bible.

"Introduction to Agile + Story Driven Development"

Carl Erickson, Atomic Object

When: January 24th, 2006, 6:00 pm – 8:00 pm Where: Atomic Object, 941 Wealthy Street SE, Grand Rapids, MI 49506

Introduction to Agile

The first half of the presentation will be a general introduction to agile software development practices in general and Extreme Programming (XP) in particular. XP has consistently proven effective in yielding software success: projects that are on time, on budget, and free of defects as well as developers, managers, and customers who actually enjoy the process and are satisfied with the results.

Story-driven Development

The second half of the presentation will concentrate on the difficulties and benefits of organizing a project around user stories. The tools Atomic Object uses to support SDD will be demonstrated. This talk draws on experience with projects from a few person-weeks to 50 person-months of effort.

Test-driven (TDD) development is revolutionizing the way software is designed, built and tested. TDD requires the automation of unit tests and applies to the daily work of software development – crafting classes, methods and blocks of source code. Applying the same approach at the project level leads to Story-driven development (SDD). SDD drives a requirements conversation between developers and customers at a scale and point in time when requirements gathering is most effective. Developers work on stories according to the customer’s priorities; developers and customers create a shared mental model of the project, one story at a time, as each story is being worked on. The concrete outcome of this conversation is a collection of acceptance tests associated with each story.

Carl Erickson

Carl Erickson is the president of Atomic Object LLC. Atomic Object provides software development services including contract programming, development training, and software process improvement consulting. Atomic has been using Extreme Programming practices for more than five years for clients in the areas of color measurement, aerospace, automated guided vehicles, health care, e-commerce, and automotive testing.

Carl received his PhD in computer engineering in 1991 from Michigan State University and taught computer science courses at Grand Valley State University, Michigan, until 2000. He maintains a part-time position as an industrial lecturer at Uppsala University in Uppsala, Sweden, where he most recently introduced a course in software craftsmanship