Just differentiate between what Agile is and how people are pushing Agile on you. Different things entirely.
Agile is best practices around iterative and incremental development. Period. Yes, there's a manifesto and there's Scrum and all sorts of other things, but at the end of the day Agile is a marketing term. A place to go find out what people are trying in order to get better.
Two secrets here. One, team success is about 90% dependent on who is on the team. Good teams do well. Bad teams do poorly. But if you're going to have more than one team, you need some way to measure and talk about what they are doing, so you have to have something.
Agile -- when done correctly -- is just the minimum amount of something you need to get your work done. It is the minimum amount of process. After all, it's not like you can have no process at all. Whatever you're doing already is a process.
I've seen hundreds of teams struggle with Agile, and to me the problem is that we do a really bad job of balancing the difference between "here's a best practice that worked awesome for about 80% of the teams that tried it" and "These are the rules. You must do this." In many shops, Agile is just another way of micro-managing teams, sadly.
I can't help that. I also can't help the fact that lots of folks are out to make a buck on certifications and such. I don't think that makes Agile bad. Heck, I'm not even against certifications, although I'd never get one. I just think we take things too far.
I like the idea of having a marketing blurb "Agile" where I can go to find out what kinds of new stuff is being done. It helps me pick better reading material.
The second secret is that most teams, frankly, are not that good. So you need some kind of way to demonstrate that as well. Making things incrementally and in small iterations lets you see how bad teams are early on. You fail early and often. Then at least you have a bit of a shot at trying to help.
But believe me, I feel your pain. Sounds to me like you are a lot more upset at modern marketing and management attitudes than Agile. Remember that we technical folks have a way of over-doing whatever we get into, as I was writing in my blog before your post appeared! (It's still very strange to me watching HN how the same topic comes from multiple writers at the same time) http://tiny-giant-books.com/blog/agile-means-stop-focusing-o...
I got involved with this stuff before the term "Agile" existed. At the beginning, it was a bunch of professionals (mostly developers) sincerely trying to find better ways of working. Extreme Programming, for example, came to be because the developers were really interested to experiment with how their team got stuff done.
It breaks my heart that in the ensuing decade it has turned into exactly the kind of bullshit, top-down, PHB-fluffing idiocy that early agilists were trying to get away from. If you look at the Agile Manifesto, the focus is supposed to be on a) people, b) shipping working software, c) collaboration, and d) being adaptable. That is sure not what it has become.
In my view, we made a crucial mistake: we didn't think about money and power enough. Now the Agile industry is 98% selling idiotic certifications and homeopathic doses of process improvement to organizations that don't really want to change anything at all.
Mad. It makes me mad. Sorry we screwed it up, everybody.
The only people that screwed it up are the ones that monetized it. They made it
a) rigid based on their definition
b) defined themselves as experts in making teams agile based on their rigid definition
c) charged for it
PS. These same guys, once they couldn't squeeze anymore blood out of the Agile stone, moved onto a new marketing term, "craftsmanship". They now charge the same clients, even more money, to teach them this "new way of doing it ..... right".
Plus, they make a mint on the books they hastily write and push out.
I eagerly anticipate that successor to Craftsmanship.
I disagree that craftsmanship is a replacement for agile. To me, being a software craftsman is more about how you write and structure code. Agile is about figuring out what the end product should be and how you get there.
I'm reflecting on how the same "agile experts that sell/sold their services" quickly abandoned selling that and moved onto selling "craftsmanship" once their Agile well dried up.
Depends on who you're talking about here. Bob Martin one of the craftsmanship people, is very sincere in his desire to make the field better. I'm not sure about who's cropped up lately, though.
Also, it's very, very rare for somebody to make a mint on a software book. I've talked with a number of authors, and their universal view is that writing code pays much better than writing a book. You do it because you have something to say, not because you want to get rich.
> Bob Martin one of the craftsmanship people, is very sincere in his desire to make the field better.
Unfortunately, you can be totally sincere in your good intentions, and yet still repeatedly be wrong. When you are a high profile figure who presumes to advise others on the best ways to do their job, that makes you a liability.
It's a shame. Some of Bob Martin's earlier work exploring OO and the SOLID principles was quite decent stuff. But I think it's obvious at this point that he and several of his colleagues at Object Mentor have collectively lost the plot.
Could be. I guess I'm not aware of the repeated wrongness on ObjectMentor's part. Got links?
The big problems I saw, though, came from people who weren't particularly sincere. They were happy to sell whatever large companies were buying. E.g., two-day "Scrum Master" courses and a splash of Agile holy water to bless whatever top-down idiocy a company was already engaging in.
At this point, a comprehensive critique of Object Mentor would be more a case of writing a book than posting a few links. However, in the interests of not attacking them completely without justification:
- Object Mentor are big advocates of XP. The fundamental principle of XP is that a certain practice is good, then doing more of it must be better. There is no logic in that position at all, and it doesn't stand up to even cursory criticism. Moreover, if XP is as superior to other processes as the typical advocacy quotes and statistics imply, how come organisations using XP aren't consistently reporting dramatically better measurable results and how come so few software development groups have chosen to adopt it? Sooner or later, people notice that the emperor has no clothes. (I suspect this is why we now have Software Craftsmanship: it's a new positive-sounding but conveniently meaningless marketing term to pitch to clients.)
- Bob Martin has repeatedly stated that anyone who doesn't do TDD is unprofessional. Safety-critical software is typically not developed using TDD; in fact, formal methods, BUFD, and other very much not Agile processes are often used in such fields.
- Michael Feathers redefined the term "legacy code" in terms of unit tests. There are decades of research studying what actually causes a project to decay to the point that it is difficult to maintain and update. To my knowledge, a lack of unit tests has not yet been cited as a causal factor by any paper on the subject. (FWIW, I do think Feathers' book on the subject offered some interesting and worthwhile ideas, I just don't accept his premise that having unit tests is what defines whether code is legacy or not for practical maintenance/project management purposes. I think when you try to co-opt an ill-defined but commonly understood term and give it a formal definition that is very different to the mainstream concept, you lose some credibility.)
- Brett Schuchert, a man writing a book on C++, managed to make "Hello, world" take five source files and a makefile totalling nearly 100 lines, using TDD of course.
- Ron Jeffries. Sudoku. Probably enough said. TDD is not an alternative to understanding the problem and how you're going to solve it.
- From a post on the Object Mentor blog, Brett Schuchert apparently advocates pair programming based on a 1975 study of something involving two-person teams, a ten-year-old study of university students, and a couple of links to secondary sources. The original research for almost every one of the primary sources he appeals to either directly or indirectly is no longer available at the cited links less than 18 months later.
- Bob Martin thinks there are no more new kinds of programming language left to find. That's roughly on par with equating Haskell and Brainfuck because they're both Turing complete, and shows a complete lack of awareness of the state of the art.
- When it comes to the amount of up-front design and formal architecture that makes sense for a project, the amount of retconning in recent comments from the TDD guys is laughable. There was a particular interview featuring Bob Martin and Jim Coplien a couple of years back that was almost painful to watch.
I could go on, but if that lot doesn't paint a clear enough picture for anyone reading this, I don't have a powerful enough Kool-Aid antidote to help them.
I do agree with you about the insincerity. That's worse in theory, but unfortunately it's probably no less damaging in practice.
Edit: Here are few links to support some of the points above.
You start out with a giant misunderstanding, which makes it hard for me to take the rest of your complaints seriously. Extreme Programming is not an arbitrary desire to turn all the knobs you can find to 11. It started as a question: what happens if we take certain practices that are good and do them more intensely? E.g. if some testing is good, what if we test pretty much everything?
That team found that they really liked turning particular practices way up. But you can't turn all the knobs up, so you are implicitly turning others down. E.g., if you turn up iteration speed, then you are turning down the sort of heavyweight waterfall requirements process ubiquitous at the time.
So the "extreme" was a way to explore the space of possible processes, not any sort of fundamental principle. Teams trying XP are explicitly encouraged to experiment similarly. I sure have; the process we use is derived from XP but departs from it in a number of areas.
I think a lot of the rest of your points are similar misunderstandings along with some cherry picking. E.g., the OM blog post on pairing. He said that people sometimes asked him for basic background materials, so he posted some links. To go from that to "Brett Schuchert apparently advocates pair programming based on.." is either very poor reading comprehension or the act of somebody with an axe to grind.
As to not doing TDD being unprofessional, I'd generally agree. I tried TDD first in 2001, and have worked on a number of code bases since. For any significant code base that's meant to last and be maintainable, I think it's irresponsible to not have a good unit test suite. I also think there's no more efficient way to get a solid suite than TDD.
If you (or anybody) wants to discuss this further, probably better to email me; that's easy to find from my profile.
Extreme Programming is not an arbitrary desire to turn all the knobs you can find to 11. It started as a question: what happens if we take certain practices that are good and do them more intensely? E.g. if some testing is good, what if we test pretty much everything? That team found that they really liked turning particular practices way up.
Well, of course they're entitled to their opinion, but that's all it is: an opinion. An argument that if some testing is good then test-driving everything must be better, or that if code review is good then full-time review via pair programming must be better, has no basis in logic. And those kinds of arguments go right back to the original book by Kent Beck, and they have been propagated by the XP consultancy crowd from the top right on down ever since.
IMHO, if a trainer is going to go around telling people that if they don't program a certain way then they are unprofessional, then that trainer had better have rock solid empirical data to back up his position. Maybe as you say, I do have a giant misunderstanding, and in fact Object Mentor do make their case based on robust evidence rather than the sort of illogical arguments I've mentioned. In that case, I assume you can cite plenty of examples of this evidence-based approach in their published work, so we can all see it for ourselves. Go ahead; I'll wait.
I think a lot of the rest of your points are similar misunderstandings along with some cherry picking. E.g., the OM blog post on pairing. He said that people sometimes asked him for basic background materials, so he posted some links. To go from that to "Brett Schuchert apparently advocates pair programming based on.." is either very poor reading comprehension or the act of somebody with an axe to grind.
This is a consultant who presumes to tell others how to do their job, openly posting asking for any source material from others to back up his predetermined position, and then claiming in almost the very next sentence to favour material based on research or experience. He says that the links he gave (the ones where much of the original research is either clearly based on flawed-at-best methodologies or simply not there at all any more) are things he often cites. And he gives no indication, either in that post or anywhere else that I have seen, of having any library of other links to reports of properly conducted studies that support his position. I don't think criticism based on this kind of post is cherry-picking at all, but of course if it is then again you should have no difficulty citing lots of other material from the same consultant that is of better quality and supported by more robust evidence, to demonstrate how the post I picked on was an outlier.
The same goes for any of my other points. If you think I'm cherry-picking, all you have to do to prove it is give a few other examples that refute my point and show that the case I picked on was the exception and not the rule. If you can't do that -- and whether or not you choose to continue the debate here, you know whether you can do that -- then I think you have to accept that I'm not really cherry-picking at all.
As to not doing TDD being unprofessional, I'd generally agree. I tried TDD first in 2001, and have worked on a number of code bases since. For any significant code base that's meant to last and be maintainable, I think it's irresponsible to not have a good unit test suite. I also think there's no more efficient way to get a solid suite than TDD.
Please note that I'm not disputing that an automated unit test suite can be a useful tool. On the contrary, in many contexts I think unit testing is valuable, and I have seen plenty of research that support such a conclusion more widely than my inevitably limited personal experience.
On the other hand, I don't accept your premise about TDD. For one thing, TDD implies a lot more than merely the creation of unit tests. Among other things, I've worked on projects where bugs really could result in very bad things happening. You don't build that sort of software by trial and error. You have a very clear statement of requirements before you start, and you have a rigorous change request process if those requirements need to be updated over time. You might have formal models of your entire system, in which case you analyse your requirements and determine how to meet them at that level before you even start writing code. At the very least, you probably have your data structures and algorithms worked out in advance, and you get them peer reviewed, possibly by several reviewers looking from different perspectives. Your quality processes probably do involve some sort of formal code review and/or active walkthrough after the code is done, too.
If you came into an environment like that, and claimed that the only "professional" thing to do was to skip all that formal specification and up-front design and systematic modelling and structured peer review, and instead to make up a few test cases as you went along and trust that your code was OK as long as it passed them all, you would be laughed out of the building five minutes later. If you suggested that working in real time with one other developer was a substitute for independent peer review at a distance, they'd just chuck you right out the window to save time.
TDD is not an alternative to understanding the underlying problem you're trying to solve and knowing how to solve it. A test suite is not a substitute for a specification. Pair programming is not a substitute for formal peer review. They never have been, and they never can be.
I haven't gone into it here, but of course there are other areas where TDD simply doesn't work either. Unit testing is at its best when you're working with pure code and discrete inputs and outputs. It's much harder to TDD an algorithm with a continuous input and/or output space. Tell me, how would you test-drive a medical rendering system, which accepts data from a scanner and is required to display a 3D visualisation of parts of a human body based on the readings? Even if this particular example weren't potentially safety-critical, how would you even start to test-drive code where the input consists of thousands of data points, the processing consists of running complex algorithms to compute many more pieces of data, and the observable output is a visualisation of that computed data that varies in real time as the operator moves their "camera" around?
If you (or anybody) wants to discuss this further, probably better to email me; that's easy to find from my profile.
I appreciate the offer, but I prefer to keep debates that start on a public forum out in the open. That way everyone reading can examine any evidence provided for themselves and draw their own conclusions about which positions stand up to scrutiny.
An argument that if some testing is good then test-driving everything must be better, or that if code review is good then full-time review via pair programming must be better, has no basis in logic.
That's not the argument at all. That is, as I just said, the reason they decided to try that. Their reasons for continuing to do it and further to recommend it are entirely different.
[...] better have rock solid empirical data [...]
You do realize that almost everything that goes on in the industry is not based on rock-solid empirical evidence, right? And also, that you're privileging an arbitrary historical accident by saying that new thing X has to have evidence when the common practice doesn't?
If you came into an environment like that, and only "professional" thing to do was to [...] make up a few test cases as you went along and trust that your code was OK [...]
That is not something I have ever heard any Object Mentor person say, and it's not something I said. It's so far from what I've ever heard somebody like Bob Martin or Kent Beck say that your misunderstanding is so dramatic that I have a hard time believing it's not willful.
I prefer to keep debates that start on a public forum out in the open.
Well, I'm not trying to have a debate. If you'd like to have one, you'll have to do it without me.
Their reasons for continuing to do it and further to recommend it are entirely different.
So you keep saying. The problem is, almost everything Object Mentor advocate does seem to be based on some combination of their personal experience and pure faith. I object to someone telling me that my colleagues and I are "unprofessional" because we happen to believe differently, particularly when we do have measurable data that shows our performance is significantly better than the industry average.
You do realize that almost everything that goes on in the industry is not based on rock-solid empirical evidence, right?
That may be so, but most people in the industry aren't telling me how to do my job, and insulting me for not believing the same things they do.
That is not something I have ever heard any Object Mentor person say, and it's not something I said.
Good for you. XP consultants have been making essentially that argument, in public, for many years. TDD without any planning ahead is inherently a trial-and-error approach, which fails spectacularly in the absence of understanding as Jeffries so wonderfully demonstrated. Plenty of consultants -- including some of those from Object Mentor -- have given various arbitrary amounts of time they think you should spend on forward planning before you dive into TDD and writing real code, and often those periods have been as short as half an hour. You may choose not to believe that if you wish. I'm not sure even they really believe it any more, as they've backpeddled and weasel-worded and retconned that whole issue repeatedly in their recent work. But I've read the articles and watched the videos and sat in the conference presentations and heard them make an argument with literally no more substance than what I wrote there.
You keep saying that I'm misunderstanding or cherry-picking evidence. Perhaps that is so and I really am missing something important in this whole discussion. However, as far as I can see, throughout this entire thread you haven't actually provided a single counterexample or alternative interpretation of the advice that consultants like those at Object Mentor routinely and publicly give. You're just saying I'm wrong, because, and there's not really anything I can say to answer that.
That's because you're trying to have a debate with the Object Mentor guys rather than a discussion with me. Your problem with them isn't my problem, and neither are your misunderstandings. It is not my job to argue you into a better understanding of something you clearly can't stand.
Like many early Agile adopters I have been shocked and saddened by what it has become. I think the problem is that brainfucked corporate IT programmers (Java, .NET, whatever) have jumped on the bandwagon and driven it into the sea.
I was recently smacked around the head by this in a meeting about how a team was going to implement Jira ticketing to help improve their development process. I am not sure entirely what I was expecting, but I was totally, totally unprepared for the mind-stunning crap that I got - an endless series of slides of things called "workflows" which apparently document the life-cycle of a ticket and need to be mapped out in excruciating gory detail. I decided to raise my hand and ask a simple question "how do I just raise a ticket and assign it to someone" - the presenter gave me this kind of stunned "why would you want to do that??" look and explained that it doesn't work that way, first you need to assign everyone into teams and roles (like tester, developer etc), and then the workflow will decide where the ticket goes based on the kind of ticket and it's status. Mind blown. I could almost appreciate how such system might be of use on a really large project where you don't know and interact with most project members directly - they had a team of 8 people including testers and analysts. And in their minds they were doing Agile.
In another case I was arguing for the use of the Jira Fisheye plugin to enable meaningful access to VC to enable things like code review and release diffs and the like. The so called "Agile practitioners" had a different idea however - they were keen on a different plugin called Greenhopper which apparently does Agile process management and the like. I knew something was wrong when it was mentioned that it does nice Gantt charts. I probed a little deeper into why they didn't see the value in Fisheye and found that their actual VC practices were a joke - ludicrously large commits with minimal comment just before a release, no consistent release tags, broken commit history (they seemed to be deleting and re-creating their trunk after every release...what I don't even...). Advanced process/project management techniques over getting the basics like VC right? That doesn't sound like Agile in any universe I am familiar with.
I really struggle to think of any other examples where a worthy movement has been so thoroughly corrupted and debased into something almost diametrically opposed to it's original vision. Fuck Agile programming and what it has become. I think I'll stick with being a Pragmatic programmer.
You are saying that Agile is supposed to mean programming in Ruby instead of Java? And only small teams can be Agile? Amazon.com would disagree with you, for one.
"You are saying that Agile is supposed to mean programming in Ruby instead of Java?"
Of course not. But in my experience and observations, corporate IT programmers, especially those with a Java background, are the worst offenders when it comes to abusing the Agile concept and failing to get the basics anywhere near right.
"And only small teams can be Agile? Amazon.com would disagree with you, for one."
I am not saying that, but I think it is a good question. I have never seen a team of hundreds coordinated to an outcome using an Agile approach. I have been involved in large projects with up to 300 people (think multiple, large teams of business analysts, developers, database designers, architects and testers and a little hierarchy of IT project management) work towards a single outcome using a highly managed, strict waterfall approach over several years. It wasn't pretty, but we got there in the end, and I would not swear on the Bible that we could have done the same using Agile.
If Amazon is really using an Agile approach to coordinate hundreds of people working on a single project, I would love to read about it.
Glad to see disdain for "agile" is finally becoming mainstream. Now, it may be that someone writing rote software should follow a process, but for anyone doing anything actually innovative, you can't get into their head and tell them what process they must follow, whether waterfall or agile. It's like telling Michelangelo the process for sculpting.
Rather than teaching people processes, they should help foster better ways of actually thinking about the code itself. E.g., learning new programming languages and frameworks gives one a variety of perspectives to think about code from.
I think you can't effectively impose process, but I do think that good teams can happily decide what process they're going to follow. Which is how we got Extreme Programming in the first place.
> In many shops, Agile is just another way of micro-managing teams, sadly.
I think this is a big source of the complaints, yeah. If I had to narrow it to one specific mis-implementation of agile, it'd be having daily standup meetings turn into daily mini-performance-reviews with the boss. Iirc the Official Scrum Rules try to avoid that by decreeing that the scrum-master should be just another team member, and not a project lead or other kind of supervisor/boss, but afaict that recommendation isn't often followed.
Yeah, you nailed it. I have seen this done so poorly so many times that I simply tell people to rotate the ScrumMaster role. Otherwise they just make it into a manager.
A lot of times they want to keep the old Project Manager around, and that's very difficult to do. Project Managers and ScrumMasters are completely different animals. A few times as a coach I have literally thrown the PM out of the stand-ups and told them not to come back because they were turning it into a status meeting.
I'm not trying to plug coaching, but if you're trying Agile, you really need an independent outsider to help referee stuff like this. Maybe it's only the SM from the team down the hall, but stuff like broken standups can be impossible to fix without a little neutral nudging.
Exactly. It was amazing how quickly, at my last place, standups devolved from
...recounting verbally what one did the previous day...
...to marking one's progress on a whiteboard...
...to having to write down on an index card beforehand and turning it in to the IT director after the scrum meeting (in addition to the above).
With the boss (and sometimes, the CEO) in attendance, there was no chance of an honest verbal exchange about roadblocks etc.
Good metrics for measuring coding progress are so elusive, I can't imagine management NOT trying to mess with pure agile principles if tight deadlines and delivery dates are at stake.
Unfortunately, I got Scrum going at a small, chaotic, mismanaged consulting shop I worked for after using it at my previous job with reasonably positive results. Management loved it because it fed their natural micromanaging attitudes (e.g., employees are basically lazy children, will not work unless coerced, and must observe strict deadlines). I realized all too late that I was contributing to that.
These days they only do the daily stand-up, which is of no value to anyone except my former manager, since people talk directly to the manager and not to each other (this is one of many reasons why managers should not be in stand-ups). It became exactly what you said: a daily mini performance review.
I've since realized that the things that make Scrum successful are the same things that make a team successful, and that you can't necessarily connect a causal line between Scrum and team success if the team would have probably been successful anyway. It basically comes down to the team having a good rapport--regular communication between team members, the ability to seek advice without judgment, and the ability to communicate blocking issues in a timely way to the right people.
I think this kind of weird misinterpretation of the buzzwords happens because most people don't actually read anything about agile. They just hear other people say they need to be doing "daily stand-up meetings", and they interpret what they think that means, instead of bothering to find out.
I had a wonderful example of this behavioral pattern last week. I was called into my project's first "sprint planning meeting" where I was asked to write down for the next 8 sprints which features would be delivered when. Of course, since we were now "doing agile" I was allowed to be off by one or two features every sprint. But it still had to all fit into 8 sprints, because the project's deadline was by the end of the 8th sprint, and we needed to be sure we would deliver the full scope by then. As pepe le pew so aptly described: "le sigh".
"The second secret is that most teams, frankly, are not that good. So you need some kind of way [...]"
Agree. Let's not forget the HR challenge most organizations have today. It's hard to find, hire, develop and keep talented people in the company. I heard the other day: "every job seeker is a potential low performer". The basis of that is that most talented, highly qualified people is either managing teams or freelancing/consulting, and inevitably making interesting amounts of cash either way. So the chances of finding this profile on every Agile team is low.
In this scenario, Agile has its intrinsic value of providing teams members of various competence levels with some minimum rules of conduct, processes and tools. Maybe good teams will think they are all about bureaucracy, or they'll think it's something the management layers of the company can look and understand, like a burnout chart.
That's why I cut the second sentence of the quote above in the middle. Sometimes, as a manager, you need a "way", like the Japanese word for "Do", to keep discipline and make things moving forward.
For instance "No true Scotsman is born in Tuscany to Romanian parents and has never ventured west of Florence" isn't an instance of the fallacy because it truly does violate what it means to be a Scotsman.
Similarly, no true agile team would, say, agree to a single 6-month "sprint" at the end of which they will deliver the end product to a customer who has never seen it before, based only on a set of specs drawn up by an analyst.
And yet corporate shops that do similar absurd, not-within-the-common-idea-of-agile things will routinely label themselves as agile shops.
The hitchhikers guide to the galaxy gave the most succinct advice for how to fly. You fall down and miss the ground.
The upon reading this and in the absence of any native examples of flying animals the pelzo decided to directly follow this grand advice. After 100 billion attempts, over a billion broken bones, and hundreds of millions of deaths the great zaxuvex finally discovered that the correct way to do so is to reach orbital velocity above the atmosphere in a space ship that provides an internal atmosphere, comfortable temperature, shielding from harmful radiation, chemical rockets to slow down, AND most importantly a heat shield and parachute for reentry. Two weeks later a vising dignitary introduced the idea for wings which was was deemed heretical and started the 42 great war.
PS: If your 'training courses' don't provide enough information to actually implement your process in the real world then it's practicably worthless.
Isn't it ironic that by simply capitalizing the word 'agile' you can change its meaning to describe something that's the very antithesis of agility?
If a methodology is so brittle that any diversion from the straight and narrow (a) means you're no longer using that methodology and (b) ostensibly (or actually) invites disaster, then that is a very, very rigid methodology.
> The second secret is that most teams, frankly, are not that good.
It's so important to remember this, as well as the corollary: I'm not that good myself. The article included this line:
> If you need to follow a prescribed process in order to be in any way an effective coder then you are mediocre at best and so is your work and your project.
I'm not sure if I agree with the statement itself, although I certainly think -- and know no one who doesn't think -- that the quality of your staff is more important than your process. But I'm sure I disagree with the way this statement is used to dismiss the idea of a prescribed process. One of the main goals of such a process is to get as good work as possible out of mediocre staff. Because there aren't enough high quality staff to go around. You won't get great work out of mediocre workers, but if you can get pretty bad work instead of total garbage, in many environments that's a big win. (In the startup environment that is hn's focus, pretty bad might be just as bad as total garbage.)
> I certainly think [...] that the quality of your staff is more important than your process.
I don't think those are independent variables.
In my experience, one of the properties of a good development team is that if a project lacks clear guidelines so the developers can work effectively together, the team will naturally tend to self-organise. They will establish enough ground rules among themselves to collaborate effectively, and they will adopt or create any tools necessary to support that basic process. A good team that does already have a decent process and competent leadership will still naturally review and update that process over time based on their experiences using it.
This also explains why one or two substandard developers on an otherwise competent team are so damaging: they are sufficient to disrupt any reasonable attempt at self-organisation, and the team now requires strong leadership rather than merely collaboration among peers. Unless the leader is capable of both instilling discpline in the rogue elements and simultaneously accepting and promoting good ideas from the remainder of the group, which is a very difficult thing to do even for an experienced manager with good intentions, the result will still be lower performance than a team without the negative elements.
I objected to the same statement but for slightly different reasons. It works when your project is of a size that can be done by a fairly small team that can communicate intimately and instantaneously at all times. We're talking 1 -5 people max. But what happens when there are 30 developers, 50 developers, or 100 developers. Without some overarching system of coordination these people will just be flailing around, constantly treading on each other, breaking each others stuff. Somehow you have to break it down and give people a way to understand what everyone else is doing, and that turns into "process". You can do it any way you like, but what ever you come up with is still going to be that evil word "process".
The problem is that said "process" then gets deployed in a cargo cult manner to any project regardless of size, divisibility, who is managing it working on it, etc.
I have nothing to say but to offer words of encouragement in your effort to write e-books to undo some of the damage.
It breaks my heart to see something that emerged from technical people to make their lives better and make them more effective be co-opted to the point where it's scorned by the same technical people who have the most to gain.
> [Obligatory plug and disclaimer: Agile professional
You sell process for a living. You claim, "After all, it's not like you can have no process at all. Whatever you're doing already is a process." When you said that, you accidentally got something right.
Programming is a process. It's the only one that fucking matters if you're a shop that sells software. All the other process comes from snake oil salesmen as yourself, or pointy haired bosses who don't know a fucking thing about programming.
Programming. Either learn and do it, or get the fuck out of its way.
Spoken like a true cowboy programmer who can't collaborate his way out of a wet paper bag and blames process when a team he's part of fails because of that...
I hate that term cowboy programmer. Very effective rhetoric, though. Let's take the archetypical example of people who get stuff done without our process, and call them cowboys! Now, anytime someone points out a dumbass process-driven decision that could have been avoided by using our brains, we can just disparage them as a cowboy.
Note, we're using small-a agile at my shop and it's working well, but that's because of good people, not because of blind adherence to a process and eschewing individual thought.
It's ridiculously easy to get stuff done without any process when you work on your own and the projects are small enough for that. You don't even need to be very good to succeed in such a scenario.
Cowboy programmers are those who extrapolate from that and believe that teams efforts can work the same way, and who think that process causes only inefficiencies rather than preventing much larger ones.
But for a team to work effectively, you need structure, you need process. Even if the team members are very good, they can get completely lost without process. If they are not completely antisocial, they will come up with some sort of process themselves, but it's not automatically going to be a very effective one (and that has nothing to do with their programming skills). The point of agile is to try and minimize process while still enabling teams to work effectively. And it has absolutely nothing to do with "eschewing individual thought". Quite exactly the opposite, really. But of course there's no concept that can can't be perverted into its opposite by a sufficiently point-haired idiot.
Yeah man, I get that. I said that I'm at a small-a agile shop and it's working well for us.
That doesn't change the fact that the only time I've seen the phrase 'cowboy programmer' used, it's as a preemptively ad hominem argument that anything besides Capital A Agile As Written On Stone Tablets can't possibly work. And if it does, it's cause they're cowboys.
As a counterpoint, demanding something is deployed every 2 weeks, whether it's a 1-week task or a 3-week task, leads to plenty of cowboyism right within the process.
Who demands 3 week tasks launch in 2 weeks? And Agile dev cycles delivers whatever is ready every two weeks, instead of shipping everything on an arbitrary deadline.
Funny thing is, both this guy and the author of the TFA touch the same problem in passing: regardless of the process, there are too many incompetent people plaguing our industry. Instead of addressing the problem, managers usually exacerbate it by trying to "fix it" with a "correct process".
No process in the universe is going to turn the Infinite Monkey Theorem [1] into solid reality.
Yet it never occurs to them when dividing people into groups that maybe they're the incompetent ones...
Who's more competent? The person using Agile successfully and getting things done, the guy using no specific development process and getting things done, or the guy writing long blog posts about why he doesn't care for a development process he doesn't understand and doesn't need to participate in if he doesn't enjoy it?
I'm currently on two missions. In one of these, we kick ass. Deliver on schedule or even in advance. It's scrum, we simply work as a team of four and ship, ship, ship, ship, feature after feature and fix after fix. We have our hands free, management is trusting our cell with the strategic future of their Tech tree, and we deliver.
In the other mission, I am not making this up, I spend 1hour/day on poor excuses of stand up meetings, 3 hours/day in various meetings, 1hour/day on various administrative tasks and 3 hours/day preparing powerpoint plannings for the meetings of the next day, and I'm a dev. I work with equally awesome devs in either mission, but in this one the customer is absolutely crazy inefficient and a control freak, and the best example of what NOT to do with scrum I've ever seen.
So who is more competent? The me that's killing it with scrum in the first mission or the me that can't do anything because of scrum in the second mission?
The first you. The second you needs to run far away from what he's doing. :)
I was in that situation before -- bad agile is BAD. I was on a team of 4 developers with literally 8 project managers. I know how that daily 1 hour crappy stand up goes. I also know how it feels to have deliverables for half the project managers in any given week, and what it's like to get constantly interrupted by 8 different people during your minimal development time to answer non-urgent questions.
But you know what? Not everywhere is like that and you usually have a choice to find something else. You're obviously competent enough to find something else; you already have two things going on.
If they aren't doing it right and they aren't listening and you aren't happy, GTFO.
I enjoy programming with other programmers. But I don't collaborate well with people who don't know a fucking thing about programming who try to put process on top of the process of programming.
Cowboy is a complement. It doesn't mean "lone ranger" or "asshole", it means lazy, hard worker who takes the shortest path to greatest output. So, yup. Cowboy here.
I don't collaborate well with people who don't know a fucking thing about programming who try to put process on top of the process of programming.
Fair enough. But what about someone who does know a lot about programming, who runs his team using agile methods. Could you collaborate well with that person?
HN's algorighm thinks I'm a spammer and wouldn't let me reply, but really, itsnotme.
I could collaborate well with that person. If I like the product and the people enough I'll work on a team that uses "Agile". But I do it knowing that process is not communication, and that standups are a fucking joke.
No but communication is part of any effective process. Standup is just having everyone say what they accomplished yesterday and what they're planning on doing today. If you're on a team with more than three or four people, it's a useful way to get a quick summary of where everyone's at. Some people are good at letting other people know when they're stuck, but some aren't, and if you've got the latter on your team standup is a good catchall. Most teams, whether they're in software, sales, or sports, have some variation of the standup. Of all the scrum paractices it seems like it should be the least controversial.
No amount of process can fix bad communication. Standups are an attempt at that. Teams that have good communication succeed more, and standups aren't integral to their success. Teams with bad communication fail more, and standups can't help them.
Sales teams and sports teams don't work as a comparison, they aren't building something (sales is a particularly bad example, most sales team members are in direct competition with each other). I'm trying to imagine standups on a building site.
Tom: "Yesterday I.."
Everyone cuts Tom off and talks over each other: "No shit, Tom. You're gonna work today, too." "That insulation isn't going to unroll itself!" "Tell us, dear Tom, will you be screwing the sheetrock in with a DeWalt or a Hitachi?"
Good teams communicate while they build, and know where each other are.
Sure it can. If I've got a team member who can't be bothered to let people know what he's doing on his own, I can hold a standup every morning to make sure the information gets communicated to the team. If he lies every day about what he does, the longest he can go is one sprint before his work gets verified.
I'm trying to imagine standups on a building site.
I don't know, I think it goes something like this:
"Hey Tom, how far did you guys get on the roofing yesterday?"
"Only about a quarter of the way. Some dipshit ordered the wrong shingles."
"Ok, Bob will you get on that today so Tom and Dick can finish the roofing? How about you Fred, where are you at on the sheetrock?"
"Living room and kitchen are done, I should finish the bedrooms today..."
etc. Pretty much what actually happens at a construction site if the builder is competent.
I don't know what kind of standups you've been in but they sound like they were run by someone with no training or experience in the process.
Good teams communicate while they build, and know where each other are.
How do they communicate? What if Tom doesn't know what Dick's doing and Dick doesn't think to tell him because he doesn't know Tom needs to know? This doesn't magically happen even if everyone is individually a star at what they do.
"Sure it can... If he lies every day about what he does, the longest he can go is one sprint before his work gets verified."
That's not fixing bad communication. That's wrapping it up in process, and calling it "good enough". The team communication, and probably its morale and output, still sucks.
"etc. Pretty much what actually happens at a construction site"
That communication happens continuously, throughout the day. If isn't happening continuously, throwing a standup at it ain't gonna fix it.
That's wrapping it up in process, and calling it "good enough".
"Good enough" is certainly better than "not good enough". And it doesn't have anything to do with morale. The team's morale might be sky-high and they can still make mistakes because person A didn't know what person B was doing in a timely fashion. People have a tendency to overestimate their skills in general, and communication is no exception. Standups are just a form of insurance and cost almost nothing. Even if your team communicates like they're telepathic, 15 minutes a day in standup isn't going to hurt them.
If isn't happening continuously, throwing a standup at it ain't gonna fix it.
If I'm managing 5 job sites, I'm not there continuously. It might not be perfect, but once a day sure beats nothing. Most of the time when someone is doing something wrong, they don't know they're doing something wrong. No one is going to call me up and say, "Hey, I'm about to do something stupid and wanted to run it by you."
"Good enough" is certainly better than "not good enough".
In the scenario that you described, someone went from not communicating at all to lying every day, and getting "caught" at the end of a sprint. Standups didn't fix anything there. Also, there is a world of difference between calling something good enough, and it being good enough.
If I'm managing 5 job sites ... once a day sure beats nothing
If, as a manager of 5 teams, you feel like you need a direct, daily report of what every person on every team does, your life is going to be hard. If you attempt to control those teams by adding more process, you will interfere with the productivity of your good teams, and your bad teams will continue to suck.
The scenario isn't about the lying, I was just anticipating the argument that the standup wouldn't improve communication if someone wanted to subvert it. The basic situation---a teammate who isn't communicative---can only be 'fixed' by getting him to communicate. You might argue that I fire him and spend weeks looking for someone who codes as well, or alternatively hire a 'communication coach' to help him with his personal problem (I'm not sure what solutions you're offering) but I prefer the vastly simpler option of just having a standup. Which also solves the problem of good communicators who don't realize they've got something worthwhile to communicate.
We don't do standups to fix broken team communication. We do standups to augment ordinary team communication. If you've got an uncommunicative team member, that's not unusual. If you don't have an uncommunicative team member, now that's unusual.
If, as a manager of 5 teams, you feel like you need a direct, daily report of what every person on every team does, your life is going to be hard.
I may or may not need the report, but I want to make sure that every member of every team knows what's going on on their team. That's not adding process for process' sake, it's just ensuring communication rather than leaving it to chance or some utopian vision of human nature. A 15-minute standup isn't going to hurt a good team's productivity, while it could greatly improve the productivity of a mediocre team.
It can send a powerful message that the team is not responsible for producing the software.
Could you elaborate? I have a hard time imagining how spending a few minutes describing your progress and your immediate plans could possibly give you the feeling that you're no longer responsible for what you're working on. It seems to me that not ever being asked about what you're doing would powerfully convey that message.
Daily stand ups would be stupid in my current project.
Professionals should have a broad responsibility for the end result and appropriate freedom over the approach taken.
Externally imposed process creates a situation where professions are responsible for executing that process.
People ask dumb questions like "Is this Agile?". Who cares? The correct question is "Is this what makes sense to get the right outcome?"
My personal experience is daily stand ups are used as a tool to mandate a start time. This time was been inconvenient for staff with young children to drop at school. I remember hearing the sigh or relief from a person when they were dropped.
A 15-minute standup isn't going to hurt a good team's productivity, while it could greatly improve the productivity of a mediocre team.
It takes a lot to noticeably hurt a good team's productivity. But it's mean, just fucking mean, to punish a good, productive, communicative team with more process because you also manage crappy teams.
[I]t's mean, just fucking mean, to punish a good, productive, communicative team...
I didn't say standups don't benefit good teams; quite the opposite. You've missed (or ignored) my point that even good communicators miss conveying important information sometimes. And if the team is really so naturally communicative, they're hardly going to view a standup as punishment. Again, I have to really wonder what kind of standups you've been subjected to that would trigger your revulsion. It's like being violently opposed to eating with a fork. Technically, no, you don't need one but it often helps. Either way, it's nothing to get worked up over.
Oh, I agree. My language was for emphasis, not representative of an emotional state. It's interesting to talk to someone who thinks that the same process can save a bad team and make a good team better. That though had certainly never occurred to me.
Well, if you mean "Programming is a process" to include techniques of effective collaboration, then agile is part of programming.
Claiming that people who try to define methods to do that better "don't know a fucking thing about programming" is A) a non-sequitur and B) indicates that you don't collaborate well, period.
No, Cowboy is not a compliment. It means a hard worker who takes the shortest path to the greatest output of code that doesn't fulfill requirements and which nobody else can maintain.
You say "techniques of effective collaboration", I say "talking to people". If you need guidelines for that, that's cool.
The thing that suggests to me that you and I wouldn't work well together is that you make broad value judgements based on twisted interpretation and extrapolation of these comment threads.
Interesting. So how do you build things that take thousands of programming hours to complete? What about projects involving complex rulesets requiring legal input and/or HR input?
Programming is a means to an end. The end is solving some problem. For many problems programming is a small part of the overall solution.
I meant to answer your original question, but HN decided I couldn't post any more under "itsnotme".
"So how do you build things that take thousands of programming hours to complete?"
By programming for thousands of hours. Through the process of programming, I'd try things out, use what works, dump what doesn't, write tests first where it makes sense, last where that makes sense, not at all where that makes sense, buy hardware where it will save me time and money or work better, interface with it by writing code, step back, look at some mistakes, rip them out, replace them with nothing if I can, or less bad mistakes if I can't replace them with nothing.
You know, programming.
If in addition to my process of programming someone wants me to tolerate some other meddlesome process, well, it depends how meddlesome that process is. If they want me to use a particular IDE, they can fuck right off. If they mandate I write code in a particular order, they can fuck right off.
On the other hand, if they are awesome people with awesome product, I'll engage in standups (or whatever), and I'll write my cards and move them along (or whatever), and burn down my time (or whatever). But I won't pretend that it part of the process of programming, or that it's speeding things up.
Well, the Agile approach is to break them down into smaller things that can be accomplished in under 1,000 hours.
"There's nothing that can be accomplished in a single chunk taking more than a 1,000 hours that will produce a system with a manageable and maintainable level of complexity unless it is broken into smaller, well-defined and loosely-coupled components," perhaps.
Lots of meaningful things take thousands of programmer hours. Projects where those thousands of hours are meticulously planned for by non-programmers (whether by using "Agile" or any other process that sits on top of the process of programming) are fucking doomed from the jump.
The software that keeps all modern airliners in the sky is written by companies that have lots and lots of process. Do you think those software projects are doomed from the jump just because there's a lot of process involved?
You said "...Agile, or any other process...", so you're explicitly not limiting your statement to Agile methodologies.
Do you think _slower_ (as in taking longer to develop) software is the only outcome of the processes that avionics software companies use? Or do you think they are wrong or would be better off not using any of these processes?
The companies have a lot of process, but how do you know how the software was written? Usually I see things like a huge project that is failing while the initial prototype that works gets upgraded to production...which has often been produced by cowboys.
However, I don't work in the airline industry. I do work somewhere where one errant line of code can destroy productivity around the entire globe, brick computers, and really piss people off, and it's low process, cause the process guys can't ever get anything shipping.
As for Agile, I would suggest reading the Mythical Man Month circa 1995, the final chapter covers iterative development and makes a lot of sense, imho. Not sure why we need people with certificates selling snake oil where common sense works just fine.
Processes implemented to save lives are the product of death, and the process is put there to slow things down. Ask your friendly agile consultant how much their process is going to slow down your company.
They will tell you that Agile will make you faster. They might even be right, if it's replacing a bigger, worse process that already exists on top of the process of software development. But if it's adding a layer, it's going to slow you down and get in your way.
Any professional has some process, explicit or implicit: it's the observable regularities in the way they work.
Any team is going to have some explicit process, because there's no way people will converge on the maximally effective way of working together without some discussion.
So as far as I can tell, what you're saying is that it's impossible to take any process ideas from an external source, because every team is already maximally efficient. That doesn't make much sense to me.
In our case, the process we've converged on is the most efficient one we can find. But we're always looking for ways to make it more efficient.
So as far as I can tell, what you're saying is that it's impossible to take any process ideas from an external source, because every team is already maximally efficient.
Nope. I'm saying that the process of programming is what we should focus on getting better at. We should take improvements from wherever we can find them. Getting better at the process of Agile or Waterfall or Scrum or XP or Kanban or whatever else is dreamed up is tangential to our progress as programmers.
I agree that nobody should focus solely on getting better at, say, Agile. I also agree we should work hard at getting better at programming.
However, I don't think that focusing on programming alone is sufficient. A lot of projects fail for reasons other than bad coding, and almost no project exists where programming is the end purpose. You can do all kinds of great programming, but if you make the wrong thing, you're screwed. Ditto if you have a bunch of individuals doing great programming that doesn't add up to a functioning project.
I also think you're creating a bit of a false dichotomy. When I first tried XP, it wasn't for the sake of doing XP. It was because I wanted to improve how we made things for our users. So exploring XP wasn't tangential to better programming; for me it was closely related, in that programming was a big part of making.
Just differentiate between what Agile is and how people are pushing Agile on you. Different things entirely.
Agile is best practices around iterative and incremental development. Period. Yes, there's a manifesto and there's Scrum and all sorts of other things, but at the end of the day Agile is a marketing term. A place to go find out what people are trying in order to get better.
Two secrets here. One, team success is about 90% dependent on who is on the team. Good teams do well. Bad teams do poorly. But if you're going to have more than one team, you need some way to measure and talk about what they are doing, so you have to have something.
Agile -- when done correctly -- is just the minimum amount of something you need to get your work done. It is the minimum amount of process. After all, it's not like you can have no process at all. Whatever you're doing already is a process.
I've seen hundreds of teams struggle with Agile, and to me the problem is that we do a really bad job of balancing the difference between "here's a best practice that worked awesome for about 80% of the teams that tried it" and "These are the rules. You must do this." In many shops, Agile is just another way of micro-managing teams, sadly.
I can't help that. I also can't help the fact that lots of folks are out to make a buck on certifications and such. I don't think that makes Agile bad. Heck, I'm not even against certifications, although I'd never get one. I just think we take things too far.
I like the idea of having a marketing blurb "Agile" where I can go to find out what kinds of new stuff is being done. It helps me pick better reading material.
The second secret is that most teams, frankly, are not that good. So you need some kind of way to demonstrate that as well. Making things incrementally and in small iterations lets you see how bad teams are early on. You fail early and often. Then at least you have a bit of a shot at trying to help.
But believe me, I feel your pain. Sounds to me like you are a lot more upset at modern marketing and management attitudes than Agile. Remember that we technical folks have a way of over-doing whatever we get into, as I was writing in my blog before your post appeared! (It's still very strange to me watching HN how the same topic comes from multiple writers at the same time) http://tiny-giant-books.com/blog/agile-means-stop-focusing-o...