Software Psychology

what happens to the code without thought

2011-09-26T15:04:00.000-04:00

I ran into this wonderful piece of java code at work. Just had to share this with everyone.
This code base was modified over last 4 years by at least 50 different developers with no TDD. Look at what the code ends up being in a long run.

// given that record and id are Strings
if (record != null && ((id != null && id.length() == 20) || (id == null || id.length() != 20))) {
// do something
}

This if condition is essentially the same as

// given that record and id are Strings
if (record != null && true) {
// do something
}

laws of software defects

2011-09-13T20:47:00.001-04:00

Just for fun but feels so true on some days

Remember these laws of thermodynamics !
Energy can not be created nor destroyed
Energy can be changed from one form to another
In an isolated system the total energy remains the same

Now replace the word "energy" by "Defects" and you get the laws of software defects !
Defects can not be created nor destroyed
Defects can be changed from one form to another
In an isolated system the total defects remains the same

heh !

Organizing unit tests

2011-08-24T10:01:00.000-04:00

I have been using a different approach to organizing the unit tests and this have turned out to be very effective. Worth sharing.

So if we have a class that looks like:
public class AccountService {
public void getAllAccounts() { ... }
public void createAccount() { ... }
public void removeAccount() { ... }
}

For such a class we would traditionally have a unit test class called AccountServiceTest and all the test cases for all methods mentioned above. We would also have a bunch of global variables in the Test class and a setup (@Before) method.

My different approach involves creating a unit test class for each method rather than just one test class. So now we would have three classes:
public class AccountService_getAllAccounts_Test {}
public class AccountService_createAccount_Test {}
public class AccountService_removeAccount_Test {}

Generalizing it the unit test class name should be:
<<class under test>>_<<method under test>>_Test.java

Each class would have test cases for only the method referenced in the file name.

So how is this approach effective?

SRP - Single Responsibility Principle - is restored
In the traditional way the SRP principle is violated with the setup method as it is now common to all the test cases for all methods for the class under test. The setup has responsibilities to setup the objects and data for all the test methods for all methods in a class. Even the Test class has multiple responsibilities to test all methods in there. With the new approach every test class has responsibility to test only one single method under test.

"Look and Feel" of unit test cases
This is more of a perception thing but it works. Say if every method under test can have 5 unit test cases, in the above example having a single test case class means we already have 15 methods in there. When a developer looks at a single class with that had large number of methods with a single setup method that looked cryptic and they were less likely to update any unit test cases or even add more tests to them. When the classes were separated the "look and feel" of the test cases improved developers are now willing to and able to extend existing code base by adding new test cases.

What are the side effects of this approach?

Popular plugins like moreunit do not work anymore
Proliferation of Test classes may be something some developers may not like at first.

Let me know what you think !!!

TDD and Technical Debt

2011-07-25T13:16:00.001-04:00

So whats the relation between Test Driven Development and Technical Debt?

Technical Debt is a term inspired from Financial Debt that people fall into
People fall into Financial Debt when the spending to earning ratio gets out of order, spending more than money earned.
In most cases spending more than limit is because the concept of monthly finance budgets is lacking or not enforced.
The monthly budget that is setup at the start of the month or year is like writing unit test case. Then living the month by the budget is like writing exactly the code needed for the test to pass.

Hey -- so just like in real lives lack of Test Driven Development or not enforcing it does cause more Technical Debt.

moving to github

2011-07-06T08:49:00.001-04:00

I am moving all my code from presentations / pet projects to github -- http://github.com/daveayan
So far I have migrated 3 projects but more are coming. So far this is what I have:

- Google App Engine Presentation I did in April 2010
- Maven in Polyglot space presentation from Columbus Polyglot user group (03/2011) and QSI Tech Lunch (05/2011)
- My pet project rjson that is a core piece for my legacy code testing project (coming soon on github)

Need to start blogging again

2011-07-05T11:47:00.001-04:00

It has been so long. I need to start blogging again. More stuff coming here.

Why do we need unit tests?

2010-02-13T10:49:00.000-05:00

Alright ... the TDD & CI analogy is good. Makes sense the way it is described. But how would you justify spending time and money on writing unit tests when you are under tight budget schedules. Why would you even need unit testing when we have a v good QA and Testing team and as good processes in place? This is an often asked question. Here is what I think.

Unit tests are not meant to rule out the need for QA / Testing. Unit testing is not even meant to complement QA / Testing, although they would help significantly reduce the burden on QA team. They are there for the developers. Let me give you an example - but first, put on your developer caps. I have a BaseDao class that has a query method on it like this:

public List<Object> query(String sql)

The method takes a SQL query as input and returns back a list of objects mapped from the query result set. The more important part is that the method returns a null back if no records were found in database. Now I have atleast 20 DAO's that extend this BaseDao and use this query method atleast 100 times.

Now in all good mind and after lot of consideration it was decided that the query method in the BaseDao will have to return a empty list (new ArrayList<Object>()) back instead of null if no records were found in database. Sounds like a easy change, but now what happens to all the 100 methods out there using this method? What happens to all those if conditions with null checks after the query method? For example: One of the DAO's has these lines of code that will break for sure.

List<Object> l = query("SELECT * FROM PERSON")
if(l == null)
return false;
else
return true;

How does a QA / Testing team plan to test this change in the system? This would become even interesting if there is a DAO method among these 100 that will be called only rarely in certain special data combinations only. Does the testing / QA team know what that method is and when is it called? The real idea with unit testing is to help the developers. If we have all these 20 DAO's and 100 methods all tested well it will be easy to find where the code would break when such a change takes place.

Another important point is that these lines of code continue to live as long as the system lives. The QA / Testing team moves on and the next time there is a new feature development / major bug fix the same QA team may not be there even. Even the same developers wont be there working on the system. Now consider this scenario - as a new developer you are put on such a project and you have to fix a defect. This method you need to fix is being used by 10 different classes from web and batch projects. How can you update this method confidently without breaking the functionality offered by all the other 10 classes? And you need to think of this even before the next QA / Testing team comes in to start testing the system.

Of course for developers, just like any other technology, there is a learning curve and it can be difficult. Specially with tight deadlines and with agile processes you may not get a lot of time to learn TDD. But who said life would be easy. Once the learning curve is conquered things become easy.

TDD and CI - An Analogy

2010-01-26T22:50:00.002-05:00

More often unit tests are considered to be time consumers, lines of code that add no real value to the core working of the system. Business Requirements have a first class place in the project as well as the "real" code but unit tests do not get such a place as often as they deserve.

Lately in my work I have to do lot of convincing and mentoring developers on TDD & CI and encourage them to write more unit tests. To convince and to justify the need for TDD and more unit tests and CI I came up with this analogy.

In my analogy consider the lines of "real" code that do business logic or UI logic or data access logic as the people in society. As there are good and bad elements of the society so are good and bad code modules and lines of code. As people we tend to take different roles and actions in day to day life. We drive, we go out to eat, we take a walk in park, we shop. So do these lines of code in the system take different contexts. The same method in a DAO or Service class may be used by a Web frontend or by a batch job. Some of the domain objects may be used by the UI in a certain way and persisted in database or other remote stores.

Now lets talk about the business requirements. To keep people behave well and in sanity laws are formed. Some are explicit - such as the driving laws and some are implicit - well, we don't toss burgers in a restaurants, its bad manners. The business requirements for a system are such laws that the code needs to adhere to. The code must satisfy the requirements.

Here is the fun part - In the society we have cops and agents that enforce these laws on the society. What do we have in software applications that enforce these requirements? This is where TDD, CI and even BDD comes in. It enforces that our code behaves well. If a line of code breaks a unit test it means it violated the law (the business requirements) in some way and this line of code must of brought to justice. Without unit tests we can only assume that the lines of code will behave, but what happens when a change in requirement comes up and we change the code without enforcing the new and old laws?

There can be an argument that functional testing is a good way to enforce the requirements. This can be true to a certain extent. The part of this that worries me are two things. One is that functional testing is performed at specific milestones in the project cycle. The lines of code live on after this milestone has passed and often times these lines of code change without anymore functional testing. Will it help if the cops come in a city only once a month and assumes that things are well in the city rest of the days? The second trouble is that it is not guaranteed that functional testing will cover all lines of code. Well good functional tests would at some point cover lines of code but where is the metric?

The point to be taken here is that write unit tests - these are the cops of your code. If some other developer comes and changes these lines of code and unknowingly causes them to be bad elements of the software society the cops (unit tests) will catch em.

Long Time !!!

2010-01-20T14:02:00.000-05:00

It has been quite a busy time the last 4 months. My little baby girl was born on Jan 11 and that marks a major milestone in the life. As the sleepless nights settle I would start blogging more.

whoNeedsArchitect

2009-09-25T16:13:00.002-04:00

http://martinfowler.com/ieeeSoftware/whoNeedsArchitect.pdf

My fav quotes from this article --- this is what I often bring up in our lunch conversations with fellow QSI'ers!

I think that one of an architect’s most important tasks is to remove architecture by finding ways to eliminate
irreversibility in software designs

Software is not limited by physics, like buildings are. It is limited by imagination, by design, by organization. In
short, it is limited by properties of people, not by properties of the world. “We have met the enemy, and he is us.”

“Architecture is the decisions that you wish you could get right early in a project.” Why do people feel the
need to get some things right early in the project? The answer, of course, is because
they perceive those things as hard to change. So you might end up defining
architecture as “things that people perceive as hard to change.”

Agile Methodology - Similiar to Creating Shareholder value

2009-09-24T15:23:00.001-04:00

I have been involved in several projects now that had used Agile methodology for project management or a derivative of this method. In these projects I have assumed several roles - Developer, Team Lead, Designer, Architect and BA in a few cases.

Something I have realized is Agile method is similar to the notion in public companies - "creating shareholder value". Agile software development has iterations - 2 to 4 weeks long and then we provide improved version of software / product to the client. The client also prioritizes the new features to be developed and / or defects to be fixed. This is similar to what public companies do - a quarter can be considered as an iteration at the end of which the money matters are shared with the shareholders and a new iteration begins. The profits the company makes is the product the shareholders are concerned with. The shareholders directly or indirectly have a say in how the company would improve profits or make new money (similar to client deciding on new features).

This idea of shareholders value creation is dangerous as well. The company tends to focus on short term gains and loose sight on long term plans. A similar thing may happen in Agile projects where lack of proper documentation, design and architecture due to short term goals (short iterations) may plague a system in long run. Of course there are exceptions - there are well run Agile projects and there are well run public companies that make profits.

Just a thought !

ERROR in Eclipse - Access restriction: Class is not accessible due to restriction on required library

2009-09-23T10:12:00.002-04:00

I get this error in Eclipse when I use the @PostConstruct and / or @PreDestroy annotations. To get rid of this error in Eclipse change the "Forbidden reference (access rules)" level to Warning or Ignore.

B2B - Back to Basics Series

2009-09-17T11:26:00.001-04:00

I have conducted a dozen or more interviews now in my career so far for different roles on projects - Developer, Sr. Developer and Team Lead. I make it a point to ask about OOAD and Design patterns in all these interviews. However in many cases than I would have expected the candidate was not able to explain the a OOAD principle (like SRP, DRY etc) or design pattern. I think understanding of these core concepts is much more important than the knowledge every API.

I am going to start writing this B2B series here on my blog. In this series I will write about the fundamentals, basics of Computer Science, Software Engineering, OOAD etc that we learned many years back in school and still use in our professional life. However lot of times we forget lot of these things while dealing with the day to day work. Hopefully someone other than me finds this helpful

Happy Coding !

Apache's 11 technologies that changed computing in last 10 years

2009-09-11T16:25:00.002-04:00

I found a nice article on eWeek.com about the 11 technologies that changed computing over the last decade and to technologies that would probably be dominant in the next decade.
http://www.eweek.com/c/a/Application-Development/11-Apache-Technologies-that-Have-Changed-Computing-in-the-Last-10-Years-469693/

Moved to blogspot

2009-09-09T18:16:00.003-04:00

I have moved my blog to from www.daveayan.com/blog to this space.
Blogspot has a lot more features obviously than the out of box liferay blog. Please update the book marks and add some comments or provide your reactions.
I will be shutting down my site (http://www.daveayan.com) in a couple of months as well and try to come up with another, better looking site.
Thanks

TestNG + EasyMock - OutOfMemoryException

2009-08-06T16:02:00.000-04:00

As much as I liked TestNG and Easy mock a few months back I am starting to hate it as well. We have a suite of around 500 unit tests (maybe just a bit less than that) and we get OutOfMemoryException. We get these running locally and on running on Hudson CI. On further research I found these two links that explain this problem. BTW we run with JAVAOPTS -Xmx128m. Increasing the memory does not help, I dont know why.
EasyMock OutOfMemoryException

TestNG OutOfMemoryException

Too bad the TestNG defect is open for almost 10 months and not fixed.

For now we segmented the unit tests into two groups - unit1 and unit2. This is working well, however I anticipate issues now with the corbertura reports that we want out of CI. Also, next time we hit the wall on unit2 we will have to start a new group - unit3 ... Arrrr !

Up Next - Spring Certification

2009-07-23T16:52:00.000-04:00

I would start preparing for Spring Certification starting this week. I would like to try and get the exam done in next three months.
It is an extensive exam with no proper study notes or mock exams and need 76% to pass. This will be fun.
Go Spring !

Liferay book by Jonas Yuan

2009-07-15T03:26:00.000-04:00

I got an oppurtunity to review the new book from Packt publishing - Liferay Portal 5.2 Systems Development by Jonas Yuan.
A few words about this book.
This is a nice begineer to intermediate to semi-expert level book. The book begins with a nice overview of the liferay portal system and WCM and introduction to JSR 286 portlets. The author then gradually raises the bar with more details on how to develop portlets, manage pages, and customizing the portal.

My favorite chapters of the book were the chapters on Social Office, Staging & Publishing and using common api.
Actually the social office chapter was quite a surprise. The author has given good treatment to the topics explaining them in sufficiant detail for a beginner and at the same time making it relevant to the advanced users. The author has also explained the events and hooks very well.

One more thing that I liked about the book was that the author explains how the code works out of the box and then how to customize it. I liked the "Whats happening?" sections of the book

One of the things I do not like is that there are a few sections in the book like "Using SCORM" that do not talk about the topic in detail. This raised the expectations but then quickly brought them down looking at just a few lines of explanation. However there were not a whole lot of such topics.

All in all I would recommend this book to anyone who wants to get started with Liferay and get a good handle on the system.

Hibernate - Annotations vs XML

2009-07-02T22:10:00.000-04:00

Recently I was involved in a healthy discussion at my work on using Annotations for Entities. We have a Java web application with Spring MVC, Hibernate and Entities with hbm.xml files. Entities with XML work well however there is an interest in the team to use annotations. Here are my thoughts - pros and cons of each approach. I have used both approaches, however I like using hbm xml files over annotations for entities. That said I do like the use of annotations in Spring MVC, TestNG, or on service layers to mark transactions.
Why stick to hbm.xml files?

Well, if we think back to days before hibernate, when we used JDBC we had the problem where the SQL code was embedded in Java classes, thus causing many maintainability problems. With Hibernate xml files we got a clean way to keep them seperate - the entity pojo's were free of the sql clutter. With Annotations in Entites we are going back to that same model !!!
Entities are cross cutting code - they are used in all layers, starting from DB all the way to JSP in many cases. Cross cutting logic should not have code that looks more biased to a certain layer.
If you are working in a team that is split into Backend Dev team and frontend dev team then the front end team does not need to see all those annotations. Well what would be the argument if frontend team wants to add their own View specific annotations to entities for ease of use?
If you have a code base that takes long time to compile and build (fortunately for us it takes a few seconds), having an hbm xml file is better. No need to recompile.
If you have to migrate from xml to annotations on an existing ongoing project what is cost justification to use annotations?

Why move to Annotations?

Well, as my collegue at work explains, it improves maintainability by having only one file
As Paul Mazak would say, the mantra is "less code characters", helps in long run.
Annotations are cool ! Learn some new stuff

Well in the end it is a religious debate and fun as well. I will stick to hbm xml so far, playing with Annotations as I get an oppurtunity for new development.

PMP Exam

2009-06-30T04:18:00.000-04:00

Passed my PMP exam today finally - what a ride it was.

It was tough, the real exam questions were very different from the mock exams I had been preparing with, in terms of the style, definitions used etc. There were a few new terms as well, that I never read of before.

Luckily the time was more than enough. I was able to run through all the 200 questions in 2 hours, 30 mins. For some of the questions I did take a lot of time to think as well. At the end I had around 25 questions marked for review and 4 not answered. The next 90 mins were enough to review and complete them all. I still had 23 mins remaining before I clicked "End Exam".

After I clicked "End Exam" I was asked for an optional survey. I wanted to fill the survey in good faith but who wants to wait 10 more mins to see a pass / fail score? They should have asked for the survey after showing the score.

Anyways, after the exam we went and watched a movie and had nice dinner at Chipotle. End of a saga and begining of a new one - whats next?

I want to take a bit of a break and then go for Sun Certified Java Web Services Developer or Sun Certified Enterprise Architect. Not sure yet which one to pick. And of course not to forget - keep an eye on those PDU's needed to keep PMP credential.

Caio

No getters and setters

2009-06-11T23:33:00.000-04:00

To design good software working with Domain language and making it part of Object's is very important. One way to get this, I think is to get away from the "get" and "set" polluted code.

consider this code:

public class Person {
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
private String name;
}

We would end up using this code in its simplest form like this:
Person theOne = new Person();
theOne.setName("Neo Anderson");
System.out.println(theOne.getName());

Instead we can easily modify this code to make it more "domain driven"
public class Person {
public String name() {
return name;
}
public void nameIs(String name) {
this.name = name;
}
private String name;
}

Our new code would be:
Person theOne = new Person();
theOne.nameIs("Neo Anderson");
System.out.println(theOne.name());

you can tell this is more english like code as well. You can easily read line 2 in the code there as "The one's name is Neo Anderson". You can easily read line 3 as "Print the one's name"

I believe in this approach. The easiest way implement this approach is:
1. If you have a private variable names "varName", your getter method would be named "varName"
2. The setter method would be named "varNameIs" .... append with an Is

Eclipse: Run a single unit test method

2009-06-10T16:00:00.000-04:00

In eclipse if we have a junit test class with several methods in it we can run a single method test. Select the method name, right click and run as junit test. The setup and teardown are run well also.

This technique, however, does not work with testng tests. Eclipse will run all the test methods.

TestNG and EasyMock

2009-06-09T17:06:00.000-04:00

Unit Testing is pretty cool once you get a hang of it and have the right tools to play with it. I started using TestNG and EasyMock recently. TestNG has this neat feature that allows us to partition the tests as unit, integration etc. EasyMock is not as easy but not bad.

Some problems I faced with unit testing were that sometimes a single unit test would run fine when they run from eclipse, but when the suite is run on CI they would fail. Most of the times the problem was that a easymock object was created and reused in its stale state

Liferay 5.2.2

2009-03-28T00:01:00.000-04:00

Liferay 5.2.2 was released in the end of Feb 09. Since then I have been playing and comparing it with 5.1.2. Here are the few things I think:
Pros:
- Journal Content renamed to a more appropriate Web Content
- More support for google adsense and google gadgets
- Nicer and more intuitive looking Control Panel for all the admin portlets
- Database configuration is now using portal-ext.properties instead of ROOT.xml
- The deploy and data directories are now not in user home. Instead they are at the same level as tomcat server itself
- There is just one built in / prepackaged compary - 7cogs instead of 11 Liferay companies 5.1.2 had
Cons:
- Larger zip file with tomcat 5.5 because it includes JRE now. (Well this will be a pro as well because now we dont have to worry about installing JDK on a machine that might not have root access)
- Build and startup takes a long time - as such it is a much heavy application now.
The cons list is not a big one and the pros are pretty good. I think however I will still stick with 5.1.2 and watch out for 5.3. The important change I would like to see is 5.3 being lighter weight and without its own JRE

Welcome

2009-03-21T20:22:00.000-04:00

Welcome