Top 5 Ways to Make Better Software

I don't think I've ever met anyone involved in software development that didn't care about making a great product.  Of course there are varying degrees of caring, but I think for the most part most people are pretty passionate about creating something worth putting their name on.

Unfortunately, for a lot of software dev shops the product that's released doesn't turn out to be as good as the developers had hoped for.  There could be several reasons for this.  It could be that the devs are inexperienced, or maybe they had to cut some corners to meet the deadline.  There could be any number of reasons, but this article is meant to help us understand the top 5 ways for making better software.

1. Test Everything (that matters)
2. Peer Code Review
3. Adopt Agile Principles
4. Teach Each Other
5. Have Fun

Before I get into the top 5 I should probably also give some context.  I've found that these 5 things are surprisingly absent in the organizations that would reap the biggest benefits: large enterprises.

Test Everything (that matters)

I'm always surprised when I get into a complex system only to find there are no tests.  So, the question of course pops up:  Writing tests takes too long, why would I want to write tests?

Reason 1: Regression tests.  The tricky thing with large and complex systems is that they can be really awkward to maintain.  The simplest change can land you into a tailspin of unexpected changes and, if things are really out of control, they can lead to new bugs being introduced.  So how can we have confidence we haven't broken anything else with the change we just made?  Hopefully there is a test suite that we can run to make sure we didn't inadvertently break anything.  Now, of course if simple changes cause huge headaches then there is probably something wrong with the software design, which leads us to reason 2.

Reason 2: As a design tool.  Test driven development(TDD) has been taking the industry by storm over the last several years and the reason is more than just making sure we have a test suite.  TDD is an excellent design tool that helps us avoid many common patterns that lead us to the problems that exist with a bad design, i.e. hard to maintain code.  

Automate the tests.  Of course even if we are writing good tests we may accidentally break something as we're adding a new feature.  To make sure we're getting the full benefit of our tests we want to make sure they run automatically.  For this reason we have continuous integration(CI) servers like Hudson or Jenkins.  They should run the tests at least once per day and send out any alerts if a test failure is found.  Another nice feature about CI servers is that they'll tie directly into your SCM, like SVN or git, so they can see what changes to the code base caused the test to fail.  

Testing what matters.  We could literally spend all day writing tests for all the code we could create so what is worth testing?  Business logic is probably the only thing worth worrying about.  All those getters and setters, probably aren't useful to test, unless there is some business logic in there.  Basically the rule I use is if the code wasn't automatically generated then it's probably worth testing.  

Peer Code Review

The peer code review is a dead simple tool that has a dramatic effect on code quality.  In its simplest form it's just having someone else looking at the code you wrote and giving you feedback.  Asking someone to look at your code doesn't mean you're a bad developer. 

Catch bugs before they go to QA.  The most obvious benefit for code reviews are going to be catching bugs before they leave development.  No matter how experienced a developer is sometimes our eyes begin to play tricks on us and we might type the wrong thing or maybe we understood the problem incorrectly.  At any rate, code reviews help catch bugs while it's still fairly cheap to fix. 

Share knowledge.  Another great benefit about code reviews is the knowledge sharing that happens. It isn't uncommon to find new ways of doing something while performing a code review.  As a developer I've learned a ton from looking at the code written by others.  

Pre-commit:  Generally there are two different ways of doing a code review.  Both are useful, but for different reasons.  A pre-commit code review would take place before you commit your code to the SCM. In essence you would call someone over and you would walk them through what you had done.  This allows for a quick 2-way dialogue and probably the most effective way to do a code review.  The only drawback is you need to find someone who is available when you're ready to commit.  

Post-commit.  In contrast the post-commit review is done after you have committed your code and is usually put into some kind of code review tool like Crucible or Gerrit.  The reviewer could then go look over the code when it's convenient.  As you can imagine any communication isn't nearly as effective as a face-to-face conversation would be.

Adopt Agile Principles

Agile is another one of those terms that has taken the industry by storm and it has come to mean a lot of different things to different people.  In a nutshell Agile is a philosophy that helps us focus on delivering the most value as soon as possible. Just like anything it takes practice to get it right, but the dividends are awesome. 

Agile values and principles.  Agile was born as just a bunch of ideas that were grouped together in the Agile Manifesto and also the Agile principles.  They're pretty easy to understand, but it might not be as easy to see how to apply them.  This is where an agile framework would be useful.

Scrum.  Scrum has become the most popular agile framework used in the industry today.  It builds off of the Agile values and principles and gives us some simple rules, roles, and meetings to follow that help us deliver working software as often as possible. A couple of paragraphs aren't nearly enough to adequately explain Agile and Scrum, but there are a ton of resources available.

Continuous Improvement. The biggest benefit your team will get from adopting Agile is the idea of continuous improvement.  This principle helps us to always keep an eye to the future to continually work better and better. 

Teach Each Other

Along the lines of always improving comes the idea of learning.  Learning is an absolute necessity for any developer to stay marketable.  I've also found that the best way to learn, and retain, an idea or skill is to teach it.  

Good for business, good for people.  Traditional organizations follow the mantra of only the leaders and mangers have ideas worth listening to and everyone else is there only to carry out those ideas.  An organization like that will generally find that their employees aren't very engaged and it's difficult to get new talent.  All in all, it's a pretty crappy place to work.  On the flip side, when an organization values their employees and their ideas people will want to work there and will feel a great desire to give their best effort. 

Culture of learning.   Part of learning is making mistakes, but the problem is our culture teaches that making mistakes is bad and because we think mistakes are bad we're afraid to make any.  This fear of making mistakes smothers our desire to innovate and experiment.  In this way the culture inhibits learning.  But, the best way I've found to overcome this fear is to create a classroom type setting where it's okay to make mistakes.  Little workshops and trainings can go a long way to help and eventually will spill over into the main day-to-day operations.

Have Fun

Creating software requires a lot more creative thought than most people think.  It can also be very stressful since the nature of software makes it very hard to predict when it will be finished.  Having fun is a great tool for battling the stresses that accompany software, but more importantly it increases our quality of life.

Laughter encouraged.  A little while ago I watched a video with famous comedian John Cleese about the effects of laughter.  Even if its forced, laughter has been shown to relax us and make us more healthy.  It also has a strange way to spreading to others.  When we hear others laugh we start laughing too, even if we didn't hear the joke. 

Change of scenery.  Sometimes the easiest way to have fun to switch things up a little.  Take time to talk with others, or take a walk as you find your brain reaching its breaking point.  Games and jokes are also good ways to bring out smiles.  We spend most of our waking time at work we should make it a place we enjoy being at.

Creative Juices.  Many people don't realize how much creativity is required for software development.  That might be because we often refer to building software as engineering which has a lot of connotations of calculations and mathematics.  For a lot of software that isn't the case.  It's more like growing a garden than building a bridge.  At any rate, fun and humor help to get our creative juices flowing so that we can keep growing our software garden into something pretty awesome.

So these are my top 5 ways to improve software that would have the biggest impact for most companies.  I'm positive there are a million other ways that would have a dramatic effect on improving software and I'd like to hear them all so please share any good ideas in the comments.  

References and Further Reading:

Testing Links:

Top 12 Reasons to Write Unit Tests: http://www.onjava.com/pub/a/onjava/2003/04/02/javaxpckbk.html

Test Driven Development: http://en.wikipedia.org/wiki/Test-driven_development
Continuous Integration: http://en.wikipedia.org/wiki/Continuous_integration

Hudson: http://hudson-ci.org/

Jenkins: http://jenkins-ci.org/

Peer Code Review Links:

Code Review: http://en.wikipedia.org/wiki/Code_review
Crucible: https://www.atlassian.com/software/crucible/overview/
Gerrit: https://code.google.com/p/gerrit/

Agile Links:

Agile Manifesto: http://agilemanifesto.org/
Agile Principles: http://agilemanifesto.org/principles.html

Scrum Alliance: http://www.scrumalliance.org/
Continual Improvement: http://en.wikipedia.org/wiki/Continual_improvement_process

Teach Links

Transforming Your Manufacturing Organization 

 Into a Learning Organization: http://www.businesstrainingexperts.com/white_papers/training_return_on_investment_roi/Learning%20Improves%20Manf%20Productivity%2017%25.pdf

How Corporate Learning Drives Competitive Advantage:

http://www.forbes.com/sites/joshbersin/2013/03/20/how-corporate-learning-drives-competitive-advantage/

Fun Links:

6 Reasons to have more fun at work: https://www.americanexpress.com/us/small-business/openforum/articles/6-reasons-to-have-more-fun-at-work-1/

Laughter is the Best Medicine: http://www.helpguide.org/life/humor_laughter_health.htm

15 Ways to Make Work Fun Again: https://www.americanexpress.com/us/small-business/openforum/articles/attitude-changes-everything-why-fun-at-work-can-mean-success-in-business/

How to Play and Boost Creativity: http://www.helpguide.org/life/creative_play_fun_games.htm
Benefits of Laughter w/ John Cleese: https://www.youtube.com/watch?v=yXEfjVnYkqM

How to Not Run Integration Tests With the Eclipse JUnit Plugin

Here is the scenario.  A Maven multi-module project being developed in Eclipse, the m2e plugin keeps the different modules in sync in eclipse, integration and unit tests mixed throughout each project, and integration tests using the naming convention *IntegrationTest.java.

Here is the problem.  Using maven to run unit tests for the entire project kind of sucks.  I mean it's easy to limit the tests being run with the maven sure fire plugin's exclude, but the problems are 1) it takes too long to build the project before the tests are run and 2) if you have altered a upstream project it won't show any compile problems in eclipse because m2e keeps them in sync, but the maven build doesn't seem to know about it so it fails to build.

Half of the solution

The first half of the solution is the JUnit plugin that comes with eclipse.  You can right-click on a project and Run As a JUnit Test, or ctrl+alt+shift+T, and it will run through all the tests really fast.

This is only half of the solution because it also runs through the integration tests, which generally take a long time.  

To work around running the integration tests you can create a test suite.  The test suite is a pretty good way to  limit what tests are run, but out of the box JUnit 4 requires you to either annotate each test with a @Category annotation on each test and then to use the @IncludeCategory or @ExcludeCategory on the test suite.  Or, the other less friendly version would be to use the @SuiteClasses annotation on the test suite and list out each test class individually.  When you have hundreds of test classes this is a lot of work.

The other half of the solution

What we really need is a way to use some wild cards to limit the tests we want to run.  Similar to the surefire plugin's exclude.  The solution I've come across is something called the JUnit Toolbox.  The cool thing about the project is that it has a couple of custom runners that allow you to specify wildcards in the classes to run. What that means is that we can write a test suite like this

   import org.junit.runner.RunWith;

   import com.googlecode.junittoolbox.ParallelSuite;

   import com.googlecode.junittoolbox.SuiteClasses;

   @RunWith(ParallelSuite.class)

   @SuiteClasses({"**/*Test.class", "!**/*IntegrationTest.class"})

   public class AllUnitTestsTestSuite {} 

and it will only run the tests in that project that don't have IntegrationTest.class.

Now the only downside I've come across is that if you have classes that match the naming convention and they don't have tests in them you will get a initialization error, but you can use categories to restrict those ones.  Also if you're looking for something to run tests for all of your projects this doesn't seem to address that either.  All in all though, I've gone from having to use a fairly manual process that takes 20-30 seconds with maven to a pretty automatic one that takes about 9.  

Java and Mockito: Patterns that Make Testing Hard

This article isn't meant to convince anyone of the importance of unit tests.  There's already a ton of information out there for that.  This article looks at a few common practices that make testing harder and might tempt a developer to skip testing altogether.  I'm in no way saying the solutions presented here are the best solutions.  Every case is different and every developer is different too.  If these solutions don't fit your style I'd love to hear of ones that do.  

Complex Private Methods

In development we're often taught that we should keep related method calls inside of the class.  This is good, but it can often lead to private methods with a lot of logic.  

Consider the following code.
The problem: We can't isolate each piece of logic in our tests.  Our tests need to account for each private method scenario and makes testing a pain to write and maintain.  In each test we also have to account for the logic in the executeSomeLogic method.  If we refactor any of the private methods it could possibly cascade to forcing us to refactor all of our tests.
A solution: Pull the private methods out into a separate service.  That way we can mock them in our original test and test each private method in isolation. Like this:

Static Util Methods

Static methods can be very handy to use since we don't need to worry about instantiating.  And, generally every static method will include its own test suite so we can be confident the static method will execute how we expect. But, because of their static nature we must always execute that logic, even when we're testing something else.

Consider the following code.
The problem: Logic that uses static methods will be executed during the test and can lead to more complex setup. Basically, you would be testing your logic and the logic of the static method.
A solution:  Wrap the static methods in an interface that can then be mocked.   Like this:

Obviously (I hope), if the logic is very simple then this would not be worth it. 

Using @Before to setup for every test 

Tests can get pretty complicated and the junit @Before annotation can be abused to the point where the test becomes hard to maintain.  

Consider the following code.
The problem: The setup up becomes too complicated so that when we refactor some existing logic and we need to update the test we find ourselves taking a lot of unnecessary time understanding how the test is setup before we can alter it.
A solution: Write more setup code in each test and keep the code in the @Before block to a minimum. This allows you to be flexible and it is easier to modify without any unforeseen consequences. Like this:

And if you find yourself repeating a lot of code create little helper methods.

Long if...else or switch clauses

If..else and switch clauses are a great way to easily step through a logical ladder to get the desired behavior. But with that quickness comes more complex testing.

Consider the following code:

The problem:   A long logical selector forces us to write a lot of tests in order to cover each possible scenario.

A solution:  Refactor to use a common interface with an implementation for each scenario.  This allows us to isolate our tests and to refactor without affecting any other code.  Like this:

Understandably your particular case probably isn't going to fall into one of the above problems exactly and you probably won't be able to make your tests and business logic 100% simple.  That's okay in my opinion.  My goal here is only to get people thinking about alternatives to some common practices and why some of these practices cause problems.

Dive Deeper with some light reading

Effective Unit Testing: A guide for Java Developers

Testing An Expected Exception Message

To test that an exception is thrown in JUnit you can use @Test(expected = "WhateverException.class"), but the only drawback is that you can't test the exception's message.

A way to test the expected exception as well as the expected message is to use the @Rule annotation with ExpectedException.

Here is an example.

SpringMVC BindingResult Model Key

When SpringMVC has validation errors with a BindingResult it stores those errors in the model map so they can be displayed on the JSP. I keep forgetting what the model map key where it's stored at so here it is.

org.springframework.validation.BindingResult.<parameter name>

FYI, I use this sometimes for testing my controllers with spring-mvc-test to make sure that validation errors are set.

Something like this:


UPDATE:

You can also use the test framework's built in methods to validate if an error exists or doesn't exist.

Check If Any Number of Objects Are Null

Here is a little snippet of code that is useful when you need to check if any of a bunch of objects are null. Instead of writing if (object 1 == null || object 2 == null || object3 == null) you can write if(MyUtility.hasNullValue(object1, object2, object3))

The Code! and test

Utility to Compare Similar Objects

Every once in a while I find I have to make similar objects that need to map to each other (basicaly copy like fields from one object to the other) and I wanted a simpler way to make sure that my mappings were correct for my unit tests.

Of course there is the manual way of doing assertEquals(object1.getValue1(), object2.getValue1()) but it does get tedious after the first two or three. Of course with this way you may be more likely not to overlook any fields since you effectively have to do it once during your mapping and then again during your unit test.

Another way to do it is to use reflection. I created a utility that will grab all the getters from one object and try to compare it to the getters of the same name of another object. For example: in Object1 we have getValue1() and in Object2 we have getValue1(). The utility would run through Object1 and grab the getter getValue1() and then try to compare the return results to a getValue1() on Object2. If the values don't match or if Object2 doesn't have that getter than the name of the getter is added to a list.

A better way to explain it is with the code. Below is the utility as well as a test that shows how it can be used.

PermGen Out Of Memory Errors on Maven test

So we have some pretty intense projects that need a lot of memory to run. We added some VM arguments to give our tests more memory, something along the lines of -Xms512m -Xmx1024m -XX:MaxPermSize=512m we're still getting PermGen out of memory errors.

I wasn't sure how much memory we were actually using during the tests so I was able to connect to our maven instance by adding the following to the command: -Dcom.sun.management.jmxremote-Dcom.sun.management.jmxremote.port=8086-Dcom.sun.management.jmxremote.ssl=false-Dcom.sun.management.jmxremote.authenticate=false.

As I monitored the test we never got above 20Mb of PermGen so what gives?

As I scoured the web I cam across a few blogs, such as this one that describe that when the surefire plugin creates a fork to run a test that those memory arguments you gave don't get passed along.

In order to make sure the surefire uses the right memory args you need to specify them using the argLine element, like so:

Another good option that I saw, but can't find again, was to use a variable as a place holder in your argLine: And then pass it in your command like mvn test -Dmaven.test.argline='-Xms512m-Xmx1024m-XX:MaxPermSize=512m'.

Creating JpaRepositoryFactoryBean in a @Configuration class

In our integration tests we use configuration classes to only load the spring beans that we actually use. So, to initilaize our @Autowired Spring Data JpaRepository in our configuration class we could do something like this:


The problem with this solution is that there is a lot of boilerplate code in configuring the JpaRepositoryFactoryBean that makes this a little tedious to repeat, especially if you have more dependencies in your test that you need to initialize.

However, with generics and a static method we can make it a lot cleaner. With this static method we can create our JpaRepositoryFactoryBean with only a few parameters.


And this is how you would call it.