Top Seven Software Project Management Tools

What do you need to manage a reasonably large software project?

I am fortunate to be on a project where the following tools are vital to managing the developers and the development of the software.... its life-cycle:

The critical items I would recommend are the following:

  1. Git
  2. Bug Tracking
  3. Database Management
  4. Unit Testing
  5. Exception Logging
  6. SQL Logging
  7. Authorization Logging

Git

Git, or it's cousin subversion is really an insurance policy the company that is developing the software must have. It keeps track of all changes, can reverse changes, and generally, keep the code that is being worked on organized. No one developer can easily delete the work you have invested in, and past work and all changes that are made can be managed. It allows teams of developers to work on the same set of code without stomping on each others feet, or cancelling out each other's effort. Github and Bitbucket are well known, mature Git repositories. Git can be run on your desktop, or on your own servers or via a provider such as the two mentioned. Git is generally opensource and free.

Bug Tracking

It is a good idea to have a bug tracking system on large projects. This allows bugs that are discovered to be entered into the system and the project manager can assign a priority to the problem and assign it to a developer. This is often used in conjunction with a wiki of some sort. I have installed various open source, free bug tracking systems while on projects. They are also cloud based as well. Jira is one, Bugzilla is another.

Database Management

When you have many databases to manage, things can go wrong fast. You may have a development, test and production database. You likely have others example a demo system database. Keeping all these in sync is an important step as the consequences of not doing so will lead to huge problems. We have had weird issues costing valuable troubleshooting time when this has happened.

There are many ways to handle this. One technique I have used on projects is to "check in" every SQL database change script into git so that those in need can run them against the other databases.

On my current project, we are using phinx, a tool that can keep track of the changes to each database, and roll them forward or backwards as needed.

An additional tool we developed gives us a "diff" between databases so that we can check that the databases, their indexes etc are identical.

Unit Testing

Unit testing is critical in certain projects, but not always useful in all software development. Where you have functions/methods that do a particular job, and the output can be tested, you should employ unit testing. It is a discipline. The payoff is both now and later. Example, if five years from now, a developer makes changes, and your unit testing environment was invested in up front, you can, with one click, run thousands of unit tests.....and, the point here is that if the developer has made a change that causes a unit test to fail, you can catch the problem right away, rather than having the error introduced into production where it will cost you dearly.

So, an investment in unit testing can save you time, money and heartache down the line, and it should be in your software project toolkit.

The remaining items are similar tools that you need to build yourself, but are nevertheless as important to your managing the development of your software. Missing here are aggregating your server error logs and other important logs on other machines. These below are the least well known, but useful logs:

This is custom logging, and this is how I have approached it on the current project I am involved with:
These should be logged off to another system. In this case, to ensure that a problem on the logging box does not halt or affect the machine that does the logging, we first used a web-service to transfer the logs. The limit to an HTTP POST request was often an issue, so we switched to Amazons Simple Queue Service, SQS which works very well for the purpose. We developed a method of splitting up logs of any size into smaller peices when needed. The base SQS message size is quite large to begin with at 256KB. This blog topic may fit within one message.

For all logs, you need to specify the environment ( example, development, testing, production ), the date and time the log was created and naturally, you will need ways to filter and order and search your logs.

Exception Logging

Keeping a log of all exceptions will help you troubleshoot problems when they pop up unannounced. In the PHP Codeigniter framework, one can add a "hook" to catch every exception and php error type. Do it!

SQL Logging

We log every sql call made, and along with it, the associated web page or mobile device call made and the time the SQL statement took. This is a huge volume of logs, and as such, the data is kept for only a month. The data is encrypted at rest, and during transport, and information such as passwords etc are first removed. The value here is twofold: you can use this to determine what is going wrong when there is an issue to troubleshoot, and just as important, keep track of the time the sql statements are taking to run, either in aggregate for all statements on a page, and for each statement. This gives you a tool to speed up slow queries (outside of the "slow query" log in MySQL) that is critical. We have a slow query report based on this data that helps in tuning the user experience.

Here is a snippet of code that will give you all the SQL calls within Codeigniter. A good place to call it from is the destructor of the "application/core/My_Router" class.

foreach ($CI->db->queries as $key => $val)  
{
    ...
    $data->log_sql_query_time = $CI->db->query_times[$key];
    $data->log_sql_query_sql = $val;
    $data->connection_db_host = $CI->db->hostname;
    $data->connection_db_database = $CI->db->database;
    ...
}

Authorization Logging

Keeping logs of all logon's, logoff's and failed logon attempts will help you defend your website when the need arises. Along with these logs, you need the usual machine logs. Within the mix are the password resets, password changes and permission failures.

Investing in logging the above will help you manage your software development when things do not go as planned.