Tag Archives: testing

Quote

On specific Agile (software development) traps

There’s much positive to be said about the benefits of Agile software development, and the shift of software development teams is one sign that many feel this way.

However, I think there are some limits to Agile, and this leads teams to fall into certain traps over and over. Indeed, the Wikipedia page highlights a common criticism of Agile, namely:

Lack of overall product design
A goal of agile software development is to focus more on producing working software and less on documentation. This is in contrast to waterfall models where the process is often highly controlled and minor changes to the system require significant revision of supporting documentation. However, this does not justify completely doing without any analysis or design at all. Failure to pay attention to design can cause a team to proceed rapidly at first but then to have significant rework required as they attempt to scale up the system. One of the key features of agile software development is that it is iterative. When done correctly design emerges as the system is developed and commonalities and opportunities for re-use are discovered.

Now, it’s not the case that teams either do design or not. But I have seen that there are a number of specific traps bigger that Agile teams fall into that arise from lack of design. These traps arise from making tactical or limited decisions outside of a larger framework or structure, which isn’t surprising since Agile followers are guided by the principles that the “best architectures, requirements, and designs emerge from self-organizing teams” and “working software is the primary measure of progress”. Unfortunately what I’ve seen that lead to is:

  • Poor middleware/database system decisions: with this trap, you get teams making a decision on deploying a specific middleware package or database system that will support the software development they are doing. However, while that may be great for the Dev, it may not be great for the Ops. If you have enough teams making tactical decisions, you may end up with a more complex system than is necessary and greater technical debt that you want. Once you get enough data into your database systems, trying to reverse the decision may result not result not only in new development and testing, but a small (or not so small) migration project as well.
  • Poor infrastructure decisions: likewise, with this Agile trap I have seen teams using IaaS pick different images to deploy their software onto. Like the database problem, developers may choose one operating system (e.g. Windows) over another (e.g. Debian) because they are comfortable with the former, even if the production environment is more of the latter. The result can be your organization ending up with multiple VMs with many different operating systems to support and thereby increasing the operational costs of the system.
  • Poor application framework decisions: I see less of this one, although it can happen where teams pick an assortment of application frameworks to use in creating their software, and as with middleware and infrastructure, this will drive up the support effort down the road.

Given these traps, I think the way to avoid them is to inject some specific design phases into the overall software development lifecycle. One way to do that is to revisit a software development lifecycle (see diagram below) used by practitioners at IBM and documented in places like this IBM redbook. It has a waterfall quality about it, but it doesn’t have to be limited to waterfall type projects. It can be highly iterative.

The lifecycle process is shown here (taken from the redbook):

 

GSMethod

The part of the lifecycle in the large box is iterative and not all that different from an agile sprint.  But here you take time to explicitly make design / architecture decisions before building  software. You continue to iterate while making important design decisions before building.

Now, before you start your iterative software development lifecycle, you should need to  make specific architectural decisions. You should make these specific decisions in the solution outline and macro design phase. For smaller teams, these two phases may blend into one. But it is here in solution outline and macro design where you make decisions that are fundamental to the overall solution.

For example, in solution outline you could make overall architectural decisions about the application architecture, the choice of infrastructure technology, what application frameworks are the target for the team. These overall architectural decisions guide the dev, test and ops teams in the future. You may also decide to park some of these decisions in order to do further discovery.  Macro design could be next, where each of the dev teams make certain design decisions about their technology choices before they proceed with their iterations. As they are building and deploying code, they can run into issues where they need to make further design decisions, either due to problems that arise or technology choices that have to finally be made: and this is where the micro design phase is useful.  Micro design decisions could be quickly made, or they may require spikes and proof of concepts before proceeding. Or there could be no decisions to be made at all.  The main thing is more design checkpoints are built into the development lifecycle, which can result in less complexity, less maintainability costs, and less technical debt down the road. What you lose in velocity you can make up in overall lower TCO (total cost of ownership).

There is a risks to this type of approach as well. For example, if the project gets hung up with trying to make design decisions instead of gather requirements and making working software. The key stakeholders need to be aware of this and push on the design teams to prevent that from happening. If anything, it can help the key stakeholders better understand the risks before getting too far down the road in terms of developing software. Overall I think the risks are worth it if it helps you avoid these common agile traps.

I create a super simple set of tools to secure your Ubuntu server

And you can get it here: blm849/supersimplehardening: A super simple way to harden your server.

I create a lot of Ubuntu test servers, and I find that as soon as I create a Ubuntu server on a cloud environment, it gets immediately attacked by automated software. This is obviously a concern. A bigger concern is that when I went  searching for recommendations on how to harden such a server, I found  a wide variety of recommendations! It can be hard to know what to do. Still, I needed something. As a result, I created this package of scripts. The scripts do a number of things:

  • prevent direct root login to your server via ssh. This was one of the things I saw consistently happen and once someone cracks the root access on your machine, it’s game over.
  • stop some basic security holes, like IP spoofing
  • download some useful software, like logwatch, ufw and others
  • upgrade all software on the server

This is just a very very limited number of things to prevent attacks. But it is better than nothing.

If you install Apache, PHP, MySQL or other software on your machine, there are even more attacks that will be launched against it. I recommend you get a firewall up and running and at least run logwatch on a regular basis to look for potential attacks being launched against you.

Finally, if it is important for you to secure your server, don’t stop with my scripts. Go out and consult with IT security specialists right away.

Good luck!

Installing SonarQube on CentOS/RHEL

The following webpage has detailed instructions for installing and configuring SonarQube on a RHEL/CentOS 7 Linux server (real or virtual) and it was one of the best guides I’ve seen (and I’ve reviewed half a dozen):

The webpage  outlines how to update your Linux server, how to install MySQL (as a data repository) on it, and how to then install SonarQube software on the server.

Some things to note. First, this procedures has you using wget to get v6.0 of SonarQube:

Check out the page https://www.sonarqube.org/downloads/ and see the latest version of SonarQube (e.g. 6.4) and replace “sonarqube-6.0.zip” with the latest version (e.g. “sonarqub-6.4.zip”.)

One important thing to note: this procedure creates a userid and database called sonarqube.

Later in the process, the changes made to /opt/sonarqube/conf/sonar.properties needs to match this:

sonar.jdbc.username=sonarqube                                   sonar.jdbc.password=password
sonar.jdbc.url=jdbc:mysql://localhost:3306/sonarqube?useUnicode=true&characterEncoding=utf8&rewriteBatchedStatements=true&useConfigs=maxPerformance

If the userid, password and database you created in MySQL do not match what it is the sonar.properties file, you will see cannot connect to the database errors in the /opt/sonarqube/logs/web.log file and SonarQube will not come up.

Once you enter: sudo ./sonar.sh start

Get the IP address of the SonarQube server and then go to a browser and enter:

If you are cleaning up an environment by deleting resources in Amazon’s EC2, here is a checklist to get you started

I just cleaned up an environment I had set up in Amazon years ago for a client. (The client wanted to use Amazon, so we did.) In doing so, I wanted to make sure I didn’t leave anything behind which would cause me to continue getting billed even though I was no longer actively using EC2. I believe that the following checklist was useful in insuring this.

My EC2 cleanup checkist:

  1. Delete my Elastic IPs
  2. Terminated instances – running and non-running  (I did this before deleting volumes, since it deleted alot of them for me)
  3. Delete remaining volumes
  4. Delete my security groups ( 1 will be left – the default one)
  5. Deregister AMIs
  6. Delete snapshots (you need to deregister your AMIs before you do this)
  7. Check your account balance
  8. In a few days, check your account balance to see if there are any charges you haven’t accounted for

After following this checklist, my EC2 environment was cleaned up. Depending on how you are using EC2, you may have more things to delete. Checking your account balance will help there: if you left things behind, they may incur charges. An increase in your account balance will help flush them out.

One thing to consider: you may delete something, but it doesn’t show in admin console. If that is the case, logout and then in. I did that when I was having trouble deregistering my AMIs. I logged out and then in and when I checked them, they were now deregistered.