How I was evaluating a tender

If you have read my previous article, you know that I have used a questioner for 1st round of the tender. In this post I'll try to explain you how I was evaluating it.

Cube in orange stock photo by www.pixmac.com

Project overview
I have described our project as a project which should be developed in next months for some thousands of € and than maintained (including developing new features) for next 5 years for some hundreds of € each month. So we are looking for some longterm relationship.


Tender attendants
I use 2 sources for attendants 1st was published project on the Elance. On Elance have answered me 16 companies, but 5 of them didn't respect our very basic criteria (which was in project description). It was that we want to have only companies which are located in GMT+1 (+- 2 hours). I sent them the questioner and 7 of them fill the questioner. Most of them were from East Europe.

2nd source were companies which I know from Czech Republic and think/know that they are good or I found them on Google and their website provide information that they are high probably a good contractor. I have sent 9 mails, and 8 (one company forward the tender invitation to partner company, because they didn't have time for this job) fill the questioner.

There was one more company which is developing prototype of the service, but we aren't sure if it's good company to develop and maintain our service.

Evaluation
I have used the decision matrix technique and I extend it with weight. Firstly I set up the criteria for evaluation. Company was able to gain from 0 to 100 points for every answer. Than I set up the minimum number of points for every answer (must-have). After that I evaluate each questioner response according to this criteria.

You can download the file which includes both (without company names, of course) in xlsx or see in Google docs (not as pretty as xlsx ;o)). Criteria are in the second sheet "settings".

I need to say, that it was my framework for doing this tender. Next time I'll probably change some of these criteria. I don't want to comment every criteria, but I'll quickly mention points which were interesting for me.

Basic information
For example I was surprised that most of companies hasn't any plan for illnesses or if somebody of the team will leave the company. It seems that they are waiting and they will solve it somehow. Only few of them mentioned what exactly they use to deal with it. Or only one company mentioned that it has some pricing method for maintaining and adding new features.

Code quality
Only few of companies was able to explain why they are using the frameworks which they have mentioned in the previous question. Sometimes they only try to describe how it is ease to work with them. In worse case how they needed to rewrite the framework they use. Btw. do you know why a lot of professional companies use SVN today? I remember how it was complicated to work together until we switch to GIT...

Development process
I was looking forward to see how they deal with deadlines and changes in specifications. Unfortunately no surprise. Only two companies mentioned that they are using agile techniques and in many other explanation there were things that showed that they are not using agile even they have wrote it in previous answer. Unfortunately nobody is using Feature driven development which I would like to see in practice.

Final decision
Who should we invite to the 2nd round? That was the question. I don't want to waste time of people in many companies just for our tender if I know that there is only minimum chance that we will choose them. And finally I don't want to compare 16 complex offers. So I decided not to only choose companies which meets our basic criteria, but pick only few (5) of them which have the highest number of weight score.

PS: As I have promised, the questioner is now public for use.