Well, I'm quite new to testing but I would like to take programming to next level and begin testing, no unit-test but high level testing. Like this for example: Insert a job and check if it's valid by checking it's input with the result. Or another example: Insert a job and worker X started it: now I will check both the job and worker's properties to know if they are in a valid state.
But this is very very complex so I've been thinking of doing an Environment and instance one in each test, each environment creates an instance of the server and allocates an empty database to work with;
For example the start job high level case:
var environment = new GenericEnvironment(test); // test will be used to throw errors within the environment when something goes wrong.
environment.insertWorker({name: 'bla bla', number 99});
environment.insertARandomProduct();
environment.insertJob({productId: product._id})
environment.startJob({workerId: worker._id})
var environmentValidator = new environmentValidators.StartJobEV()
var job = environment.getLastInsertedJob(environment, test);
environment.validate(job);
// node.js (Javascript on server)
This example is very simplificated because in real world it uses async callbacks.
The environmentValidator uses data from the environment to check if the job is in a valid state. Also get the worker that is working in the job to check if it's state is busy, for example.
But I don't thing this is very great, and as I'm in the beginning of the test creation I ask you if it might be possible to improve the way of doing these high level tests.
Also I would like to know if there are any existing frameworks for those kind of test?
thanks
Real world code, but only for checking the state after inserted the job:
var serverManager = new ServerManager(test.getPathForDatabase());
require('production/tests/tottys/_environments/DefaultEnvironment');
var environment = new production.tests.tottys._environments.DefaultEnvironment(test, serverManager);
var dummyData = {}
test.addStep('initEnvironment', function(nextStep){
environment.init();
environment.addListener('afterInitialized', function(){
nextStep();
}, this);
});
test.addStep('insertWorker', function(nextStep){
dummyData.worker = environment.insertRandomWorker(function(data){
nextStep();
});
});
test.addStep('insertProduct', function(nextStep){
dummyData.product = environment.insertRandomProduct(function(data){
nextStep();
});
});
test.addStep('insertJob', function(nextStep){
var worker = environment.getLastInsertedWorker();
var product = environment.getLastInsertedProduct();
dummyData.job = environment.insertRandomJob(worker, product, function(data){
nextStep();
});
});
test.addStep('updateWorker', function(nextStep){
environment.updateWorker({workerId: environment.getLastInsertedWorker()._id}, function(data){
nextStep();
});
});
test.addStep('validateInsertedJob', function(nextStep, test){
require('production/tests/tottys/_environmentValidators/AfterJobInserted');
var EV = new production.tests.tottys._environmentValidators.AfterJobInserted(environment, test);
var job = environment.getLastInsertedJob();
EV.addListener('validationEnded', function(e){
nextStep();
});
EV.validate(job, dummyData);
});
test.start();
And the validation file:
qx.Class.define("production.tests.tottys._environmentValidators.AfterJobInserted", {
extend: qx.core.Object
,properties: {
}
,events: {
validationEnded: 'qx.event.type.Data'
}
,construct: function(environment, test){
this.__environment = environment;
this.__test = test;
}
,members: {
validate: function(job, dummyData){
this.__job = job;
this.__dummyData = dummyData;
this.__environment.getWorkerForJob(job, this.__afterWorkerReturned, this);
this.__environment.getProductForJob(job, this.__afterProductReturned, this);
}
,__afterWorkerReturned: function(worker){
this.__worker = worker;
this.__tryStartValidation();
}
,__afterProductReturned: function(product){
this.__product = product;
this.__tryStartValidation();
}
,__tryStartValidation: function(){
if(this.__preConditionsAreValid()){
this.__startValidation();
}
}
,__preConditionsAreValid: function(){
if(this.__worker && this.__product){
return true;
}
return false;
}
,__startValidation: function(){
var test = this.__test;
var dummyData = this.__dummyData;
var job = this.__job;
var worker = this.__worker;
var product = this.__product;
test.add("Expect job not to be done.", function(expect){
expect(job.done).toBe(false);
})
test.add("Expect job's requested quantity to be the same as requested.", function(expect){
expect(job.qtdRequested).toBe(dummyData.job.qtdRequested);
})
test.add("Expect job's missing quantity be the same as requested.", function(expect){
expect(job.qtdMissing).toBe(dummyData.job.qtdRequested);
})
test.add("Expect to be requested by the same request type.", function(expect){
expect(job.requestedByType).toBe('manager')
})
test.add("Expect job not to be done.", function(expect){
expect(job.done).toBe(false);
})
test.add("Expect job's done quantity to be '0' (Number)", function(expect){
expect(job.qtdDone).toBe(0);
})
test.add("Expect job to not have any time frames.", function(expect){
expect(job.timeFrames && job.timeFrames.length > 0).notToBe(true);
})
test.add("Expect job's date end to not exist.", function(expect){
expect(job.dateJobEnd).notToExist();
})
test.add("Expect job's date request to exist.", function(expect){
expect(job.dateRequested).toExist();
})
test.add("Expect job's state to be 'todo'.", function(expect){
expect(job.state).toBe('todo');
})
test.add("Expect job's order to be '0' (Number).", function(expect){
expect(job.order).toBe(0);
})
test.add("Expect job's worker's name to be the same as the requested one.", function(expect){
expect(job.forWorker.name).toBe(dummyData.worker.name);
})
test.add("Expect job's worker's number to be the same as the requested one.", function(expect){
expect(job.forWorker.number).toBe(dummyData.worker.number);
})
test.add("Expect job's worker's _id to be the same as the generated one.", function(expect){
console.log(worker);
expect(job.forWorker._id.toString()).toBe(worker._id.toString());
})
test.add("Expect job's product's code to be the same as the requested one.", function(expect){
expect(job.product.code).toBe(dummyData.product.code);
})
test.add("Expect job's product's description to be the same as the requested one.", function(expect){
expect(job.product.description).toBe(dummyData.product.description);
})
test.add("Expect job's product's _id to be the same as the requested one.", function(expect){
expect(job.product._id.toString()).toBe(product._id.toString());
})
this.fireDataEvent('validationEnded', {err: false})
}
}
});
If you start with testing, doing "high level" or "integration testing" (IT) is always tempting. It feels natural and more "easy" and more powerful than "mere" unit testing. You already have the whole system running, why not base tests on that?
But as always with software development, IT means you have more dependencies. If you want to run those tests, the database has to be in a certain state. The server will have to run. You will need a browser. Networking should work. For IT, the whole system has to be in an exactly defined state and you need tools to bring the system in this very state - you will have to do that before each individual test.
The problem: Each dependency is a source of errors. If the number of dependencies reaches a certain point, one of them will always be broken and your test will fail. Not because the code is broken, but because creating the initial state gets too complex.
This is why you should really start with unit tests. Find the smallest component in your system which has no dependencies and test that.
Advantages of this approach:
Conclusion: Get jasmine and start with some unit tests. When you have unit tests for everything, writing ITs for the whole system will become much more simple ... if you need them at all.