I like perfection but don't expect to see it in everyday life.
So my way of rating a contractor is partially on the quality of the job they initially did, and partially on how they respond when a problem is discovered.
A & J Roofing (the J is Jerry - a great guy with whom most of my conversations were held, the A is his brother Art who is also a great guy) had done 3 houses on our side street containing 12 houses. And they came highly recommended. So when it came time for me to replace the roof, due to a recently discovered leak, I called them in.
They were price competitive. They came when they said they would. They got the job done on time. And they cleaned up nicely after they were done. What more could you ask?
Unfortunately, a new leak developed shortly after the new roof was on. Back came the team to look it over. They were upset that their work had not been perfect and that a leak was happening. They thought the water was coming in a side vent or a window flashing, so they caulked everything up (at no cost to me) and away they went.
Darned if the leak kept happening, so back they came, really upset (with themselves) now and literally cut into the roof, re-did some shingles (again at no cost to me) and asked me to call them immediately if anything was wrong.
Guess what, it still leaked. So they came back, and replaced the entire side of the roof (at no cost to me) and voila, no more leak.
In addition, they offered, without me asking, to bring in a contractor at their expense to repair the kitchen ceiling that had been damaged during the leaking.
I know how difficult it is to track down water leaks, Where the leak shows is rarely where the leak originates (water travels along rafters, ...). These guys really cared about the quality of their job and did everything possible to rectify a problem.
To me, that is the sign of a conscientious contractor and I would hire them again in a heartbeat.