Why is that the shape of the tension test specimen a dogbone-like shape?
I know that it is so that the deformation is confined to the narrow center region and to reduce the likelihood of fracture to occur at the ends of the specimen. But why is this so important? Why can't a cylinder shaped specimen that has a uniform radius all along the specimen be used? Why does it, for example, matter if it fractures near the end (near the region where you clasped the specimen in the machine)?
You half-answered your own question. Preventing failure in the grips is important. Additionally, grips of tensile testing machines have teeth to achieve a sufficiently strong grip that can withstand the forces required to deform the sample longitudinally. The teeth typically cause plastic deformation of the gripped portion of the sample. The plastic deformation at the grips may change the material properties, and definitely changes the sample geometry. The sample geometry changes in such a way as to create stress concentrations in the gripped region. Both changes would contribute to an improper measurement if failure occurred at the grips in a cylindrical or bar specimen without enlarged ends.
Starrise gave a good explanation of the reasons why a dog-bone shape is important to the tension portion of a tension test, but there is another property that is typically measured at the same time: Elongation.
Elongation is measured by placing a gauge on the reduced section. It is important that the elongation occurs in this area so that the measurements are accurate. If the movement of the machine grips is used, then any plastic deformation of the grips will add to the elongation measured.
Once I tried to break a rod without the reduction of diameter, just to see what happens. The rod broke right at the grip and the force released at the breaking point was transferred to the grip and it also broke it; it was an expensive "let's see what happens" thing.