To a mathematician, mathematics is about solving problems. Mathematics is not just a collection of “facts” about the numerical and spatial aspects of the world, it is a process by which we use and develop them. The interesting part is in the reasoning, the justification, and the way different concepts are brought together to answer a question.
Outside of the halls of education, we have many real-world ways to test knowledge and skills. If we want to teach someone to bake a cake or do a J-stroke in a canoe, we don’t give a paper and pencil test – we look for successful demonstrations of the skills. Even where we need a paper test for some head knowledge stuff, like when we get a driver’s license, the road test is the real proof of the pudding. In school settings, then, what we really need are valid measures of what students know and can do.