Yours is a very Libertarian view. in a perfect world, it may work, but it doesn't in the world in which we live. Society has to step in where families ought to but don't. This is because the lack of parental control over minors has a big effect on society and its norms.

Governmental standards put in place what private persons do not: succession, guardianship, education, health care, etc.

The Libertarian world breaks down in practice, even if it appears to be a great model in theory.