Why do you believe there must be a line? Eve right-wing libertarians have widely different opinions about the proper role of the State, or even its existence. In any case, I'd say most are not in favor of conscription itself.
But there has to be a philosophical distinction right? Some way of clearly marking where the state's power ends and individual rights begin. Otherwise libertarianism is just a matter of "I know it when I see it", which means people will twist a philosophy of small government to fit their particular political needs.
