Do people really believe that libertarians are drug addicts or can they only argue with straw men?

People who are against drug legalization like to claim that people who are pro-legalization only want to legalize drugs because they use drugs. Do they seriously believe this or do they say it because they can't form an adequate argument without poisoning the well?