Is it bad to clean your cosmetic brushes with nail polish remover?

By the way nail polish remover smells I would think it would kill the germs and bacteria, and it gets all the makeup on them off really well. I'm just wondering if there's any reason I shouldnt be doing this.