You could go a step further and make the fungus what makes them go nuts. Like that ant-zombie-fungus, it makes it go on killing sprees to spread the spores around. None-infected trolls are fairly intelligent.