Druids as low as level 60 can kill them relatively easily by summoning a Demon Skeleton and using Ice Strike and Hailstorm Rod to kill them. You can even try killing them as a level 28 mage but it's proven to be a huge waste of supplies (requiring 100+ Mana Potions, 100+ Icicles and 60+Intense Healing Runes). Magma Set is recommended. You can also try wearing Dragon Necklace, although they will wear off really fast since you're almost constantly on fire while hunting them. From about level 90 it becomes possible to kill one using just a Hailstorm Rod and Ice Strike although it can still be wasteful until higher levels. To make visible cast Envenom.
Knights should attack them as soon as possible with a two-handed weapon since they are weak to physical attacks melee and in defence. Try to trap them in a corner, as they can be an annoyance when they are running around everywhere. When blocking Infernalists it is normally more efficient to use full offence rather than full defence. Running in their direction and using Groundshaker is a good way to make them visible again, you can also use Whirlwind Throw shortly after they go invisible or Inflict Wound and they will be visible again for a good time.
Paladin could solo a Infernalist by trapping it in a corner and attacking with Assassin Stars while using Divine Missile to break their invisibility, alternatively Divine Caldera when trapped or Holy Flash.