Mine also makes a very high-pitched noise when it is switched on. This is most likely a coil whine problem: some components charge and discharge at an audible frequency and this charging/discharging introduces a slight physical movement.
If the noise bothers you too much, you should probably switch to another product.