That's correct! You will have to remove the batteries from the derailleurs in order to connect them to the charger.
Take a look at our Overview video of the SRAM RED eTap System for a tutorial on the removal, installation, and charging of the batteries and the indicator LEDs.