This is the same situation as when you want to shutdown your computer. You always need to shutdown it safely, and then remove the power supply.
The top button is made for that. You need to press the top button 3 seconds, wait that the LED switches to purple, and release the button. After a few seconds, the computer inside the robot (Raspberry Pi 3) will be turned off, and the LED will become red. At this time, you can safely pull the plug or switch the power switch.
Note that if you don’t shutdown the robot normally, the SD card on the Raspberry Pi 3 may be damaged, and you will need to re-flash it.