The piece with the black rubber gasket needs to be installed or water will come out of the spot where the handle would go.
Biggest mistake a person can make is to remove that part with the water pressure turned on.
Also I recommend you grow out your thumb nail, it looks too short