The nexus doesnt ask for what type of encryption it is when asking for a password, you just put your password in and thats it.
Do you use the wifi for anything else? Are you definately using the right password.
If you follow the above instructions, try removing the password to see if that makes a difference, just dont forget to put it back on.
Also try taking your nexus somewhere else to see if it connects to that.
My nexus connects at home with WPA2 but for some reason wont connect to my works wifi with no password, other people with android devices have trouble connecting as well but iphones connect straight away.