When using LDAP with SSL and a LDAP server which uses a self-signed SSL certificate normally no connection will be established. Therefor you have to allow such connections explicitly.
With Linux (e.g. Debian, Ubuntu) you have to add "TLS_REQCERT never" to your /etc/ldap/ldap.conf. On other distributions this config file may be located somewhere else.