OpenLDAP Monitoring

OpenLDAP is an open-source Lightweight Directory Access Protocol (LDAP) server software. It provides a customizable and extensible directory service for storing and managing various types of information, such as user accounts, organizational structures, and configuration information. This information is organized and stored in a tree-like structure, facilitating fast query and retrieval. Many companies use OpenLDAP for centralized user account and permission management. Users can use OpenLDAP for unified authentication and authorization when accessing various company systems and resources, such as logging into corporate email, VPNs, GitLab, and Jenkins. Of course, it can also be used at home. For example, during the Lunar New Year holiday, when many people are around, you could set up a Linux virtual machine running OpenLDAP and connect it to your NAS, which seems quite useful.

This article won't cover the deployment and maintenance of OpenLDAP itself, as there's already a wealth of information available on that topic. It will focus on how to connect to OpenLDAP for monitoring.

The background is this: I joined a startup mid-year. A while ago, someone in the technical group reported that OpenLDAP login issues were impacting business operations. In an emergency, the relevant colleagues quickly restarted the OpenLDAP container service and resolved the issue. This situation is very common in small companies. Perhaps there weren't many services connected to it, or the maintainer had long since left, or there were no external impacts. A reboot might actually be the quickest and most effective solution. 😂

My wild guess that morning was that the default FD configuration of 1024 wasn't enough. With a large number of services, this situation will likely persist. It's a historical issue, understandably so. Perhaps there were only five people using it at the time, and the company only planned to close it after two years.

As long as one of the people and the program is running, it's fine. If you happen to arrive and it doesn't work, but you want peace of mind, what can you do? Add monitoring. I searched Google for "OpenLDAP monitoring," but there was surprisingly little information in Chinese. It was time to fill in the blanks.

Prometheus is a very common monitoring system these days. It's easy to use, has an active community, and personally feels much more hassle-free than it was years ago.

To be frank, the solution I chose was Prometheus + openldap_exporter [https://github.com/tomcz/openldap_exporter]. Unfortunately, the author has archived this repository and no longer maintains it, but it's still usable. Feel free to fork it and experiment with it yourself if you have the skills.

Since the original OpenLDAP was deployed using a containerized approach, connecting to the monitoring system was a bit cumbersome and challenging. The container repository is: https://github.com/osixia/docker-openldap

1. How to connect to the monitoring system

It is recommended that you find a test environment and then enter the container.

• Load monitoring module module

 1root@xxxxxxx:/# ldapsearch -Y EXTERNAL -H ldapi:/// -b "cn=module{0},cn=config" 
 2SASL/EXTERNAL authentication started 
 3SASL username: gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth 
 4SASL SSF: 0 
 5# extended LDIF 
 6# 
 7#LDAPv3 
 8# base <cn=module{0},cn=config> with scope subtree 
 9# filter: (objectclass=*) 
10# requesting: ALL 
11# 
12
13# module{0}, config 
14dn: cn=module{0},cn=config 
15objectClass: olcModuleList 
16cn: module{0} 
17olcModulePath: /usr/lib/ldap 
18olcModuleLoad: {0}back_mdb 
19olcModuleLoad: {1}memberof olcModuleLoad: {2}refint 
20
21# search result 
22search: 2 
23Result: 0 Success 
24
25#numResponses: 2 
26#numEntries: 1 
27root@9e93cb7f031d:/# ldapmodify -Y EXTERNAL -H ldapi:/// <<EOF 
28> dn: cn=module{0},cn=config 
29> changetype: modify 
30> add: olcModuleLoad 
31> olcModuleLoad: back_monitor 
32> EOF 
33SASL/EXTERNAL authentication started 
34SASL username: gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth 
35SASL SSF: 0 
36modifying entry "cn=module{0},cn=config" 
37
38root@9e93cb7f031d:/# ldapsearch -Y EXTERNAL -H ldapi:/// -b "cn=module{0},cn=config" 
39SASL/EXTERNAL authentication started 
40SASL username: gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth 
41SASL SSF: 0 
42# extended LDIF 
43# 
44#LDAPv3 
45# base <cn=module{0},cn=config> with scope subtree 
46# filter: (objectclass=*) 
47# requesting: ALL 
48# 
49
50# module{0}, config 
51dn: cn=module{0},cn=config 
52objectClass: olcModuleList 
53cn: module{0} 
54olcModulePath: /usr/lib/ldap 
55olcModuleLoad: {0}back_mdb 
56olcModuleLoad: {1}memberof 
57olcModuleLoad: {2}refint 
58olcModuleLoad: {3}back_monitor 
59
60# search result 
61search: 2 
62Result: 0 Success
63
64# numResponses: 2
65# numEntries: 1

The output of the above operation confirms that the monitoring module has been loaded.

• Adding a monitoring user

I replaced the password with xxx. Operation example:

 1root@4433d2af4a1d:/# slappasswd -s your_password
 2{SSHA}xxx
 3
 4ldapadd -Y EXTERNAL -H ldapi:/// <<EOF
 5dn: cn=monitor,dc=example,dc=org
 6objectClass: simpleSecurityObject
 7objectClass: organizationalRole
 8cn: monitor
 9description: LDAP monitor
10userPassword: {SSHA}xxx
11EOF

The first display shows no permissions. -w admin means writing with administrative permissions.

 1root@9e93cb7f031d:/# ldapadd -Y EXTERNAL -H ldapi:/// <<EOF 
 2> dn: cn=monitor,dc=example,dc=org 
 3> objectClass: simpleSecurityObject 
 4> objectClass: organizationalRole 
 5> cn: monitor 
 6> description: LDAP monitor 
 7> userPassword: {SSHA}3W8Q0HWiBVRyy2qX47qPVZQxQvZ3/R9+ 
 8> EOF 
 9SASL/EXTERNAL authentication started 
10SASL username: gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth 
11SASL SSF: 0 
12adding new entry "cn=monitor,dc=example,dc=org" 
13ldap_add: Insufficient access (50) 
14additional info: no write access to parent 
15
16root@9e93cb7f031d:/# ldapadd -x -D cn=admin,dc=example,dc=org -w 'admin' <<EOF
17> dn: cn=monitor,dc=example,dc=org
18> objectClass: simpleSecurityObject
19> objectClass: organizationalRole
20> cn: monitor
21> description: LDAP monitor
22> userPassword: {SSHA}xxx
23> EOF
24adding new entry "cn=monitor,dc=example,dc=org"

• Verify that the newly added user can read information normally

1root@9e93cb7f031d:/# ldapwhoami -x -D "cn=monitor,dc=example,dc=org" -w "your_password"
2dn:cn=monitor,dc=example,dc=org

As shown above, the query results indicate that the user information is correct. Enter the password you set above.

• Add a monitoring database

1ldapadd -Y EXTERNAL -H ldapi:/// <<EOF
2dn: olcDatabase={2}Monitor,cn=config
3objectClass: olcDatabaseConfig
4objectClass: olcMonitorConfig
5olcDatabase: {2}Monitor
6olcAccess: {0} to dn.subtree="cn=Monitor" by dn.base="cn=monitor,dc=example,dc=org" read by * none
7EOF

If the database exists:

1ldapmodify -Y EXTERNAL -H ldapi:/// <<EOF
2dn: olcDatabase={2}Monitor,cn=config
3changetype: modify
4replace:olcAccess
5olcAccess: {0}to dn.subtree="cn=Monitor" by dn.base="cn=monitor,dc=example,dc=org" read by * none
6EOF

 1root@9e93cb7f031d:/# ldapmodify -Y EXTERNAL -H ldapi:/// <<EOF 
 2dn: olcDatabase=Monitor,cn=config 
 3changetype: modify 
 4replace:olcAccess 
 5olcAccess: {0}to dn.subtree="cn=Monitor" by dn.base="cn=monitor,dc=example,dc=org" read by * none 
 6EOF 
 7SASL/EXTERNAL authentication started 
 8SASL username: gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth 
 9SASL SSF: 0 modifying entry "olcDatabase=Monitor,cn=config"
10ldap_modify: No such object (32)
11matched DN: cn=config

The above shows the error message when the database does not exist. If there is no database, it is definitely not possible to modify it directly (if it exists: ldapmodify, if it does not exist: ldapadd, which is reasonable and correct).

It looks very simple, right? If it is the first time to understand it, it will take a long time to understand the meaning of the above statement. I just read the official documentation + AI assistance + hands-on practice for a long time.

2. Deploy the monitoring component openldap_exporter

Isn't there a monitoring module above? Why is there another monitoring component here? It is different. The above is the server, and this is the client, which is used to read the information of the above monitoring module and then output it to Prometheus. The link is probably like this:

Prometheus <--- openldap_exporter monitoring component <--- OpenLDAP server monitoring module

This part is very simple and there is nothing much to say. See the operation:

 1xxx@xxx: cat /etc/systemd/system/openldap_exporter.service 
 2[Unit] 
 3Description=OpenLDAP Exporter 
 4After=network.target 
 5
 6[Service] 
 7Type=simple 
 8ExecStart=/opt/openldap_exporter/openldap_exporter --promAddr ":8080" --interval "10s" --config /opt/openldap_exporter/config.yml 
 9Restart=on-failure 
10
11[Install] 
12WantedBy=multi-user.target 
13
14xxx@xxx:~$ cat /opt/openldap_exporter/config.yml 
15--- 
16ldapUser: "cn=monitor,dc=example,dc=org" 
17ldapPass: "your_password"

If monitoring component If metrics aren't displaying correctly, you need to patiently troubleshoot. First, confirm that there are no server issues, then check the server logs, and even read the monitoring component's source code. Don't ask, I've gotten there myself. You're luckier than me; no Chinese user has written this before me, and I couldn't find it anyway.

3. Connecting to Prometheus

A simple example is as follows:

1- job_name: "openldap"
2static_configs:
3- targets: ["10.10.20.98:8080"]

I'm too lazy to take screenshots.

4. Connecting to Grafana

Monitoring dashboards are very useful, and leaders and superiors love them, as you know.

Unfortunately, there aren't many ready-made dashboards in the community, so I built one myself. If you'd like one, follow my WeChat official account and I'll send it to you. If you really follow me, I'll send it to you.

See the example images in the stress test below.

5. Stress Testing

As mentioned above, the initial diagnosis is that the problem is caused by the misconfigured number of FDs. This requires stress testing to verify.

The stress test program is released:

 1
 2import ldap
 3import time
 4
 5def main():
 6connections = []
 7target_connection_count = 1023
 8try:
 9while len(connections) < target_connection_count:
10try:
11# Try to connect to the openldap service on localhost:389
12conn = ldap.initialize("ldap://localhost:389")
13conn.simple_bind_s("cn=admin,dc=example,dc=org", "admin") # Please use a valid administrator DN and password
14connections.append(conn)
15result = conn.search_s("dc=example,dc=org", ldap.SCOPE_SUBTREE)
16print(result)
17time.sleep(0.5)
18except ldap.LDAPError as e:
19print(f"LDAP error occurred: {e}")
20break
21while True:
22# Keep the program running to maintain the file descriptor.
23pass
24except Exception as e:
25print(f"An error occurred: {e}")
26finally:
27# Close all established LDAP connections
28for conn in connections:
29try:
30conn.unbind()
31except ldap.LDAPError:
32pass
33
34if __name__ == "__main__":
35main()

sleep controls the speed of connection establishment and can be adjusted or removed. Note that the ldap package has several system dependencies; otherwise, it will not install.

Stress test results: