How-to use Python LDAP paged results control to handle large LDAP searches

python-ldap evolved through the years and a lot of the information still found on the Internet regarding LDAP-paged-results only works with older versions of Python. The purpose of this post is to illustrate the use of the LDAP paged-results-control with Python 2.7 and higher.

Snippet Objective

Most LDAP servers limit the number of results returned by searches, this code uses an LDAP paged results control to overcome the limitation.

Snippet Results

Generates a list of all LDAP queried data and stores it in the results variable.

RFC 2696: pagedResultsControl

An LDAP client application that needs to control the rate at which results are returned MAY specify on the searchRequest a pagedResultsControl …

from ldap.controls import SimplePagedResultsControl

… cookie set to the zero-length string.

COOKIE = ''

 Active Directory maximum page size

http://technet.microsoft.com/en-us/library/cc770976(v=ws.10).aspx

For Active Directory on Windows Server 2003-2008 the default maximum page size supported for LDAP responses is 1,000 records.

PAGE_SIZE = 1000

RFC 2696: Criticality

… if the client requested it as critical”

CRITICALITY = True

“… the server MUST return an error of unsupportedCriticalExtension …if criticality is false then “otherwise the server SHOULD ignore the control.

With python-ldap if the LDAP-page-control is not supported it should theoretically raise the ldap.UNAVAILABLE_CRITICAL_EXTENSION exception, however I can’t find any example code where anyone handles this exception and all of the LDAP servers I use support the pagedResultsControl. I don’t normally like to use a generic exception handler, but until I can test out the exception handling in more detail it currently just displays the raw error if it encounters an exception during the LDAP search.

Note

The Following code is just a snippet and assumes that LDAPObject is an established ldap connection and that LDAP search parameters are already set, i.e. LDAP_CONN, BASE_DN, LDAP_SCOPE, FILTERSTR & ATTRIBS

import sys
import ldap
from ldap.controls import SimplePagedResultsControl

COOKIE = ''
PAGE_SIZE = 1000
CRITICALITY = True

results = []
first_pass = True
pg_ctrl = SimplePagedResultsControl(
        CRITICALITY, PAGE_SIZE, COOKIE)
while first_pass or pg_ctrl.cookie:
    first_pass = False
    try:
        msgid = LDAP_CONN.search_ext(
                BASE_DN, LDAP_SCOPE,
                FILTERSTR, ATTRIBS,
                serverctrls=[pg_ctrl])
    except:
        sys.err.write("ERROR: \n" +
                      str(sys.exc_info()[0]))
        sys.exit(1)
    result_type, data, msgid, serverctrls = \
            ldap.result3(msgid)
    pg_ctrl.cookie = serverctrls[0].cookie
    results += [i[0] for i in data]
Advertisements

4 Responses to “How-to use Python LDAP paged results control to handle large LDAP searches”

  1. Christopher Says:

    Thanks for this. The other examples I could find were incompatible with the current version of python-ldap and I had trouble debugging.

  2. graingert Says:

    ldap_.result3(msgid) should be ldap.result3(msgid)

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s


%d bloggers like this: