Add NTDS parser

[joost-j]

Adding a parser for the New Technology Directory Services Directory Information Tree (NTDS.dit) file, present on Windows Domain Controllers. This type of ESE database holds information related to an Active Directory environment.

Although multiple people have contributed to this tool within Fox-IT over the years, special acknowledgement goes to colleagues @Schamper and Aman Asarfi, whose work and research were invaluable. Performing in-depth investigation of the file format mainly comes from their end, and this pull request mainly puts all the code and knowledge gathered over the years into a proper Dissect-compatible format.

Several features include:

• Interacting with the NTDS database using Python functions (e.g. calling ntds.users()
• Interacting with the NTDS via LDAP queries (e.g. being able to call ntds.query("(objectClass=user)")
• Optimizations and caching to speed up the database lookups
• Testing most of the functionality above

Still work in progress:

• Create a wrapper tool around the NTDS class that outputs data in Bloodhound-supported format. Within Fox this goes by the namefoxhound. This tool can be added later on as well in a separate pull request.
• Add more objects and useful functions. E.g. a Domain object, and more functionality related to checking whether objects are related to each other. For example, a function where you could callObject.has_priv_over(obj2, GENERIC_ALL).
• Increase test coverage
• Adjusting NTDS.dit Plugin dissect.target#1347 or this code to be compatible
• Further code readability & style improvements

Related to fox-it/dissect.target#1348

[codecov]

Codecov Report

❌ Patch coverage is 0% with 1551 lines in your changes missing coverage. Please review.
✅ Project coverage is 0.00%. Comparing base (cba61a9) to head (5bb48bb).

Files with missing lines	Patch %	Lines
dissect/database/ese/ntds/database.py	0.00%	172 Missing ⚠️
dissect/database/ese/ntds/util.py	0.00%	141 Missing ⚠️
dissect/database/ese/ntds/objects/object.py	0.00%	139 Missing ⚠️
dissect/database/ese/ntds/schema.py	0.00%	107 Missing ⚠️
dissect/database/ese/ntds/objects/__init__.py	0.00%	105 Missing ⚠️
dissect/database/ese/ntds/sd.py	0.00%	101 Missing ⚠️
dissect/database/ese/ntds/query.py	0.00%	87 Missing ⚠️
dissect/database/ese/ntds/pek.py	0.00%	67 Missing ⚠️
dissect/database/ese/cursor.py	0.00%	41 Missing ⚠️
dissect/database/ese/ntds/ntds.py	0.00%	29 Missing ⚠️
... and 111 more

❗ There is a different number of reports uploaded between BASE (cba61a9) and HEAD (5bb48bb). Click for more details.

HEAD has 2 uploads less than BASE

Flag	BASE (cba61a9)	HEAD (5bb48bb)
unittests	14	12

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #8       +/-   ##
==========================================
- Coverage   82.05%   0.00%   -82.06%     
==========================================
  Files          31     146      +115     
  Lines        2346    3854     +1508     
==========================================
- Hits         1925       0     -1925     
- Misses        421    3854     +3433     

Flag	Coverage Δ
unittests	0.00% <0.00%> (-82.06%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Schamper]

Okay I think I've fueled my addiction enough for now. In my opinion we only need to add a few more unit tests based on GOAD (I already added a test file) and perhaps replace the existing tests with the GOAD file. Most of everything "basic" (and a little more) is already done. We can work on improving the "useful objects" later, as well as the additional tooling.

The only part I'm a little divided on are the bajillion source files for all objects, but I guess there's not really a nicer way to do that.