Add a first-party Python DirectoryLoader with a per-extension loader map (JS parity)

[false200]

Submission checklist

• This is a feature request, not a bug report or usage question.
• I added a clear and descriptive title that summarizes the feature request.
• I used the GitHub search to find a similar feature request and didn't find it.
• I checked the LangChain documentation and API reference to see if this feature already exists.
• This is not related to the langchain-community package.

Package (Required)

• langchain
• langchain-openai
• langchain-anthropic
• langchain-classic
• langchain-core
• langchain-model-profiles
• langchain-tests
• langchain-text-splitters
• langchain-chroma
• langchain-deepseek
• langchain-exa
• langchain-fireworks
• langchain-groq
• langchain-huggingface
• langchain-mistralai
• langchain-nomic
• langchain-ollama
• langchain-openrouter
• langchain-perplexity
• langchain-qdrant
• langchain-xai
• Other / not sure / general

Feature Description

Python has no first-party way to load a mixed folder (pdf + txt + csv + docx) with the correct parser per file in one call.

JS DirectoryLoader already takes a map of extension → loader factory, and that is the documented JS API:

const loader = new DirectoryLoader("./docs", {
  ".pdf": (path) => new PDFLoader(path),
  ".txt": (path) => new TextLoader(path),
  ".csv": (path) => new CSVLoader(path),
  ".docx": (path) => new DocxLoader(path),
});

Python DirectoryLoader only accepts a single loader_cls, so every RAG app has to spin up one loader per extension and merge the results. That class also lives in langchain-community, which is being sunset (langchain-community#674). A matching suffix_loader_map PR was closed as part of that wind-down (langchain-community#571).

Please add a first-party Python DirectoryLoader (in langchain or langchain-core) that:

1. Accepts loaders={".pdf": PyPDFLoader, ".txt": TextLoader, ...} (classes or factories).
2. Keeps a single-loader path for backward compatibility (loader_cls=).
3. Skips unknown extensions with a warning (or silent_errors=True), instead of failing the whole folder.
4. Matches JS behavior closely enough that the docs can show the same example in both languages.

This is JS/Python parity for the most common RAG ingest path: “here is a docs folder, load it.”

Use Case

I'm building RAG over a real knowledge folder, not a simple dataset. The folder has PDFs, notes (.txt/.md), spreadsheets (.csv), and Word files (.docx).

Today in Python I have to write:

pdfs = DirectoryLoader("docs", glob="**/*.pdf", loader_cls=PyPDFLoader).load()
txts = DirectoryLoader("docs", glob="**/*.txt", loader_cls=TextLoader).load()
csvs = DirectoryLoader("docs", glob="**/*.csv", loader_cls=CSVLoader).load()
docs = pdfs + txts + csvs

That is easy to get wrong (missed globs, duplicated files, one bad PDF aborting a run) and it is the first thing every RAG tutorial hits.

JS users already write one call with a loader map. Python users should be able to do the same:

loader = DirectoryLoader(
    "docs",
    loaders={
        ".txt": TextLoader,
        ".pdf": PyPDFLoader,
        ".csv": CSVLoader,
        ".docx": Docx2txtLoader,
    },
    silent_errors=True,
)
docs = loader.load()

With langchain-community sunsetting, this should live in first-party LangChain so the standard ingest path does not disappear.

Proposed Solution

Add DirectoryLoader to langchain (or langchain-core) with a backward-compatible API:

• loader_cls=... stays valid (current Python behavior).
• loaders={".ext": Loader | Callable[[str], BaseLoader]} is the new path.
• Lookup is case-insensitive on path.suffix.
• Unknown suffixes: warn + skip when silent_errors=True, else raise a clear error naming the file.
• Optional later: load_with_errors() -> (docs, [(path, exc), ...]).

from langchain.document_loaders import DirectoryLoader, TextLoader
from langchain_community.document_loaders import PyPDFLoader, CSVLoader  # or partner pkgs

loader = DirectoryLoader(
    "docs",
    loaders={
        ".txt": TextLoader,
        ".md": TextLoader,
        ".pdf": PyPDFLoader,
        ".csv": lambda path: CSVLoader(path, encoding="utf-8"),
    },
    silent_errors=True,
    show_progress=True,
)
docs = loader.load()

I can implement this and add unit tests (mixed folder, unknown suffix, old loader_cls still works) if a maintainer assigns the issue.

Alternatives Considered

I've tried using multiple DirectoryLoader calls and concatenating the results.

Alternative approaches I considered:

1. Multiple DirectoryLoader calls + list concat - works, but it is boilerplate every RAG app copies.
2. MergedDataLoader - still requires constructing one loader per type/file.
3. GenericLoader.from_filesystem - one parser for the whole tree, not a per-extension map.
4. Patching langchain-community DirectoryLoader - community is sunsetting; PR Is human_prefix necessary for ConversationalAgent? #571 (suffix_loader_map) was closed for that reason.
5. silent_errors=True - already exists; it skips failures, it does not pick the right loader per file type.

These don't work as a first-class solution because they either stay in a sunset package, or they still force users to write one loader per file type.

Additional Context

Related:

• DirectoryLoader for different file types #9749 (Python users asking for the JS loader map; closed as stale with a workaround)
• https://docs.langchain.com/oss/javascript/integrations/document_loaders/file_loaders/directory (JS API)
• Sunsetting langchain-community langchain-community#674 (community sunset)
• fix: add suffix_loader_map to DirectoryLoader for unsupported file types langchain-community#571 (suffix_loader_map closed during sunset)

I am happy to open a PR once a maintainer approves the approach and assigns this issue.

Social handles (optional)

No response

[AnakinSolo1983]

I would like to work on this, if you don't mind.