feat(llm): add support for langchain partner and community chat models #1085

Pouyanpi · 2025-04-02T10:27:07Z

Description

Introduces a new abstraction layer for initializing LLM models with:

refactor providers module
implement model initialization logic
clear separation between text completion and chat models
proper error handling with dedicated ModelInitializationError
consistent provider name handling and discovery
type safety improvements for LangChain models

Added tests, increase number of providers related tests from 2 to 69.

Requires:

Following PRs must get merged first (those are cherry-picked to this PR)
#1084
#1083

Resolves

#1055
#1044
#1070
#124
#902
#753
#520
#865
might resolve #933
need to check #992

which discussions?
which issues?

TODO:

Add more tests (pending test restructuring PR)
Add chat and completion mode to Model class -> allows user to avoid our selection order
Figure out what to do with provider name inconsistencies -> mention in the docs in case of conflict we prefer chat completions over text and if the user wants to use the text completion mode ' must be set to text`
Add a fzf search to allow the user find the provider name (engine) easily feat(cli): add fuzzy find provider capability to cli #1088
Add fzf search to CLI feat(cli): add fuzzy find provider capability to cli #1088

nemoguardrails/llm/models/langchain_initializer.py

nemoguardrails/llm/providers/providers.py

nemoguardrails/rails/llm/config.py

nemoguardrails/rails/llm/llmrails.py

tests/test_configs/with_custom_llm/config.yml

tests/test_custom_llm.py

tests/test_providers.py

tgasser-nv

Looks good, approving. We need to define a subset of models for each of the initializer types and add them to integration tests. Then we can add documentation saying that we support every Langchain model out there, and list the models we actively test against. Also we need to add integration tests calling into these providers (could you create a follow-on task for that?)

nemoguardrails/llm/models/langchain_initializer.py

nemoguardrails/rails/llm/config.py

nemoguardrails/llm/providers/providers.py

Introduces a new abstraction layer for initializing LLM models with: - refactor providers module - implement model initialization logic - clear separation between text completion and chat models - proper error handling with dedicated ModelInitializationError - consistent provider name handling and discovery - type safety improvements for LangChain models fix: remove configurable fix: remove lru_cache feat(config): add mode field to Model feat(llmrails): integrate mode field from Model feat(initializer): integrate mode field from Model feat(initializer): add mode support to LangChain model initialization - introduced `ModelInitStrategy` class to encapsulate initialization strategies with supported modes. - updated `try_strategy` to skip strategies that do not support the requested mode. - enhanced logging - adjusted strategies to include supported modes for better control over initialization logic. fix(providers): change to log.debug refactor: move deprecated llm providers to a list fix: remove Langchain warnings refactor: improve langchain initialization logic and names add validation and improve model initialization add custom_chat_model registeration test add tests skip failing test as it should be run in LIVE mode Update nemoguardrails/rails/llm/config.py Signed-off-by: Pouyan <13303554+Pouyanpi@users.noreply.github.com>

correct regex pattern in test for model initialization

Pouyanpi force-pushed the feat/lc-chat-providers branch from 69e3889 to 1c72e4a Compare April 2, 2025 10:29