chunking

Here are 242 public repositories matching this topic...

jiesutd / NCRFpp

NCRF++, a Neural Sequence Labeling Toolkit. Easy use to any sequence labeling tasks (e.g. NER, POS, Segmentation). It includes character LSTM/CNN, word LSTM/CNN and softmax/CRF components.

natural-language-processing crf cnn pytorch artificial-intelligence lstm named-entity-recognition neural-networks chunking ner char-rnn part-of-speech-tagger sequence-labeling nbest lstm-crf char-cnn

Updated Jun 30, 2022
Python

systemd / casync

Star

Content-Addressable Data Synchronization Tool

http synchronization download tar upload archive file-system chunking delivery

Updated Dec 21, 2023
C

smooks / smooks

Sponsor

Star

An extensible Java framework for building event-driven applications that break up XML and non-XML data into chunks for data integration

java etl analytics xml pipelines sax stream-processing event-driven chunking smooks enterprise-integration

Updated Jun 9, 2025
Java

mirth / chonky

Star

Fully neural approach for text chunking

ai ml chunking rag text-splitter llms semantic-chunking

Updated Apr 27, 2025
Python

folbricht / desync

Star

Alternative casync implementation

golang synchronization archive chunking casync

Updated Apr 17, 2025
Go

isaacus-dev / semchunk

Star

A fast, lightweight and easy-to-use Python library for splitting text into semantically meaningful chunks.

python nlp text splitting chunking text-chunking text-splitting semantic-chunking isaacus

Updated Jun 9, 2025
Python

lazyFrogLOL / llmdocparser

Star

A package for parsing PDFs and analyzing their content using LLMs.

nlp ocr chunking document-analysis pdf-parser pdfparser rag llm text-chunking

Updated Aug 6, 2024
Python

microsoft / rag-experiment-accelerator

Star

The RAG Experiment Accelerator is a versatile tool designed to expedite and facilitate the process of conducting experiments and evaluations using Azure Cognitive Search and RAG pattern.

experiment information-retrieval azure evaluation indexing openai sparse vectors chunking acs embedding dense rag llm genai

Updated Apr 8, 2025
Python

26hzhang / neural_sequence_labeling

Star

A TensorFlow implementation of Neural Sequence Labeling model, which is able to tackle sequence labeling tasks such as POS Tagging, Chunking, NER, Punctuation Restoration and etc.

tensorflow python3 named-entity-recognition chunking punctuation sequence-labeling pos-tagger sentence-boundary-detection lstm-networks

Updated Nov 27, 2018
Python

zeroentropy-ai / zchunk

Star

A new chunking strategy developed by ZeroEntropy for general semantic chunking using Llama-70B.

retrieval chunking llm

Updated Jan 28, 2025
Python

jparkerweb / semantic-chunking

Star

🍱 semantic-chunking ⇢ semantically create chunks from large document for passing to LLM workflows

vector embeddings chunking text-splitter llm text-chunking text-splitting semantic-chunking

Updated Feb 28, 2025
JavaScript

jordicenzano / go-ts-segmenter

Star

Live TS segmenter and HLS manifest creation in Go

golang video hls chunk chunking transport-stream chunked lhls

Updated Oct 31, 2021
Go

safakan / TalkWithYourFiles

Star

An LLM GUI application; enables you to interact with your files, offering dynamic parameters that can modify response behavior during runtime.

python docker embeddings openai question-answering text-processing chunking strategy-pattern factory-pattern dependency-inversion-principle similarity-search faiss streamlit langchain openai-chatgpt vectorstore

Updated Dec 7, 2023
Python

swarmauri / swarmauri-sdk

Star

a modular multimodal framework for ai applications

nlp modular tools ai parsing tooling metrics orchestration-framework monorepo orchestration factories vectors chunking measures agents llm-framework

Updated Jun 14, 2025
Python

xtabbas / The-Ultimate-Boilerplate

Star

webpack 2, react hotloader 3, react router v4, code splitting and more

react redux boilerplate webpack hot-reloading server-side-rendering chunking react-router-v4 reactrouter

Updated Sep 22, 2017
JavaScript

Sammyjo20 / laravel-chunkable-jobs

Sponsor

Star

📑 Split Laravel jobs into multiple separate job chunks

php laravel jobs chunking hacktoberfest

Updated May 14, 2024
PHP

esastack / esa-restclient

Star

An asynchronous event-driven HTTP client based on netty.

asynchronous netty https http2 filter httpclient haproxy retry interceptor chunking h2c

Updated Jul 6, 2022
Java

neondatabase-labs / pgrag

Star

Postgres extensions to support end-to-end Retrieval-Augmented Generation (RAG) pipelines

postgresql embeddings chunking rag pgrx

Updated May 8, 2025
Rust

ronomon / deduplication

Star

Fast multi-threaded content-dependent chunking deduplication for Buffers in C++ with a reference implementation in Javascript. Ships with extensive tests, a fuzz test and a benchmark.

nodejs chunking deduplication content-dependent

Updated Mar 1, 2020
JavaScript

Koziev / GrammarEngine

Star

Грамматический Словарь Русского Языка (+ английский, японский, etc)

nlp machine-learning syntax-parser chunking lemmatizer nlp-parsing nlp-library part-of-speech-tagger morphological-analysis russian-morphology morphological-analyser lemmatization

Updated Aug 10, 2020
C++

Improve this page

Add a description, image, and links to the chunking topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the chunking topic, visit your repo's landing page and select "manage topics."

Learn more

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chunking

Here are 242 public repositories matching this topic...

jiesutd / NCRFpp

systemd / casync

smooks / smooks

mirth / chonky

folbricht / desync

isaacus-dev / semchunk

lazyFrogLOL / llmdocparser

microsoft / rag-experiment-accelerator

26hzhang / neural_sequence_labeling

zeroentropy-ai / zchunk

jparkerweb / semantic-chunking

jordicenzano / go-ts-segmenter

safakan / TalkWithYourFiles

swarmauri / swarmauri-sdk

xtabbas / The-Ultimate-Boilerplate

Sammyjo20 / laravel-chunkable-jobs

esastack / esa-restclient

neondatabase-labs / pgrag

ronomon / deduplication

Koziev / GrammarEngine

Improve this page

Add this topic to your repo