仅用于站内搜索，没有排版格式，具体信息请跳转上方微信公众号内链接
Datawhale推荐
开源项目：llm-engineer-toolkit
编辑：Coggle数据科学团队
在大语言模型（LLM）迅速发展的今天，开发者们面临着海量的资源和工具选择。如何高效地筛选和利用这些资源，成为了每一个LLM开发者的关键任务。今天，我们要介绍的GitHub仓库——LLMEngineerToolkit，或许能成为你的得力助手！
https ://github.com/KalyanKS-NLP/llm-engineer-toolkit
这个由KalyanKS-NLP创建的仓库，精心整理了超过120个LLM相关的库，并按照类别进行了分类。无论是训练、推理、应用开发，还是数据提取、安全评估，你都能在这里找到对应的工具。
🚀LLMTraining：专注于LLM训练和微调的工具，帮助你更快、更高效地优化模型。
🧱LLMApplicationDevelopment：从框架到多API接入，再到缓存和低代码开发，为应用开发提供全方位支持。
🩸LLMRAG：Retrieval-AugmentedGeneration（检索增强生成）相关的库，提升模型的知识检索能力。
🟩LLMInference：推理加速和优化工具，让模型运行更流畅。
🚧LLMServing：模型部署和推理服务的解决方案。
📤LLMDataExtraction：数据提取工具，帮助你从各种来源获取高质量数据。
🌠LLMDataGeneration：生成合成数据，丰富你的训练集。
💎LLMAgents：构建智能代理，实现自动化任务和多代理协作。
⚖️LLMEvaluation：评估工具，确保模型性能达到预期。
🔍LLMMonitoring：监控模型运行状态，及时发现并解决问题。
📅LLMPrompts：优化和管理提示词，提升模型输出质量。
📝LLMStructuredOutputs：生成结构化输出，让模型结果更易用。
🛑LLMSafetyandSecurity：保障模型的安全性和可靠性。
💠LLMEmbeddingModels：提供先进的文本嵌入模型。
❇️Others：其他实用工具，涵盖更多开发场景。
Library
Description
unsloth
Fine-tuneLLMsfasterwithlessmemory.
PEFT
State-of-the-artParameter-EfficientFine-Tuninglibrary.
TRL
Traintransformerlanguagemodelswithreinforcementlearning.
Transformers
Axolotl
Tooldesignedtostreamlinepost-trainingforvariousAImodels.
LLMBox
LitGPT
Trainandfine-tuneLLMlightningfast.
Mergoo
AlibraryforeasilymergingmultipleLLMexperts,andefficientlytrainthemergedLLM.
Llama-Factory
EasyandefficientLLMfine-tuning.
Ludwig
Low-codeframeworkforbuildingcustomLLMs,neuralnetworks,andotherAImodels.
Txtinstruct
Aframeworkfortraininginstruction-tunedmodels.
Lamini
AnintegratedLLMinferenceandtuningplatform.
XTuring
RL4LMs
AmodularRLlibrarytofine-tunelanguagemodelstohumanpreferences.
DeepSpeed
torchtune
APyTorch-nativelibraryspecificallydesignedforfine-tuningLLMs.
PyTorchLightning
Alibrarythatoffersahigh-levelinterfaceforpretrainingandfine-tuningLLMs.
Library
Description
LangChain
LangChainisaframeworkfordevelopingapplicationspoweredbylargelanguagemodels(LLMs).
LlamaIndex
LlamaIndexisadataframeworkforyourLLMapplications.
HayStack
Promptflow
Asuiteofdevelopmenttoolsdesignedtostreamlinetheend-to-enddevelopmentcycleofLLM-basedAIapplications.
Griptape
AmodularPythonframeworkforbuildingAI-poweredapplications.
Weave
WeaveisatoolkitfordevelopingGenerativeAIapplications.
LlamaStack
BuildLlamaApps.
Library
Description
DataPrepKit
Library
Description
LiteLLM
Librarytocall100+LLMAPIsinOpenAIformat.
AIGateway
ABlazingFastAIGatewaywithintegratedGuardrails.Routeto200+LLMs,50+AIGuardrailswith1fast&friendlyAPI.
Library
Description
RouteLLM
Library
Description
mem0
TheMemorylayerforyourAIapps.
Memoripy
Letta(MemGPT)
Memobase
Auserprofile-basedmemorysystemdesignedtobringlong-termusermemorytoyourGenerativeAIapplications.
Library
Description
Streamlit
Gradio
Buildandsharedelightfulmachinelearningapps,allinPython.
AISDKUI
Buildchatandgenerativeuserinterfaces.
AI-Gradio
CreateAIappspoweredbyvariousAIproviders.
Simpleaichat
Pythonpackageforeasilyinterfacingwithchatapps,withrobustfeaturesandminimalcodecomplexity.
Chainlit
Buildproduction-readyConversationalAIapplicationsinminutes.
Library
Description
LangFlow
Library
Description
GPTCache
Library
Description
FastGraphRAG
Chonkie
RAGchunkinglibrarythatislightweight,lightning-fast,andeasytouse.
RAGChecker
AFine-grainedFrameworkForDiagnosingRAG.
RAGtoRiches
Build,scale,anddeploystate-of-the-artRetrieval-AugmentedGenerationapplications.
BeyondLLM
SQLite-Vec
AvectorsearchSQLiteextensionthatrunsanywhere!
fastRAG
FlashRAG
APythonToolkitforEfficientRAGResearch.
Llmware
UnifiedframeworkforbuildingenterpriseRAGpipelineswithsmall,specializedmodels.
Rerankers
AlightweightunifiedAPIforvariousrerankingmodels.
Vectara
BuildAgenticRAGapplications.
Library
Description
LLMCompressor
Transformers-compatiblelibraryforapplyingvariouscompressionalgorithmstoLLMsforoptimizeddeployment.
LightLLM
vLLM
High-throughputandmemory-efficientinferenceandservingengineforLLMs.
torchchat
RunPyTorchLLMslocallyonservers,desktop,andmobile.
TensorRT-LLM
TensorRT-LLMisalibraryforoptimizingLargeLanguageModel(LLM)inference.
WebLLM
High-performanceIn-browserLLMInferenceEngine.
Library
Description
Langcorn
ServingLangChainLLMappsandagentsautomagicallywithFastAPI.
LitServe
Library
Description
Crawl4AI
Open-sourceLLMFriendlyWebCrawler&Scraper.
ScrapeGraphAI
Docling
Doclingparsesdocumentsandexportsthemtothedesiredformatwitheaseandspeed.
LlamaParse
GenAI-nativedocumentparserthatcanparsecomplexdocumentdataforanydownstreamLLMusecase(RAG,agents).
PyMuPDF4LLM
PyMuPDF4LLMlibrarymakesiteasiertoextractPDFcontentintheformatyouneedforLLM&RAGenvironments.
Crawlee
Awebscrapingandbrowserautomationlibrary.
MegaParse
Parserforeverytypeofdocument.
ExtractThinker
DocumentIntelligencelibraryforLLMs.
Library
Description
DataDreamer
fabricator
Aflexibleopen-sourceframeworktogeneratedatasetswithlargelanguagemodels.
Promptwright
SyntheticDatasetGenerationLibrary.
EasyInstruct
AnEasy-to-useInstructionProcessingFrameworkforLargeLanguageModels.
Library
Description
CrewAI
Frameworkfororchestratingrole-playing,autonomousAIagents.
LangGraph
Buildresilientlanguageagentsasgraphs.
Agno
BuildAIAgentswithmemory,knowledge,tools,andreasoning.ChatwiththemusingabeautifulAgentUI.
AgentsSDK
BuildagenticappsusingLLMswithcontext,tools,handofftootherspecializedagents.
AutoGen
Anopen-sourceframeworkforbuildingAIagentsystems.
Smolagents
Librarytobuildpowerfulagentsinafewlinesofcode.
PydanticAI
PythonagentframeworktobuildproductiongradeapplicationswithGenerativeAI.
BeeAI
Buildproduction-readymulti-agentsystemsinPython.
gradio-tools
APythonlibraryforconvertingGradioappsintotoolsthatcanbeleveragedbyanLLM-basedagenttocompleteitstask.
Composio
ProductionReadyToolsetforAIAgents.
AtomicAgents
BuildingAIagents,atomically.
Memary
OpenSourceMemoryLayerForAutonomousAgents.
BrowserUse
MakewebsitesaccessibleforAIagents.
OpenWebAgent
AnOpenToolkittoEnableWebAgentsonLargeLanguageModels.
Lagent
AlightweightframeworkforbuildingLLM-basedagents.
LazyLLM
ALow-codeDevelopmentToolForBuildingMulti-agentLLMsApplications.
Swarms
TheEnterprise-GradeProduction-ReadyMulti-AgentOrchestrationFramework.
ChatArena
Swarm
Educationalframeworkexploringergonomic,lightweightmulti-agentorchestration.
AgentStack
ThefastestwaytobuildrobustAIagents.
Archgw
IntelligentgatewayforAgents.
Flow
AlightweighttaskengineforbuildingAIagents.
AgentOps
PythonSDKforAIagentmonitoring.
Langroid
Multi-Agentframework.
Agentarium
FrameworkforcreatingandmanagingsimulationspopulatedwithAI-poweredagents.
Upsonic
ReliableAIagentframeworkthatsupportsMCP.
Library
Description
Ragas
RagasisyourultimatetoolkitforevaluatingandoptimizingLargeLanguageModel(LLM)applications.
Giskard
Open-SourceEvaluation&TestingforML&LLMsystems.
DeepEval
LLMEvaluationFramework
Lighteval
All-in-onetoolkitforevaluatingLLMs.
Trulens
EvaluationandTrackingforLLMExperiments
PromptBench
Aunifiedevaluationframeworkforlargelanguagemodels.
LangTest
EvalPlus
ArigorousevaluationframeworkforLLM4Code.
FastChat
Anopenplatformfortraining,serving,andevaluatinglargelanguagemodel-basedchatbots.
judges
AsmalllibraryofLLMjudges.
Evals
EvalsisaframeworkforevaluatingLLMsandLLMsystems,andanopen-sourceregistryofbenchmarks.
AgentEvals
Evaluatorsandutilitiesforevaluatingtheperformanceofyouragents.
LLMBox
Opik
Anopen-sourceend-to-endLLMDevelopmentPlatformwhichalsoincludesLLMevaluation.
Library
Description
MLflow
Anopen-sourceend-to-endMLOps/LLMOpsPlatformfortracking,evaluating,andmonitoringLLMapplications.
Opik
Anopen-sourceend-to-endLLMDevelopmentPlatformwhichalsoincludesLLMmonitoring.
LangSmith
Providestoolsforlogging,monitoring,andimprovingyourLLMapplications.
Weights&Biases(W&B)
W&BprovidesfeaturesfortrackingLLMperformance.
Helicone
Evidently
Anopen-sourceMLandLLMobservabilityframework.
Phoenix
Anopen-sourceAIobservabilityplatformdesignedforexperimentation,evaluation,andtroubleshooting.
Observers
ALightweightLibraryforAIObservability.
Library
Description
PCToolkit
AUnifiedPlug-and-PlayPromptCompressionToolkitofLargeLanguageModels.
SelectiveContext
SelectiveContextcompressesyourpromptandcontexttoallowLLMs(suchasChatGPT)toprocess2xmorecontent.
LLMLingua
LibraryforcompressingpromptstoaccelerateLLMinference.
betterprompt
TestsuiteforLLMpromptsbeforepushingthemtoproduction.
Promptify
PromptSource
PromptSourceisatoolkitforcreating,sharing,andusingnaturallanguageprompts.
DSPy
DSPyistheopen-sourceframeworkforprogramming—ratherthanprompting—languagemodels.
Py-priompt
Promptdesignlibrary.
Promptimizer
Promptoptimizationlibrary.
Library
Description
Instructor
XGrammar
Anopen-sourcelibraryforefficient,flexible,andportablestructuredgeneration.
Outlines
Robust(structured)textgeneration
Guidance
Guidanceisanefficientprogrammingparadigmforsteeringlanguagemodels.
LMQL
Alanguageforconstraint-guidedandefficientLLMprogramming.
Jsonformer
ABulletproofWaytoGenerateStructuredJSONfromLanguageModels.
Library
Description
JailbreakEval
Acollectionofautomatedevaluatorsforassessingjailbreakattempts.
EasyJailbreak
Aneasy-to-usePythonframeworktogenerateadversarialjailbreakprompts.
Guardrails
Addingguardrailstolargelanguagemodels.
LLMGuard
TheSecurityToolkitforLLMInteractions.
AuditNLG
NeMoGuardrails
Garak
LLMvulnerabilityscanner
DeepTeam
TheLLMRedTeamingFramework
Library
Description
Sentence-Transformers
State-of-the-ArtTextEmbeddings
Model2Vec
FastState-of-the-ArtStaticEmbeddings
TextEmbeddingInference
Library
Description
TextMachina
LLMReasoners
Alibraryforadvancedlargelanguagemodelreasoning.
EasyEdit
AnEasy-to-useKnowledgeEditingFrameworkforLargeLanguageModels.
CodeTF
CodeTF:One-stopTransformerLibraryforState-of-the-artCodeLLM.
spacy-llm
pandas-ai
Chatwithyourdatabase(SQL,CSV,pandas,polars,MongoDB,NoSQL,etc.).
LLMTransparencyTool
Anopen-sourceinteractivetoolkitforanalyzinginternalworkingsofTransformer-basedlanguagemodels.
Vanna
ChatwithyourSQLdatabase.AccurateText-to-SQLGenerationviaLLMsusingRAG.
mergekit
Toolsformergingpretrainedlargelanguagemodels.
MarkLLM
AnOpen-SourceToolkitforLLMWatermarking.
LLMSanitize
Anopen-sourcelibraryforcontaminationdetectioninNLPdatasetsandLargeLanguageModels(LLMs).
Annotateai
AutomaticallyannotatepapersusingLLMs.
LLMReasoner
MakeanyLLMthinklikeOpenAIo1andDeepSeekR1.
“点赞”三连↓
