llm
an archive of posts with this tag
-
Running a local LLM with llama.cpp: build, download, serve, benchmark
A minimal llama.cpp workflow for running GGUF models on your own machine — from the cmake build to an OpenAI-compatible server and llama-bench.
-
Pydantic with llm.with_structured_output — pitfalls and safeguards
Combining Pydantic models with LangChain's with_structured_output cleanly — avoiding prompt collisions, validation errors, and runaway evaluation loops.