Architecture question

Show HN: Architecture question: running an LLM as core infrastructure

Architecture question is a company in show hn launch.

Profile

I've been experimenting with running an LLM not as a chatbot but as the core runtime of a business system, and I'm curious how others approach this.The idea is that the model doesn't just answer questions but orchestrates tools and interacts with real application logic.The architecture I'm currently testing includes:Runtimetool orchestration parallel tool execution loop detection circuit breaker / timeout guards token budgeting Contextcontext compression dynamic token ceiling Cachingdeterministic LLM response cache semantic cache using pgvector Memoryshort-term session memory longer-term semantic memory Evaluationprompt evaluation set to test tool reasoning and failures I'm trying to figure

As a business in show hn launch, Architecture question is listed here with the public details available about it.

It is recorded at the Show HN launch stage.

Website: automazionezeli.com