# Give your AI workers a bounded place to run

> Separate model inference from orchestration, size the actual worker and limit what its tools can change before putting an AI workflow on a server.

Canonical human page: https://swissoctet.com/insights/give-ai-workers-a-bounded-runtime/

SwissOctet Editorial. Topic: compute. Level: Applied. Reading time: 4 minutes.

## Direct answer
An AI workflow needs resources for its actual tools and concurrency, and permissions that remain enforceable outside the model. Remote inference and local inference lead to different CPU, memory, storage and GPU decisions.

## Name the job before choosing a machine
An AI worker combines a model with software that receives tasks, gathers context and calls tools. A scheduled document classifier, an internal search assistant and a browser-driven workflow have different operating needs. Write down the input, the expected output, the systems it may contact and the point where a person approves a consequential action. This small specification tells you more about the host than the label agent.

Start with one repeatable job and a narrow account. A worker that drafts a response needs less authority than one that sends it. Keep those stages separate in the application. The server should support the approved workflow without turning every generated instruction into an executable operation. Completion means a verified result or a clear failure state, rather than an indefinitely running loop.

## Separate a remote model from a local model
When the model runs behind an external API, the VPS mainly hosts the queue, application, tools and persistent state. Its memory demand comes from those processes, including any browsers or document parsers. A GPU on this machine does not accelerate a model running elsewhere. Record which content crosses the API boundary and evaluate that provider independently from the location of the worker.

Local inference adds model storage, model memory and runtime compatibility to the budget. Inference tools such as Ollama expose information about CPU or GPU placement and concurrent requests. Test the chosen model, context length and concurrency together. A larger context or more simultaneous jobs can raise memory demand; a model file size alone is an incomplete sizing method. Check the runtime's supported hardware before selecting a GPU configuration.

| Worker pattern | Main resource question | Boundary to review |
| --- | --- | --- |
| Remote model API | Application and tool memory | Content sent to the model provider |
| Local model | Model, context and working memory | Runtime and device compatibility |
| Browser workflow | Peak browser processes and temporary files | Web content and permitted actions |

## Measure a representative queue
Prepare a small set of tasks with normal and unusually large inputs. Measure peak memory, elapsed processing time, temporary storage and the number of external calls. Include failure cases: an unavailable API, a malformed document and a tool timeout. These measurements are a planning exercise for your software, not a benchmark or performance promise for a SwissOctet plan.

Set a maximum number of active jobs and a maximum execution time. Queue work that cannot start safely and reject overload in a predictable way. Retrying every failure immediately can multiply both server pressure and API spending. Use a bounded retry policy and an identifier that prevents the same completed action being applied twice. Reserve capacity for the operating system, monitoring and maintenance while jobs are active.

## Give tools less authority than the worker's ambition
Documents and web pages supplied to a model can contain instructions intended to redirect it. Treat retrieved content as input, not as a source of operating permission. OWASP's guidance on prompt injection recommends several layers of control, including restricted tools and checks around sensitive actions. Put permissions in the tool implementation and the operating environment, where the model cannot rewrite them through a response.

Use separate credentials for separate jobs, with access only to the records or directories each needs. Prefer structured parameters over model-generated shell commands. Validate file destinations and outbound targets. Require a deliberate approval boundary for sending messages, publishing changes or modifying important records. A read-only research worker and a production administrator should never inherit the same credentials merely because both run on one host.

## Keep resource controls and recovery explicit
Containers can organise dependencies and process boundaries, but they need deliberate configuration. Docker documents that containers have no resource limits by default. Set appropriate memory and CPU limits, use an unprivileged process where supported and mount only the required data. Keep model endpoints and administration interfaces restricted rather than exposing them simply to make initial testing convenient.

Persist the task state separately from temporary working files. Record enough information to recognise completed work after a restart without storing every prompt forever. Protect credentials from logs and generated artifacts. Define retention for inputs, outputs and diagnostic data. A recovery test should restart the worker and verify that unfinished jobs resume safely, completed jobs remain completed and failures remain visible to the operator.

## Make the first deployment an acceptance test
Run the narrow workflow with synthetic or approved sample data before granting wider access. Check useful output, permissions, queue behaviour and recovery from a failed dependency. Observe the whole process, including browser children and data downloads. If the model is local, include a cold start and the intended number of concurrent requests in the test.

Use Vector or Pulse as a specification shortlist for ordinary workers, and compare Tensor only when the inference runtime needs suitable GPU resources. Confirm compatibility and availability before a real order. Expand the worker's permissions only after its behaviour can be reviewed. The aim is a service with a clear operating envelope, a measurable resource budget and a straightforward way to stop and recover work.

## Checklist
- Describe one job and its approval boundary.
- Separate remote API processing from local inference.
- Measure tools, browsers and peak concurrency.
- Apply time, resource and permission limits.
- Test restarts without duplicating completed actions.

## Frequently asked questions
### Does every AI worker need a GPU?
No. A worker calling a remote model API usually needs resources for orchestration and tools. Local inference has separate hardware and runtime requirements.

### Does a container make an autonomous workflow safe?
No. Configure its permissions, mounts, network access and resource limits, and check consequential tool actions outside the model.

### Does hosting the worker in Switzerland keep every prompt there?
No. Remote model APIs and other integrations may process content elsewhere. Map those dependencies explicitly.

## Primary sources
- [Docker: Resource constraints](https://docs.docker.com/engine/containers/resource_constraints/)
- [OWASP: LLM Prompt Injection Prevention](https://cheatsheetseries.owasp.org/cheatsheets/LLM_Prompt_Injection_Prevention_Cheat_Sheet.html)
- [Ollama: Runtime and concurrency FAQ](https://docs.ollama.com/faq)

## Related infrastructure
- [Vector VPS](https://swissoctet.com/services/vector/index.txt): Your own Linux environment, with KVM virtualization and a clear path from small projects to larger workloads.
- [Pulse NVMe](https://swissoctet.com/services/pulse/index.txt): Fast NVMe storage for applications where I/O is part of the performance equation.
- [Tensor GPU](https://swissoctet.com/services/tensor/index.txt): Dedicated GPU configurations for accelerated compute, rendering and machine-learning workloads.

## Related guides
- [Size VPS CPU and memory from a real workload](https://swissoctet.com/insights/size-vps-cpu-and-memory/index.txt): Build a memory budget, inspect contention and separate one busy thread from work that can use several virtual cores.
- [Build a GPU VRAM budget before choosing hardware](https://swissoctet.com/insights/gpu-vram-budget-for-real-workloads/index.txt): Account for weights, activations, caches and temporary buffers, then test the peak workload rather than the model file alone.
- [A practical threat model for your hosted project](https://swissoctet.com/insights/threat-model-a-hosted-project/index.txt): Map assets and trust boundaries, choose a few realistic failure scenarios and turn them into controls you can test.