High-Performance Rust LLM Gateway
Challenge
Provide centralized, low-latency access to AI models, through an API compatible with the OpenAI and Anthropic formats, for critical transcription and automation workflows without heavy infrastructure overhead or exposing private credentials.
What it solves
- Unified high-performance gateway: Compiled Rust backend (Axum, Tokio, SQLx) with protocol parity for industry specs, real-time SSE token streaming, and media support.
- Virtual models and commercial controls: Model abstraction pairing virtual keys with immutable system prompts, SvelteKit administration dashboard, and Stripe billing integration (prepaid wallets, usage-based metering, and recurring plans).
What it avoids in production
- Budget overruns: Strict Redis rate limiters and token quotas preventing runaway execution or unauthorized traffic spikes.
- Cascading outages: Priority queues and circuit breakers isolating AI model failures without crashing the ingress layer.
Engineering Notes & Systems Design
Technical trade-offs, performance benchmarks, and lessons learned building mission-critical software.
Laya on CPU, No GPU: What I Measured on Three Machines Before Putting It in Production
38ms was with a GPU. On CPU, 94 to 848ms depending on the machine. Real benchmarks, the thread trap, ONNX vs PyTorch, and how the deployment ended up on a VPS.
Hexagonal Architecture and DDD in Laravel: Decoupling Domain from Eloquent with DTOs and Repositories
How to architect complex enterprise applications in Laravel without letting Eloquent pollute business rules: Ports and Adapters, immutable PHP 8.2 Value Objects, and strict layer boundaries.
PHP at Planetary Scale: Slack, Meta, Strict Typing with PHPStan, and Fibers Concurrency (Part 4)
Why tech giants like Slack, Wikipedia, and Meta continue investing in PHP: the operational economics of stateless workloads in Kubernetes, maximum static analysis, and cooperative concurrency with Fibers.