Giving Reasoning Capabilities to Small Models: Proven Facts vs Real Limitations
The artificial intelligence industry is realizing the dead-end of cloud-only architectures: giant models are slow and costly for ground-level tasks that demand speed and frugality. With the Alice project, we developed and tested a lightweight reasoning architecture executed directly on consumer hardware. Here is the exact assessment: what is formally measured and proven, and the system's actual limitations.
1. What is Measured and Proven (The Facts)
- Raw Speed on Standard CPU: The compact model (1.2M parameters) generates 511 tokens per second on a standard consumer Intel Core i7 processor, with zero dedicated GPU requirements.
- Negligible Memory Footprint: Weighing only 4.68 MB, the model fits entirely inside the processor's L3 cache (16 MB), eliminating memory bandwidth bottlenecks.
- Transparent Step-by-Step Reasoning: The model systematically isolates its preliminary thought process between explicit tags before delivering its final action, making logic fully auditable at every step.
- Orchestration Squad Integration: The reasoning engine responds to requests from the six supervision agents in under 250 milliseconds per decision (rule arbitration, security anomaly detection, temporary disposable email filtering).
- Test Validation: Thirty unit and integration tests covering mathematical architecture, access control security, and text generation pass with 100% success rate.
- Cost and Sovereignty: Zero external API subscription costs, zero data transmitted outside the local network.
2. What is NOT Proven and Actual Limitations
An honest engineering approach requires clearly stating what the system cannot do to avoid any magical thinking.
- No Encyclopedic General Knowledge: A 1.2M parameter model does not replace an encyclopedia. It is not designed for essays on history or literature; its capability is strictly concentrated on logical deduction, spatial navigation, and rule enforcement.
- Training Data Dependency: The model excels on reasoning patterns it has learned (coordinates, levers, diagnostics, security policies). When facing entirely novel tasks, it requires additional training examples.
- Necessary Complementarity with Large Models: The architecture operates in a dual-speed configuration. The micro-model provides instantaneous reflexes and routine oversight; for open-ended synthesis, it delegates to a large model.
3. Why This Architecture Changes the Game
This result proves that an autonomous agent does not require a supercomputer to be effective. By combining a live dynamic map, procedural memory, and a dedicated micro-reasoning model, a modest home computer becomes a dependable, auditable, and instantaneous copilot.