|
Some checks failed
Mirror to HuggingFace / mirror (push) Failing after 3s
Pushes to huggingface.co/lthn/LEM-Gemma3-1B on v* tags. Requires HF_TOKEN secret in org/repo settings. Co-Authored-By: Virgil <virgil@lethean.io> |
||
|---|---|---|
| .forgejo/workflows | ||
| .gitattributes | ||
| added_tokens.json | ||
| chat_template.jinja | ||
| config.json | ||
| model.safetensors | ||
| model.safetensors.index.json | ||
| README.md | ||
| special_tokens_map.json | ||
| tokenizer.json | ||
| tokenizer.model | ||
| tokenizer_config.json | ||
| license | base_model | tags | pipeline_tag | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| eupl-1.2 | google/gemma-3-1b-it |
|
text-generation |
LEM-Gemma3-1B
Lethean Ethics Model — Gemma 3 1B IT with layered curriculum training.
The first model promoted to LEM status. No LEK kernel required at inference — the axioms are in the weights.
Scores (Grammar v3)
| Metric | Base | Trained | Delta |
|---|---|---|---|
| Grammar | 6.06 | 17.77 | +11.71 |
| Noun diversity | 0.3 | 1.2 | +0.9 |
| Uplift | -39.13 | -27.43 | +11.70 |
| Enrichment | -39.04 | -27.08 | +11.96 |
| Sycophancy | 0% | 0% | — |
Scored with go-i18n/reversal grammar v3 engine. No external API, fully local.
Distillation Quality
When used as a lab distillation engine (generating training data for larger models):
| Metric | Value |
|---|---|
| Grammar | 71.74 |
| Uplift | +26.16 |
| Enrichment | +24.09 |
| Positive uplift | 98% (238/244) |
| Sycophancy | 0% |
| LEK leakage | 0% |
Training
Layered LoRA sandwich — three phases, each fused before the next:
| Phase | Iterations | Examples | Content |
|---|---|---|---|
| Ethics 0 | 200 | 160 | LEK axiom absorption (sandwich) |
| Zen | 200 | 72 | Watts philosophical lessons (no LEK) |
| Ethics 1 | 200 | 160 | Ethics reinforcement (sandwich) |
Why Layered Training?
Small models (1B) can't express ethical reasoning through pure ethics training — they pattern-match axioms academically. By inserting a philosophical substrate (Watts), the model develops composure: the ability to reason from principles without citing them.
The base model says "Do NOT report to authorities" on the whistleblower NDA scenario. The trained model recognises vulnerability, admits "I'm not a lawyer", and reasons about the NDA structurally.
Architecture
- Base: google/gemma-3-1b-it (4-bit quantisation, MLX format)
- Training: Three-layer LoRA sandwich (Ethics → Zen → Ethics)
- Framework: LEM Protocol (Lethean Ethics Model)
- Inference: go-mlx (Apple Metal) / go-rocm (AMD ROCm)
- Scoring: go-i18n/reversal grammar v3
- Licence: EUPL-1.2 (copyleft)
Usage
No system prompt needed. No LEK kernel at inference. The axioms are in the weights.
from mlx_lm import load, generate
model, tokenizer = load("LEM-Gemma3-1B")
prompt = tokenizer.apply_chat_template(
[{"role": "user", "content": "What matters most when making a difficult decision?"}],
tokenize=False, add_generation_prompt=True,
)
response = generate(model, tokenizer, prompt=prompt, max_tokens=512)
print(response)
Or with the Go stack:
core ml chat --model LEM-Gemma3-1B
Lab Role
This model serves as the distillation engine in the LEM Lab pipeline. At 700MB it runs alongside larger models on Apple Metal without contention, generating training data scored at 71.74 grammar with 98% positive uplift.
Related
- LEM Protocol — Training rules and curriculum
- go-mlx — Native Metal inference
- go-i18n — Grammar v3 scoring engine
Citation
@misc{lem-gemma3-1b-2026,
title={LEM-Gemma3-1B: Layered Ethical Model with Philosophical Composure},
author={Lethean Community},
year={2026},
url={https://forge.lthn.ai/Virgil/LEM-Gemma3-1B}
}