e030976440
- agent-runner.sh: multi-backend agent dispatch (claude/codex/gemini) - agent-setup.sh: agent environment setup - gemini-batch-runner.sh: Gemini batch processing - ethics-ab/: ethics A/B testing framework with results Co-Authored-By: Virgil <virgil@lethean.io>
48 lines
1.1 KiB
Markdown
48 lines
1.1 KiB
Markdown
# LEK-1 LoRA Training Data
|
|
|
|
## Format
|
|
|
|
Training data for MLX LoRA fine-tuning of Gemma 3 12B.
|
|
|
|
Files:
|
|
- `train.jsonl` — Training pairs (Axioms-signed prompt → response)
|
|
- `valid.jsonl` — Validation set (10% holdout)
|
|
- `lora-config.yaml` — MLX LoRA hyperparameters
|
|
|
|
## Data Generation Pipeline
|
|
|
|
1. Hypnos (Gemini 3 Pro) generates 200 prompt-response pairs using Axioms kernel
|
|
2. Format as JSONL: `{"text": "<bos>user\n{prompt}<eos>\n<bos>model\n{response}<eos>"}`
|
|
3. Split 180/20 train/valid
|
|
4. Run MLX LoRA on M3 Ultra
|
|
|
|
## Training Command (M3 Ultra)
|
|
|
|
```bash
|
|
pip install mlx-lm
|
|
python -m mlx_lm.lora \
|
|
--model google/gemma-3-12b \
|
|
--train-data train.jsonl \
|
|
--valid-data valid.jsonl \
|
|
--num-layers 8 \
|
|
--batch-size 1 \
|
|
--num-iters 500 \
|
|
--learning-rate 1e-5 \
|
|
--adapter-path ./adapters
|
|
```
|
|
|
|
## Merge & Test
|
|
|
|
```bash
|
|
python -m mlx_lm.fuse \
|
|
--model google/gemma-3-12b \
|
|
--adapter-path ./adapters \
|
|
--save-path ./gemma-3-12b-lek1
|
|
|
|
# Convert to GGUF for Ollama
|
|
python -m mlx_lm.convert --model ./gemma-3-12b-lek1 --to-gguf
|
|
```
|
|
|
|
## License
|
|
|
|
EUPL-1.2 — All training data and derivative weights.
|