Skip to content
News
Research

ORCNEIT Lab / Research & engineering papers

Research

How we train and evaluate our own model. Experiments, measurements and findings, with test conditions and explicit limitations.

Laboratory publications

№ 04Results overview · In progress

ORCNEITGPT: how the project began and where it stands

From the first experiments with our own language model to a revised training approach and a new corpus. What we tested, which approaches fell short, and why the direction changed.

Observation period
—
Published
Read the study

№ 05Fine-tuning study

Learning memory: retaining old skills does not establish new ones

Three small-model experiments exposed forgetting, the benefit of replay, and the absence of a reliable memory protocol despite lower training loss.

Experiment
Published
Read the study

Instruction retention

Correct responses out of 25

Before fine-tuning

23

Memory only

5

Memory with replay

23

Stronger memory with replay

20
The same general instruction test, not a memory score. Four model states; the axis starts at zero.

№ 06Numerical precision study

Preserving small updates during training

A bounded numerical comparison reduced mean signed update-writeback error by about 90-fold. A separate later run established stable training with high-precision state, but not model readiness.

Experiment
—
Published
Read the study

Update writeback error

|Mean signed error|

Round to nearest

7.76E-8

Stochastic rounding

8.58E-10
Lower is better. First-step measurement on identical real weights and gradients. Mean signed error, not mean absolute per-weight error or response quality.

№ 07Text representation study

Choosing tokenization through training, not compression alone

The compression leader did not lead proxy training. Equal-source-text comparison changed the selection and showed why tokenized length alone is insufficient.

Experiment
Published
Read the study

Proxy-training result

Bits per byte; lower is better

Compression leader

2.374638

Second alternative

2.298133

Selected alternative

2.288238
Three alternatives on identical source text. Byte-normalized measure, zero-based axis. Proxy-model text prediction, not finished-assistant quality.

№ 08Generation study

Response safeguards and model capability: what actually changes

Rules around a model can improve observed output without changing its weights. We examine an early assistant-versus-model comparison and a separate test of encoding and repetition safeguards.

Experiment
—
Published
Read the study

Severe generation degeneration

Responses out of 1,600; lower is better

Without external safeguards

499

With external safeguards

433
August paired comparison on unchanged model weights: 400 prompts × 4 modes. Mechanical generation defects, not semantic correctness.

№ 01Model training

Training a language model from scratch: measurements and evaluation

A completed run of 80,000 steps and 327.68 million training tokens. We report learning curves and a separate capability evaluation, including the criteria that were not met.

Experiment
—
Published
Read the study

Loss during training

Loss
040 00080 000

Training step

Lower is better. 17 observations; training loss is a 250-step mean, development loss uses a held-out development sample. Lines connect measured points, not continuous observations.

№ 02Fine-tuning experiment

Dialogue fine-tuning: fewer repetitions, but not enough correct answers

We compared the base model and its fine-tuned version on the same tasks. Answer termination improved and severe degeneration decreased, but assistant acceptance criteria were not met.

Experiment
Published
Read the study

Before and after fine-tuning

Responses out of 256

Correct responses ↑

Base model: 0 / 256
After fine-tuning: 43 / 256

Severe degeneration ↓

Base model: 238 / 256
After fine-tuning: 36 / 256

Normal termination ↑

Base model: 18 / 256
After fine-tuning: 239 / 256
Each model: 128 tasks × 2 modes = 256 responses. The axis starts at zero. Correctness, degeneration and termination are distinct measures, not parts of one total.

№ 03Engineering study · In progress

Why we are rebuilding the training corpus

Failed model evaluations prompted us to revisit the data, tokenization and evaluation itself. We report measured changes and the preparation of a new corpus, still in progress as of 26 September 2026.

Observation period
—
Published
Read the study

Vocabulary selection: text compactness

Tokens per UTF-8 byte

Small vocabulary

0.365322

Medium · selected

0.354379

Large vocabulary

0.348899
Lower is more compact. Equal-weight average across four text groups on the same sample, measured on 20 August 2026. This is not a measure of model response quality.

Experiment dates are checked against the working log. Publication dates record when papers appeared on this site. These studies are not a product release announcement or an independent model certification.