Real-Time Teacher-Led Adaptive Learning
Live context stays on the device. When a session closes coherently, traceable evidence can enter durable memory and later return as a small, scoped packet for a teacher decision.
What continues after the call
The original lesson moment, a teacher-confirmed correction, later recall and the next decision stay connected. Each step belongs to the same human-led learning path.
The teacher notices and decides. LingoSide preserves where the evidence came from, brings back the relevant part and leaves room to say there is not enough evidence.
See the product layersDuring the lesson
Observe the lesson
Keep source text, translation and contextual reply suggestions in a sidecar that takes one glance, so a meaningful phrase, correction or difficulty can be noticed while the conversation keeps going. Earlier moments from the same session inform what is suggested.
After the lesson
Keep what happened
Preserve the session as a finalized record: saved phrases, teacher-confirmed corrections and evidence-linked moments. Every item keeps a pointer to the exact utterance it came from, so it can be checked against the lesson instead of taken on trust.
Between lessons
Practise it later
Ask the learner to retrieve first, then reveal the source moment and its context. Selected material comes back at useful intervals and in the situation where it first mattered, rather than as another generic list of exercises.
At every teaching proposal
Teacher decides
Show the teaching proposal together with the lesson moment behind it, so the teacher can approve, edit or reject it. That decision, not the model’s confidence, is what the rest of the loop builds on.
Before the next lesson
The next lesson adapts
Bring the right earlier material back into reach: due reviews, saved phrases, open corrections and possible patterns, each one leading to the evidence behind it. LingoSide is building this stage so that earlier decisions support better choices for learner and teacher, without turning the lesson into a dashboard or an automated course.
Evidence-based memory
LingoSide does not need to remember everything. It preserves useful lesson evidence and the teacher’s correction, and it is building the memory of what comes next: the learner’s later recall and what happens when the language returns.
A phrase, correction or difficulty remains linked to the source utterance.
Teacher confirmation, edits and rejection become part of the record.
The learner tries to retrieve before seeing the answer and context again.
The loop is designed to distinguish independent recall, transfer, prompted use and no attempt.
An observation can be checked against the lesson moment that supports it.
When a teacher corrects or removes an item, later views are designed to use the corrected record.
A hypothesis is not presented as a fact about the learner.
Every memory item is anchored to a lesson moment. Its confidence is computed from admission, validity window and revision history, not declared by whoever produced it. A model proposal never becomes servable without a recorded teacher decision, and a correction becomes the next revision while the earlier one is marked superseded, not overwritten. A teacher’s confirmation records accountability, not extra accuracy.
Status: model and tests only, not yet connected to storage
An Open Learner Model is an established research idea: a picture of what a learner knows that the learner and teacher can see and question, rather than a hidden score. LingoSide is studying it as a changing learning picture with evidence and uncertainty kept visible. It produces no learner score and no lasting label of a person.
Status: research, not built in the product
Teacher authority
LingoSide can surface a traceable moment and a possible next move. The teacher reads the learner, chooses the timing and decides what happens in the lesson.
Observation
The lesson evidence suggests a retrieval problem, not proof of a broad weakness.
Possible strategy
Bring the phrase back through active recall in a related conversation.
Under the hood
During a lesson, LingoSide serves the active working set from client RAM. With session archiving enabled (the default), it checkpoints finalised content locally. After the session closes coherently, its Evidence is intended to enter the backend index for durable, correctable recall across lessons.
For everyone
Think of it as a librarian with a strict access card: search only the right learner and course, combine exact-word and meaning matches, then hand the AI a few traceable excerpts—not the whole history.
Established terms, one architecture
RAM + default-on local checkpoints + backend memoryRAG (retrieval-augmented generation) means that before AI helps, a retrieval layer looks for relevant Evidence instead of pouring the whole transcript history into the prompt. The terms below name separate parts of the architecture; together they describe LingoSide’s design, not a new industry standard.
Finalised moments keep their speaker, language, time and source. Across lessons, a raw transcript or AI hypothesis is not treated as a fact: teacher confirmation, uncertainty and correction status stay explicit.
The server limits candidates to the permitted learner and course before Evidence is ordered. Structured fields remove invalid or superseded items; full-text search (FTS) and embeddings then find exact and semantic matches.
Time, review state, teacher-confirmed corrections and the current lesson goal can change the order of Evidence candidates. The consumer receives only top-k Evidence with provenance and why it matched—or no result when confidence is too low.
In the live lesson, a retrieved line that scores below a minimum similarity is dropped, so the AI receives fewer lines, or none, rather than a weak match. A reworded question can score below that threshold and return nothing. The same packet shape for the durable cross-lesson index is being built.
Status: shipped for the live lesson
Access is checked before retrieval, so only authorised people can read learner-and-course memory.
Corrections and invalidations affect recall, so stale evidence does not quietly return.
Corpus size is not prompt size: AI receives a small, traceable packet instead of the whole history.
In deployment terms, the edge–cloud split has two client-side layers: RAM serves latency-sensitive working state, while default-on local session checkpoints protect finalised lesson content during the session. Only after the session closes coherently is cross-lesson Evidence intended to enter the server-side index. That topology is separate from hybrid retrieval, which describes how backend Evidence is found.
Structured filters, FTS and embeddings generate Evidence candidates; temporal and lesson-goal signals order them by relevance. Only the best small packet reaches the consumer. A larger history does not require GraphRAG or a longer context window by default.
Every retained item keeps its source, date, language, status and teacher decision. A correction creates a new version; invalidated or superseded evidence leaves active recall without erasing the audit trail.
Search is scoped on the server to the requester, learner and course before Evidence is ordered by relevance. Without the required grant, a requester sees nothing, however similar the text may be.
Session boundaries
The learner sidecar holds no provider key of its own. It sends one speech stream for live recognition, from the source selected for the lesson: the learner microphone or system audio from the lesson call. The teacher sidecar reads the shared lesson state and opens no audio connection.
Two connected sidecars. One speech stream from the learner sidecar: the source selected for the lesson, its microphone or system audio from the lesson call. It goes directly to the STT provider for live recognition. The teacher sidecar reads the shared lesson state, transcript and teaching support; it sends no audio.
The Windows package is built without a provider key inside it, so it cannot connect to anything on its own. It has to ask the backend first, every single time.
Each connection attempt gets its own pass that expires about a minute later. Nothing long-lived sits on the computer waiting to be copied or reused.
The backend opens the session, issues the pass and records what happened. The sound itself goes directly from your machine to the recognition service, so there is no shared middle stop to wait behind.
A teacher reaches a learner’s memory only through a grant the learner gave for that course, split into separate read, save and review scopes, and only for an explicitly selected participant: there is no default learner. The learner can revoke a scope at any time, and the backend then refuses further access. An append-only audit log records session lifecycle and speech-token decisions. Grants have no expiry date, and there is no real-time policy engine.
Status: shipped in the backend
For live speech recognition, the provider is selected from the language pair before the session starts. If a connection or token request fails, LingoSide does not quietly switch to another provider. Production refuses a provider that has no recorded retention attestation. The text model provider is a single deployment setting with no failover.
What the privacy page says about providers
Status: shipped for speech recognition
For a teacher this means one practical thing: putting a lesson through LingoSide does not drop it into a shared pipe, and it does not leave a reusable key sitting on the computer afterwards.
Where it runs
A lesson behaves the same wherever it runs, because this is not four products pretending to be one. The sidecar is a single application compiled for each system.
The desktop sidecar, delivered as a signed package that updates itself in the background.
The same desktop sidecar on Apple hardware.
A phone beside the lesson rather than a second screen to manage.
The same phone-sized sidecar on iPhone and iPad.
Flutter is Google’s open-source toolkit for building one application in Dart and compiling it into a real native app for each operating system — not a website in a wrapper. LingoSide uses it for the whole sidecar, so the lesson logic, the interface and the backend contracts live in one codebase instead of four. That is why adding a platform is mostly packaging, permissions and store rules rather than rebuilding the product.
The learner sidecar sends one speech stream for recognition, from the source selected for the lesson: its microphone or system audio from the lesson call. The teacher sidecar reads the shared lesson state and sends no microphone or system audio.
Technology behind LingoSide
Data architecture
Events move through the system, but their meaning stays owned. Stream processing, the learning-data warehouse, AI diagnostics and product analytics remain separate by design.
Domain event path
In the target architecture, the client event stream enters Proton, where streaming SQL shapes live events and sends durable batches to ClickHouse.
Real-time event processing
Proton OSS is Timeplus’s Apache 2.0 open-source streaming SQL engine: a single C++ binary with no JVM or ZooKeeper. The target architecture assigns it the client event stream and streaming SQL, with durable batches routed to ClickHouse.
Evidence for our own benchmark—not a capacity promise.
Why a separate layer
The live stream can absorb pressure and be replayed without making the domain depend on a stream engine API.
Domain warehouse: RAW + Serving
The target architecture assigns ClickHouse the durable learning-data warehouse—not the live stream, AI trace store or product telemetry.
RAW · evidence history
Append-oriented events and educational evidence. New interpretation never overwrites the historical record.
Serving / Derived · usable state
Rebuildable projections, aggregates and current learner state used by the backend.
Same RAW + same derivation version → same semantic state.
Why a separate layer
Evidence stays auditable; current state can change and still be reproduced from history.
AI execution observability
The backend’s metadata-only trace layer links prompt versions, models, retrieval stages, latency and cost. Typed evaluations can join this layer when they exist.
Why a separate layer
Engineers need the execution waterfall without copying lesson content into diagnostics.
Product analytics and experiments
Product behaviour belongs here only through explicit consent and a closed allowlist: flows, feature use, feature flags and experiment outcomes.
Why a separate layer
Product behaviour can be measured without turning raw educational data into analytics.
Teacher Action Loop projection
A read-only view correlates domain evidence with AI execution metadata. Stable references cross the boundary; the underlying records stay with the system that owns their meaning.
Domain path · ClickHouse
ClickHouse contributes event and evidence IDs, causation, the teacher action, the executed intervention and the next learner attempt.
AI path · Langfuse
Langfuse contributes the AI trace ID, prompt and config digests, model, retrieval stages, latency and cost—metadata only.
Correlated result
The read-only Teacher Action Loop projection joins both paths through stable references. It owns no canonical data; a missing critical edge returns INCOMPLETE TRAJECTORY instead of a guess.
Typed importer boundary
The import contract gives a typed projection importer one job: read authorised domain rows from ClickHouse and metadata-only trace records through Langfuse’s supported API or export boundary, then map stable IDs into the read-only trajectory shape. It does not depend on Langfuse’s internal ClickHouse tables or copy lesson content.
Supporting stack
Speech recognition and AI services connect to LingoSide through specialized, replaceable adapters. The learning layer depends on stable internal contracts rather than any single vendor’s protocol. For speech recognition, that contract includes timestamped segments, speaker attribution, confidence and language identification. Speaker attribution separates voices; it does not name them. Which voice is the learner and which is the teacher is a separate decision, confirmed in the lesson and correctable there, and it stays unassigned until someone confirms it.
Audio is sent directly from the learner sidecar to the STT provider; it is not relayed through the LingoSide backend. To evaluate ASR integrations, LingoSide is building a separate, versioned test corpus and a streaming harness that measures accuracy, code-switching, speaker attribution, partial-result revisions, reconnect behavior and latency. It also replays saved structured session events—not production audio—through the downstream learning pipeline.
This architecture allows providers to be evaluated or replaced as the technology improves without rebuilding the lesson experience.
ASR evaluation harness
The core ASR Evaluation Harness is implemented and under active development. Provider evaluation and Live ASR Harness test different boundaries.
The evaluation harness scores recognition against verified references and supports the current STT provider. The Live ASR Harness sends audio at real-time pace and records streaming events for replay. Provider-only measurements do not validate the complete LingoSide lesson pipeline.
Transcript quality
Versioned references support repeatable WER/CER scoring by language and condition. Corpus coverage and reference quality bound every result; an implemented scorer is not a provider-quality verdict.
Streaming quality
Paced replay and event-tape checks cover duplicate finals, revisions of finalized audio, overlapping final spans and timing. Latency is meaningful only when audio was sent at real-time pace.
Attribution quality
Speaker attribution requires labelled reference scenarios and a separate coverage check. Correct transcript text alone does not establish correct speaker attribution.
The evidence standard
No quality claim without a versioned fixture, an expected result and a reproducible run.
Harness engineering
LingoSide uses a versioned quality contract to define risk and required proof. Harness engineering turns that contract into a repeatable question for each learning, AI and speech path: what evidence must pass before a release?
The operating rule
Claims and changes must be testable against evidence.
Input
A candidate enters as something to test, not something to trust.
Exercise
Contracts · replay · evaluations · fault cases
Verify
Tests · privacy · regressions · required evidence
Review
After repository tests, Code Review examines the implementation; Decision Review checks alignment with product decisions. Open findings return the change for revision.
Decide
Invariants are written down before implementation and checked with the standard test runner. Layout validation is fuzzed with a fixed seed: 10,000 mutated manifests must never fail and must give the same answer twice, and 5,000 random trees must keep every size contract. Memory-model rules are checked by example and by looping over every admission value. Memory and learner-state rules have no randomised tests yet, so generative testing does not cover learner state.
Status: shipped for layout validation; learner state is checked by example only
One method, four boundaries
Lesson flow, speech, teacher AI and later recall are different surfaces. They share the same demand: a scenario, an expected outcome and evidence that can fail.
01 · Lesson path
The executable lesson contract covers a 60-minute lifecycle, six EN/PL/RU directions, archive reopen and four controlled failures. That makes lifecycle, language coverage, recovery and failure handling reviewable as one contract.
No LingoSide audio recording · The harness replays a synthetic event scenario, not a sound file.
02 · Speech path
A defensible ASR quality gate requires a versioned corpus and streaming measurements sliced by accuracy, code-switching, speaker attribution, partial-result churn, reconnects and latency. Missing coverage is a gap, never proof.
03 · Teacher AI
Provider-neutral policy checks block invalid suggestion shapes. Beyond shape validation, a defensible AI evaluation requires evidence-linked teacher scenarios and keeps the final decision with the teacher.
04 · Learning path
The learning data model preserves review attempts and outcomes. A defensible learning evaluation compares those records with delayed recall instead of treating engagement as learning.
How replay works
Replay is not ‘press play.’ LingoSide does not create or save an audio recording. The contract reruns a versioned synthetic lesson scenario and compares the structured result with explicit expectations.
Synthetic transcript events carry text, translations and timestamps; the scenario also declares language pairs and controlled failures.
The harness injects the same synthetic events into the production session controller and advances a virtual lesson clock.
It checks the expected outcomes, opens the resulting structured session in a fresh archive store and verifies the archive UI and failure paths.
A real lesson starts differently: audio goes directly to the STT provider for live recognition, while LingoSide archives structured outputs—not a recording. Speech accuracy is a separate gate: it requires an explicit, versioned audio corpus.
An engineering view
This is AI infrastructure for continuity of human-led learning: one selected audio source becomes one speech stream; provider adapters create structured events; provenance travels with evidence; automation stops before the teacher's decision.
one selected source · one speech stream
the provider protocol ends here
proof scope travels with the result
automation stops before judgment
Provider protocols end at adapters. Shared layers consume stable contracts.
Synthetic events replay the lesson contract without creating an audio recording.
Provenance and proof scope accompany every engineering conclusion.
Automation can propose. The teacher owns pedagogical judgment.
The architectural core is domain-independent: its invariants do not depend on language learning.
Language learning is the first domain, not a constraint of the architecture.
100000111101000000001010100110110101110101010