Built at the YC Gemma 4 Voice Agents Hackathon

Marginalia

marginalia (n.) — Latin, from marginalis. The handwritten notes a reader inscribes in the margins of a book. Private commentary, meant only for the reader's eyes.

The private intelligence layer for conversations
that can't leave the room.

Edge compute. Local inference. Never a server.
Gemma 4 running on your phone, in your pocket, under your control — not in somebody else's cloud.

Cloud AI writes in the main text. Marginalia writes in the margin.

Watch the Demo View Source

See It Work

You're walking into an interview. Ask Gemma for salary benchmarks before the meeting. During the meeting, ask how to respond to a counter-offer. All on your glasses. All private.

Cactus Compute + Gemma 4 — local on-device inference
Even Realities G2 — BLE glasses connection
Marginalia dashboard — G2 connected, models loaded, activity log
Dashboard. Status capsules (G2, LLM, server), VAD → STT → LLM pipeline, activity log with timestamps.
Marginalia chat — Gemma 4 responding on-device with model status
On-device chat. Gemma 4 E2B responding locally. VAD, STT, LLM all showing Ready. G2 glasses found via BLE.
Phase 1 — Shipped at the hackathon

Native iOS app running Gemma 4 E2B and Parakeet STT fully on-device via Cactus Compute's inference engine. Direct Bluetooth connection to Even Realities G2 glasses, bypassing the Even Realities consumer app entirely. The iPhone is the brain, the glasses are the display, nothing touches the cloud.

Phase 2 — In progress

Streaming text output to the G2 lens HUD in real-time, ring-tap selection via the R1 controller, and end-to-end voice-to-function-call pipeline. When complete: a private whisper layer that listens, reasons, and executes — all before you finish the sentence.

1
Before the meeting
"Hey Gemma — I'm interviewing an executive assistant, 5 years experience, San Francisco. What's the expected salary range?"
↓
2
Gemma responds on your HUD
SF EA, 5yr exp: $75-95K base
+ $5-15K bonus typical
Top range $105K w/ C-suite exp
↓
3
During the interview
"She's asking for $110K. How should I respond? What questions should I ask to justify it?"
↓
4
Real-time tactical coaching on lens
Ask: What C-suite execs have you supported?
Ask: Do you manage travel for multiple execs?
$110K justified if managing 3+ executives

Why On-Device Changes Everything

Every smart-glasses AI runs in the cloud. That disqualifies it from every regulated conversation — HIPAA, attorney-client privilege, M&A due diligence, HR, GDPR. Marginalia doesn't.

Gemma 4 E2B

Google DeepMind. Apache 2.0. Released April 2, 2026. 1.5 GB at INT4. Native audio, native vision, function calling. The first open-source model that makes real edge compute possible.

Cactus Compute

YC S25. Announced day-one Gemma 4 support immediately after Google's release. C++ inference engine for ARM. Zero-copy memory mapping, Apple Neural Engine acceleration. 48 tok/s decode on iPhone.

End-to-End On-Device

The first product in this category where "on-device" is literally true. Audio in, intelligence out, zero bytes to any server. Verified in airplane mode.

The Hardware

Even Realities G2. Smart glasses that look like regular glasses. No camera. No recording capability. Just a private heads-up display and a microphone.

576×288
Micro-LED per eye
(green, 4-bit greyscale)
No camera
Privacy by design
nothing to record
~1.5 GB
Gemma 4 E2B model
(INT4 quantized)
~36g
All-day wearable
looks like normal glasses
<5s
End-to-end latency
(voice in → HUD out)
0 bytes
Data sent to cloud
(verified airplane mode)
 ┌──────────────┐                 ┌──────────────────────────────────┐
 │  G2 Glasses  │      BLE       │  iPhone                          │
 │              │◄──────────────▶│                                  │
 │  Mic        ─┤── audio ──────▶│  Marginalia                      │
 │  HUD Lens   ◄┤◄─ response ──│    ├── Parakeet STT (0.6B)       │
 │  R1 Ring    ─┤── input ──────▶│    ├── Gemma 4 E2B (1.5GB INT4)  │
 │              │                 │    └── Function calling           │
 │  No camera.  │                 │                                  │
 │  No cloud.   │                 │  Airplane mode. Zero bytes out.  │
 └──────────────┘                 └──────────────────────────────────┘

Built for the Rooms Where Cloud Isn't Allowed

Each vertical, a fine-tuned model on privileged domain corpora. Each one, deployable where generic cloud AI cannot legally go.

Tier 1

Marginalia Counsel

Litigators, M&A attorneys, in-house counsel

Attorney-client privilege is a legal absolute. Cloud AI in a client meeting is a potential privilege waiver — a malpractice event.

$200-500/attorney/month. AmLaw 100 firm = $1-3M ACV.

Tier 1

Marginalia Clinical

Physicians, specialists, surgeons

HIPAA prohibits cloud AI with patient audio. Period. Marginalia is the only legal ambient clinical intelligence.

$150-300/physician/month. Hospital system = $5-20M ACV.

Tier 1

Marginalia Diligence

Investment bankers, deal teams

Material non-public information. Cloud AI in a deal room is a securities violation waiting to happen.

$500-1000/banker/month. Bulge bracket = $10M+ ACV.

Tier 2 — Wedge Vertical

Marginalia Floor

Plant managers, line supervisors, maintenance techs

Manufacturing IP is aggressively guarded. Apple supply chain, defense contractors, pharma — none can use cloud AI on-site. Founder has 6 years at Apple — Day One distribution.

$50-150/operator/month. Plant rollout = $200K-2M ACV.

Tier 2

Marginalia Executive

C-suite, HR partners, sensitive 1:1s

Performance reviews, terminations, compensation discussions. The conversations where AI is most useful and least safe to put in the cloud.

Tier 2

Marginalia Defense

Intel analysts, policy staff, classified environments

Classified environments cannot use cloud AI by federal mandate.

Why Now

The model exists

Gemma 4 E2B shipped April 2, 2026 — 17 days ago. The first open-weight model with native audio, native vision, and function calling at 1.5 GB. Before this, edge AI meant chaining separate models. Now it's one model, one forward pass.

The hardware exists

Smart glasses crossed from prototype to product. The G2 has a mic, a lens, and a ring — and looks like regular glasses. The form factor is finally invisible enough for a boardroom.

The regulations tightened

Updated HIPAA enforcement, attorney-client privilege rulings, and AI safety executive orders made cloud AI in sensitive environments legally radioactive. The demand isn't hypothetical — it's compliance-driven.

The Founder

Ben Shyong

Hardware TPM, Meta Reality Labs (AR/VR)

A decade in consumer electronics manufacturing. 6 years at Apple shipping hardware programs used by hundreds of millions. 4 years at Meta building the next generation of AR/VR devices.

benjamin.shyong@gmail.com

Open Source

Marginalia is Apache 2.0 licensed. The full source code, architecture, and demo scripts are on GitHub.

View on GitHub

Built with

Gemma 4 E2B (Google DeepMind) Cactus Compute (YC S25) Even Realities G2