The marketing name is "hand tracking" or "bare-hand control"; the filing name is hand pose estimation. Magic Leap's granted patent US11775836B2, "Hand pose estimation," issued October 3, 2023, claims a neural-network architecture for reading a hand from camera images — and, tellingly, an architecture built to run on the headset itself. Its CPC tags pair gesture-input (G06F 3/017) and hand-recognition (G06V 40/11) with neural-learning class G06N 3/084.

On the record, bare-hand control is the input modality that competes with controllers, and it lives or dies on pose estimation. The system has to infer an articulated hand from 2D camera views in real time, robust to lighting, occlusion (fingers hiding fingers), and motion. That inference is a deep-learning problem, which is why the claim sits at the intersection of computer vision and neural networks. But the patent's real subject is narrower than "a neural net that sees hands": it is how to make such a network small and fast enough to live on a wearable.

"A neural network in multi-task deep learning paradigm for machine vision includes an encoder that further includes a first, a second, and a third tier. The first tier comprises a first-tier unit having one or more first-unit blocks."— U.S. Patent No. 11,775,836 source

Claim 1 makes the embedded ambition explicit. It is not a method claim in the abstract; it is "a mobile electronic device with an embedded implementation of a neural network," and that implementation includes "a vision processing unit having a plurality of super hi-vision (SHV) processors or processor cores." The encoder's three tiers are arranged so that "each succeeding tier... has a higher network complexity than the immediately preceding tier" — a deliberate ramp that spends compute where it pays off. This is a claim about silicon-aware network design as much as about hand tracking.

The "multi-task" framing is the second key idea. Rather than one network for one output, claim 1 hangs three decoder heads off the shared encoder: a "visibility decoder layer" that determines masks (which parts of the hand are actually seen versus occluded), a "keypose decoder layer" that classifies the predicted pose "from one or more keypoints," and an "orientation decoder layer" that "regularize[s] predictions" using a "categorical orientation loss." Estimating visibility, keypose, and orientation together — sharing one encoder — is how the system stays cheap: the expensive feature extraction happens once, and three lightweight heads read off different answers. That is precisely the trick that lets finger-level tracking fit a headset's power budget.

The dependent claims read like an efficiency checklist for constrained hardware. Claim 4 describes "nonlinearly expanding a receptive field of the neural network without loss of resolution or coverage" using "dilated convolution layers" — a way to see a large area without the cost of downsampling and upsampling. Claim 8 specifies tiers "configured not to include deconvolution layers while maintaining a resolution," and claim 10 groups convolution filters so their count "is equal to a multiple of a total number of lanes of an SHV processor or processor core" — literally shaping the math to match the chip's parallel lanes. Claim 11 adds a "spatial path" for low-level detail and a "context path" for "global context information," fused (claim 12) and re-weighted (claim 16) — a two-stream design that captures both the fine geometry of fingers and the coarse shape of the whole hand.

The two-stream detail in claims 11 through 17 is worth dwelling on, because it maps cleanly onto why hand tracking is finicky in practice. The "spatial path... configured for encoding low-level information" preserves the fine detail needed to tell a fingertip from a knuckle, while the "context path... configured for encoding high-level information" — "global context information," in the claim's words — carries the coarse sense of where the whole hand is and which way it faces. A "fusion block" (claim 12) "concatenates" the two outputs and a "normalization block... balances a plurality of scales," then a "pooling block" (claim 16) "determines a weight vector... and re-weighs the concatenated data." That weighting is an attention-like mechanism: the network learns, per frame, whether to trust the detail stream or the context stream — useful precisely when fingers occlude each other and local detail becomes ambiguous, so the system can lean on global hand shape to resolve what it cannot see directly. Claim 15 even notes the attention layers "refine predictions... without up-sampling," another nod to keeping the compute budget flat.

Why this is the harder path: controllers report their own position directly via onboard sensors; hands carry no sensors and must be reconstructed entirely from the outside, under self-occlusion and changing light. Getting finger-level precision good enough to pinch, point, and type in mid-air, while running on the headset rather than offloading to a phone or cloud, is a substantial vision-and-systems achievement — and the IP around it is correspondingly valuable.

Novel, or just renamed? Hand-pose estimation is an active research area with a broad literature, and multi-task encoders and dilated convolutions are known building blocks. What is claimed is the specific assembly — a three-tier complexity-ramped encoder, three task-specific decoders for visibility, keypose, and orientation, and a set of choices (no deconvolution, lane-matched grouping, dual spatial/context paths) tuned for embedded "SHV" hardware. The field is old; this particular efficiency-first architecture is the claimed work.

The strategic frame is that input is the headset platform moat, and the controller-versus-hands question is unsettled. Magic Leap — a pure-play AR company — holding hand-pose IP written around on-device execution reflects a bet that bare-hand interaction has to run locally, with no latency budget for a round trip, a bet Meta and Apple also hedge in their own headsets.

Follow the filing, not the demo. When a headset lets you control it with your bare hands, the enabling work is hand-pose estimation like this 2023 grant — a multi-task deep-learning reconstruction of your fingers, architected to run on the device's own vision processor, dated and classified.