What limitations has Google identified?
Genie 4 remains an experimental research model. In its launch announcement, Google lists visual drift and object permanence, spatial accuracy, complex physical interactions, legible text, audio accuracy and longer-term consistency as active areas of research.
These are stated limitations, not failure rates. They do not tell you how often a particular task will succeed, how long your account’s session will last or whether a result meets a production requirement.
Turn a limitation into an observable question
| Area | Suggested observation | What to record |
|---|---|---|
| Object permanence | Look away from an object and return | Identity, count, position and appearance changes |
| Spatial relationships | Follow a short route between landmarks | Route continuity and landmark relationships |
| Physical interaction | Observe one simple collision or contact | Visible sequence, penetration or discontinuities |
| Text | Inspect the same short sign twice | Exact visible characters and changes |
| Audio | Pair one sound with a visible event | Source, onset, duration and uncertainty |
| Scene editing | Change one condition | Requested change and unrelated changes |
These are suggested tests. Their inclusion does not claim that the interface exposes every required control or that the model is expected to pass.
Download the original run log
Download the world-model run log CSV and open it in a spreadsheet. It contains field headings and one example row showing how to record an untested plan. Replace that row with your own observations.
Keep the date, model version, access context, complete prompt and action sequence. Add the expected observation separately from the result. Attach a recording or screenshot reference when your access terms permit it. Use “not assessed” when the event was not observable.
The template is a note-taking aid. It contains no benchmark data, scores or claimed model results. You can also copy it into a private document if a spreadsheet is unnecessary.
Make a small test repeatable
Write down the route before running it: start beside the door, look at the red case, turn toward the window, then turn back. Specify which property matters. You may care about the case’s position without needing pixel-perfect reflections.
Save a baseline before adding scene edits. If you change the weather, viewpoint and number of characters together, an apparent improvement or failure has several possible causes. A smaller question produces a more interpretable result.
Where repetition is possible, keep the observation window and input sequence similar. Record differences in the conditions instead of silently treating every run as equivalent.
Report cases before reporting scores
A few selected clips cannot support a general reliability percentage. If you summarize several tests, explain the task selection, number of runs, success criterion and excluded or unassessable cases. Preserve failures as well as successes.
Do not compare two models using different scene complexity and call the result a measured model advantage. Our Genie 4 vs Genie 3 guide compares published claims and labels the absence of a controlled benchmark.
Simulation is not real-world validation
Visual plausibility does not establish physical accuracy. A generated experiment can suggest a question or a visual direction, but it cannot replace engineering measurements, a validated simulator or a real-world safety evaluation.
For a creative workflow, the useful output may be a camera idea or a prototype discussion. For a task requiring repeatable mechanics or trustworthy predictions, define those requirements separately and validate them with an appropriate system. The game-development guide shows how to turn observations into a concrete handoff.