AI’s Wrestle to Learn Analogue Clocks Might Have Deeper Significance

A brand new paper from researchers in China and Spain finds that even superior multimodal AI fashions similar to GPT-4.1 battle to inform the time from photos of analog clocks. Small visible modifications within the clocks may cause main interpretation errors, and fine-tuning solely helps with acquainted examples. The outcomes elevate issues concerning the reliability of those fashions when confronted with unfamiliar photos in real-world duties.

 

When people develop a deep sufficient understanding of a website, similar to gravity or different primary bodily rules, we transfer past particular examples to know the underlying abstractions. This permits us to use that data creatively throughout contexts and to acknowledge new situations, even these we have now by no means seen earlier than, by figuring out the precept in motion.

When a website carries sufficient significance, we might even start to understand it the place it doesn’t exist, as with pareidolia, pushed by the excessive price of failing to acknowledge an actual occasion. So robust is that this pattern-recognizing survival mechanism that it even disposes us to find a wider range of patterns the place there are none.

The sooner and extra repetitively a website is instilled in us, the deeper its grounding and lifelong persistence; and one of many earliest visible datasets that we’re uncovered to as kids comes within the type of teaching-clocks, the place printed materials or interactive analog clocks are used to show us how you can inform time:

Although changing fashions in watch design might generally problem us, the resilience of this early domain-mastery is kind of spectacular, permitting us to discern analogue clock faces even within the face of complicated or ‘eccentric’ design selections:

People do not need thousands of examples to learn the way clocks work; as soon as the fundamental idea is grasped, we are able to acknowledge it in nearly any type, even when distorted or abstracted.

The problem that AI fashions face with this job, against this, highlights a deeper difficulty: their obvious energy might rely extra on high-volume publicity than on understanding.

Past the Imitation Sport?

The stress between surface-level efficiency and real ‘understanding’ has surfaced repeatedly in current investigations of huge fashions. Final month Zhejiang College and Westlake College re-framed the query in a paper titled Do PhD-level LLMs Actually Grasp Elementary Addition? (not the main focus of this text), concluding:

‘Regardless of spectacular benchmarks, fashions present crucial reliance on sample matching relatively than true understanding, evidenced by failures with symbolic representations and violations of primary properties.

‘Express rule provision impairing efficiency suggests inherent architectural constraints. These insights reveal analysis gaps and spotlight the necessity for architectures able to real mathematical reasoning past sample recognition.’

This week the query arises once more, now in a collaboration between Nanjing College of Aeronautics and Astronautics and the Universidad Politécnica de Madrid in Spain. Titled Have Multimodal Giant Language Fashions (MLLMs) Actually Discovered to Inform the Time on Analog Clocks?, the new paper explores how nicely multimodal fashions perceive time-telling.

Although the progress of the analysis is roofed solely in broad element within the paper, the researchers’ preliminary checks established that OpenAI’s GPT-4.1 multimodal language mannequin struggled to accurately learn the time from a various set of clock photos, typically giving incorrect solutions even on easy circumstances.

This factors to a potential hole within the mannequin’s coaching knowledge, elevating the necessity for a extra balanced dataset, to check whether or not the mannequin can really study the underlying idea. Due to this fact the authors curated an artificial dataset of analog clocks, evenly protecting each potential time, and avoiding the same old biases present in web photos:

Earlier than fine-tuning on the brand new dataset, GPT-4.1 persistently did not learn these clocks. After some publicity to the brand new assortment, nevertheless, its efficiency improved – however solely when the brand new photos seemed like ones it had already seen.

When the form of the clock or the type of the palms modified, accuracy fell sharply; even small tweaks, similar to thinner palms or arrowheads (rightmost picture under), have been sufficient to throw it off; and GPT-4.1 struggled moreover to interpret Dali-esque ‘melting clocks’:

The authors deduce that present fashions similar to GPT-4.1 might due to this fact be studying clock-reading primarily via visible sample matching, relatively than any deeper idea of time, asserting:

‘[GPT 4.1] fails when the clock is deformed or when the palms are modified to be thinner and to have an arrowhead. The Imply Absolute Error (MAE) within the time estimate over 150 random instances was 232.48s for the preliminary clocks, 1380.69s when the form is deformed and 3726.93s when palms are modified.

‘These outcomes counsel that the MLLM has not realized to inform the time however relatively memorized patterns.’

Sufficient Time

Most coaching datasets depend on scraped net photos, which are likely to repeat sure instances – particularly 10:10, a popular setting in watch advertisements:

Because of this restricted vary of instances depicted, the mannequin may even see solely a slim vary of potential clock configurations, limiting its capability to generalize past these repetitive patterns.

Concerning why fashions fail to accurately interpret the distorted clocks, the paper states:

‘Though GPT-4.1 performs exceptionally nicely with commonplace clock photos, it’s stunning that modifying the clock palms by making them thinner and including arrowheads results in a major drop in its accuracy.

‘Intuitively, one may anticipate that the extra visually complicated change – a distorted dial –would have a better affect on efficiency, but this modification appears to have a comparatively smaller impact.

‘This raises a query: how do MLLMs interpret clocks, and why do they fail? One chance is that thinner palms impair the mannequin’s capability to understand path, weakening its understanding of spatial orientation.

‘Alternatively, there might be different components that trigger confusion when the mannequin makes an attempt to mix the hour, minute, and second palms into an correct time studying.’

The authors contend that figuring out the basis trigger of those failures is vital to advancing multimodal fashions: if the problem lies in how the mannequin perceives spatial path, fine-tuning might supply a easy repair; but when the issue stems from a broader problem in integrating a number of visible cues, it factors to a extra basic weak spot in how these programs course of data.

Superb-Tuning Checks

To check whether or not the mannequin’s failures might be overcome with publicity, GPT-4.1 was fine-tuned on the aforementioned and complete artificial dataset. Earlier than fine-tuning, its predictions have been broadly scattered, with important errors throughout all varieties of clocks. After fine-tuning on the gathering, accuracy improved sharply on commonplace clock faces, and, to a lesser extent, on distorted ones.

Nevertheless, clocks with modified palms, similar to thinner shapes or arrowheads, continued to supply massive errors.

Two distinct failure modes emerged: on regular and distorted clocks, the mannequin sometimes misjudged the path of the palms; however on clocks with altered hand kinds, it typically confused the perform of every hand, mistaking hour for minute or minute for second.

This implies that the mannequin had realized to affiliate visible options like hand thickness with particular roles, and struggled when these cues modified.

The restricted enchancment on unfamiliar designs raises additional doubts about whether or not a mannequin of this sort learns the summary idea of time-telling, or merely refines its pattern-matching.

Hand Indicators

So, though fine-tuning improved GPT-4.1’s efficiency on typical analog clocks, it had far much less affect on clocks with thinner palms or arrowhead shapes, elevating the chance that the mannequin’s failures stemmed much less from summary reasoning and extra from confusion over which hand was which.

To check whether or not accuracy may enhance if that confusion have been eliminated, a brand new evaluation was carried out on the mannequin’s predictions for the ‘modified-hand’ dataset. The outputs have been divided into two teams: circumstances the place GPT-4.1 accurately acknowledged the hour, minute, and second palms; and circumstances the place it didn’t.

The predictions have been evaluated for Mean Absolute Error (MAE) earlier than and after fine-tuning, and the outcomes in comparison with these from commonplace clocks; angular error was additionally measured for every hand utilizing dial place as a baseline:

Complicated the roles of the clock palms led to the most important errors. When GPT-4.1 mistook the hour hand for the minute hand or vice versa, the ensuing time estimates have been typically far off. In distinction, errors brought on by misjudging the path of a accurately recognized hand have been smaller. Among the many three palms, the hour hand confirmed the best angular error earlier than fine-tuning, whereas the second hand confirmed the bottom.

To deal with directional errors alone, the evaluation was restricted to circumstances the place the mannequin accurately recognized every hand’s perform. If the mannequin had internalized a normal idea of time-telling, its efficiency on these examples ought to have matched its accuracy on commonplace clocks. It didn’t, and accuracy remained noticeably worse.

To look at whether or not hand form interfered with the mannequin’s sense of path, a second experiment was run: two new datasets have been created, every containing sixty artificial clocks with solely an hour hand, pointing to a special minute mark. One set used the unique hand design, and the opposite the altered model. The mannequin was requested to call the tick mark that the hand was pointing to.

Outcomes confirmed a slight drop in accuracy with the modified palms, however not sufficient to account for the mannequin’s broader failures. A single unfamiliar visible function appeared able to disrupting the mannequin’s general interpretation, even in duties it had beforehand carried out nicely.

Conclusion

Whereas the paper’s focus could seem trivial at first look, it doesn’t particularly matter if vision-language fashions ever study to learn analog clocks at 100% accuracy. What offers the work weight is its deal with a deeper recurring query: whether or not saturating fashions with extra (and extra numerous) knowledge can result in the type of area understanding people purchase via abstraction and generalization; or whether or not the one viable path is to flood the area with sufficient examples to anticipate each possible variation at inference.

Both route raises doubts about what present architectures are really able to studying.

 

First revealed Monday, Might 19, 2025