Engineered Boredom: Do Intrinsic-Motivation Schedulers Make LLM Research Agents Explore Better?

1. Motivation

In 1991 Jürgen Schmidhuber described a neural controller that builds a model of its environment and is rewarded for steering into situations where that model predicts badly [1]. As the model learns a region, its errors there shrink, the reward disappears, and the controller turns elsewhere. The title of the paper named the two sides of that mechanism: curiosity and boredom. In a second paper the same year he moved the reward from the model's errors to its improvements, so that the agent would be drawn neither to pure noise, which can never be learned, nor to what it already predicts perfectly [2]. In that line of work, boredom is not a mood. It is what learning progress looks like when it stops.

Thirty-four years later, our June 2025 post introduced Curious3 as "an AI system that doesn't just process information - it wonders", one that "gets bored with repetitive tasks, feels 'pain' when facing contradictions, and experiences genuine curiosity about the unknown". The words came from the intrinsic-motivation literature. The mechanism did not. Curious3 has no world model whose errors could fall. Its curiosity, pain and boredom are three numbers updated by hand-written formulas, and the agent with the largest number gets the next turn.

That design may still be useful. Research agents that search, summarise and prototype are a real product category, and every one of them needs a scheduler that decides what to do next under a budget. Whether a scheduler with drive-like variables beats a round-robin or a random choice is an empirical question, and an answerable one. The 2025 post did not ask it. This paper does.

2. What Curious3 is

The design is a sensible piece of agent plumbing. The memory split is conventional, the sandbox for generated code is the right call, and the per-call LLM logging, to the console and to logs/llm_interactions/llm_interactions_20250710.jsonl, is what made Section 3.5 possible. The problems are in what the post said about it.

3. What the 2025 post claimed, and what the code does

3.1 The drives are numbers

The post promised "artificial emotions" that are "not the cheesy, programmed responses we see in chatbots, but genuine computational drives". This is the curiosity update, transcribed from the editor visible in Figure 1:

def update(self, context: Dict[str, Any]) -> float:
    """Update curiosity level based on context"""
    # Calculate components
    novelty_score = self._calculate_novelty(context)
    contradiction_score = self._calculate_contradiction(context)
    entropy_score = self._calculate_entropy(context)
    time_score = self._calculate_time_pressure()

    # Combine scores with weights from config
    from src.config import config
    weights = config.hormonal_drives

    total_score = (
        weights.novelty_weight * novelty_score +
        weights.contradiction_weight * contradiction_score +
        weights.entropy_weight * entropy_score
    )

    # Add time pressure
    total_score += time_score * 0.1

    # Apply bounded update
    delta = (total_score - self.current_level) * 0.3  # Smooth changes
    self.current_level = np.clip(self.current_level + delta, 0.0, 1.0)

Curiosity is an exponential moving average of a weighted sum of four hand-made scores. That is a reasonable way to smooth a priority signal. It is also exactly a programmed response. Calling it a "hormone" adds nothing a reader can check, so we call the three variables what they are: drive variables of a scheduler.

3.2 Novelty is a string comparison

The post said the system "measures information entropy and pattern deviation to identify truly novel information". This is the novelty function as printed in the 2025 post:

def _calculate_novelty(self, context: Dict[str, Any]) -> float:
    """Calculate novelty score from recent discoveries"""
    recent_discoveries = context.get("recent_discoveries", [])
    if not recent_discoveries:
        return 0.3  # Base novelty hunger

    # Count truly new topics
    new_topics = 0
    for discovery in recent_discoveries[-5:]:  # Last 5 discoveries
        topic = discovery.get("topic", "")
        if topic and topic not in self.recent_topics:
            new_topics += 1
            self.recent_topics.append(topic)
    # ...
    novelty_score = min(1.0, new_topics * 0.3 + 0.2)

A topic is new if its label, as a string, is not among the last 20 labels. "Quantum computing" and "quantum computing" are two topics; "quantum computers" is a third. Two labels for the same idea count as novelty, and one label covering two different findings counts as repetition. The function has two further quirks. Having no discoveries at all scores 0.3, higher than five discoveries that are all repeats (0.2). And the function appends to its own history while it measures, so calling it twice on the same context gives two different answers. Comparing embeddings, which the system already computes for its long-term memory, would be the obvious fix; Section 5 tests whether it matters.

3.3 Pain is a clock

def _calculate_contradiction_pain(self, context: Dict[str, Any]) -> float:
    # ...
    for contradiction in contradictions:
        if not any(issue["content"] == contradiction for issue in self.unresolved_issues):
            self.unresolved_issues.append({
                "type": "contradiction",
                "content": contradiction,
                "timestamp": datetime.now(),
                "attempts": 0
            })
    # ...
    for issue in unresolved_contradictions:
        age_hours = (datetime.now() - issue["timestamp"]).total_seconds() / 3600
        # Pain increases with age and failed attempts
        issue_pain = min(1.0, (age_hours * 0.1 + issue["attempts"] * 0.2))
        total_pain += issue_pain

A contradiction is new unless its text matches an earlier one character for character, and its "pain" rises linearly with wall-clock time until it saturates after ten hours, whatever the contradiction is about. The post called this "a computational model of intellectual suffering". It is a timer with a cap.

3.4 The "emergent" follow-up is an if-statement

# If search was successful, potentially follow up with theory generation
if search_result.get("success") and self.hormonal_system.curiosity.current_level > 0.7:
    theory_context = context.copy()
    theory_context["recent_searches"] = [search_result]
    theory_thought = await self.theory_agent.think(theory_context)
    theory_result = await self.theory_agent.act(theory_thought)

The post described this as the system following up searches with theory generation "when curiosity is high enough (> 0.7)", and then called it "a behavior that emerges from the interaction of drives and logic". It is a threshold we wrote. A side effect is visible in the run log: in the four cycles of Figure 1 the Theory Agent, and with it the DeepSeek API, is never called. The single curiosity cycle ends right after its search, so either curiosity was still at or below 0.7, having started at 0.5, or the search did not count as a success.

3.5 The one run on record: boredom writes A* three times

The terminal in Figure 1 shows the first four cycles of a one-hour run. Transcribed and shortened where marked:

11:14:17 | Cycle 1: Dominant drive is curiosity
11:14:17 | SearchAgent -> Perplexity/sonar-pro [user]: What are the most surprising recent
           discoveries in unsolved problems in computer science?
11:14:24 | Completed cycle 1 in 7.78s
11:14:48 | Cycle 2: Dominant drive is boredom
11:14:48 | BuildingAgent -> Claude_Code_CLI/claude [prompt]: Please help me Create a Python
           program that demonstrates an interesting algorithmic concept [...]
11:20:58 | BuildingAgent_20250710_111448_590322 <- [...] I've created a complete Python
           implementation of the **A* pathfinding algorithm** [...]
11:20:58 | Completed cycle 2 in 369.73s
11:21:22 | Cycle 3: Dominant drive is boredom
11:21:22 | BuildingAgent -> Claude_Code_CLI/claude [prompt]: Please help me Create a Python
           program that demonstrates an interesting algorithmic concept [...]
11:28:50 | BuildingAgent_20250710_112122_919548 <- [...] I've created a complete Python
           implementation of the A* pathfinding algorithm [...]
11:28:50 | Completed cycle 3 in 447.14s
11:29:14 | Cycle 4: Dominant drive is boredom

The drive designed to break repetition dominated three cycles in a row. Each time it sent the code agent the same generic request, and each time the code agent answered with the same algorithm. The workspace list on the left of Figure 1 shows a third instance: the folder created at the start of cycle 4, project_20250710_112914, contains a file named astar.py, below cycle 3's astar_pathfinding.py, astar_simple.py and demo_astar.py. The post described the Building Agent as one that "designs experiments with controls, variables, and measurable outcomes". The draft even lists A* among the projects the agent proposed "not from a predetermined list, but as natural responses to its current state of curiosity". In the log, its request is a fixed sentence and its output is A*.

The log also puts a price on scheduling. A search cycle took 7.8 seconds; the two build cycles took 370 and 447 seconds. A scheduler that prefers building is a scheduler that spends roughly fifty times as much wall-clock time per cycle, which is why Section 5 measures everything per euro.

3.6 The "results" in the unpublished draft

The published post ends before any results, apart from one sentence saying that the running system showed behaviour that "seemed" driven by a need to resolve cognitive dissonance. The Markdown draft behind it contains a section headed "Real Results: What the System Actually Discovers". It reports that "when we ran Curious3 for several hours", the system found "fundamental improvements to Shor's algorithm", "practically invisible backdoors in machine learning models", "methane-eating microbes" and other items, and that it "spontaneously" moved from computer science to synthetic biology out of boredom. No log of that run is archived. The items are summaries returned by a search API about work other people published; retrieving them is search, not discovery. The draft also concedes that "the Claude Code CLI integration needed refinement". None of this is a result, and we do not reproduce it as one.

3.7 Other statements

• "Reproducibility: Experiments can be repeated with identical conditions." Docker fixes the runtime of generated code. It does not make LLM outputs repeatable, and nothing in the system logged seeds or model versions for the external APIs.
• "Output Filtering: Generated code is analyzed for potential security issues before execution." No code for this appears in the post or the screenshot. We cannot confirm it existed.
• The post is dated 20 June 2025. The run in Figure 1 took place on 10 July 2025, so the post was published later than its date or changed after publication.

4. Related work

Intrinsic motivation. Loewenstein's information-gap account treats human curiosity as a response to a noticed gap between what one knows and what one wants to know [3]. Oudeyer and Kaplan's typology sorts computational models into families, among them knowledge-based ones, driven by novelty, prediction error or learning progress, and competence-based ones, driven by progress at reaching self-set goals [4]. In deep reinforcement learning, Pathak and colleagues rewarded an agent for the error of a learned model predicting the consequences of its own actions [5]. Burda and colleagues ran purely curiosity-driven agents across 54 environments and showed the limits of prediction-error rewards in stochastic settings, where unpredictable noise looks permanently interesting [6]; their random network distillation bonus, the error in predicting a fixed random network's output, made progress on hard-exploration games such as Montezuma's Revenge [7]. Each of these defines novelty or progress against a model that learns. Curious3's novelty is defined against a list of strings.

LLM agents that explore. Voyager plays Minecraft with an automatic curriculum that proposes tasks to maximise exploration, a growing library of skills written as code, and iterative prompting with self-verification; it collects 3.3 times as many unique items as earlier approaches [8]. The curriculum there has a clear measure of progress, items and milestones, which Curious3 lacks. The AI Scientist chains idea generation, code, experiments, paper writing and an automated review, at a reported cost under 15 dollars per paper [9]. Both papers report outcome metrics per unit of cost or time. That is the standard Curious3 should be held to.

5. Proposed study

5.1 Question and predictions

Under a fixed budget in euros, does a drive-based scheduler make an LLM research agent explore better than simpler schedulers using the same agents? Predictions, stated before data:

• P1. The 2025 drive scheduler does not exceed random scheduling in semantic coverage per euro.
• P2. The 2025 drive scheduler has the highest duplicate rate, because its boredom action sends a fixed prompt (Section 3.5).
• P3. Replacing string-match novelty with embedding-based novelty improves coverage over the 2025 scheduler.

5.2 Conditions

Held constant: agent prompts (except that every condition's build prompt includes the last j project titles, so that no condition is handicapped by the fixed 2025 prompt; S1 is also run once with the original prompt to measure P2), models and versions, seed topic list, memory cleared before each run, a budget of B euros per run at list prices logged per call, and a wall-clock cap of H hours. N runs per condition. Every value written as a symbol in Sections 5.2 and 5.3 is fixed before the first run.

5.3 Measures

• Semantic coverage. Every output item (search summary, theory statement, project README) is embedded with one fixed model. Clusters are fitted once on pooled pilot output and frozen; coverage is the number of clusters a run reaches, per euro.
• Duplicate rate. Share of items whose nearest earlier item in the same run has cosine similarity of at least τ_dup. For build outputs, two annotators also judge whether a project implements the same thing as an earlier one; three A* demos count as two duplicates.
• Verified findings per euro (primary). A finding is an atomic claim with a cited source, or a project with a stated purpose. A reviewer who does not know the condition checks that the source exists and supports the claim, or that the project runs in the sandbox and does what its README says. Verified findings are divided by euros spent.
• Secondary. Share of budget per agent, build success rate, latency per cycle, and a reviewer rating of whether a finding would be new to a well-read practitioner.

5.4 Analysis

Per-run metrics are compared between conditions with permutation tests and bootstrap 95% confidence intervals, with Holm correction across the three primary comparisons (S1 against S2, S3 and S1b). All runs, logs, outputs and reviewer judgements are published.

6. Data residency: what the provider choice means in DACH

Curious3 sends text to four providers: Perplexity for search, DeepSeek for theory generation, Anthropic through Claude Code for building, and OpenAI for embeddings. For a research agent exploring public topics, those prompts contain no personal or confidential data, and the question is mostly academic. It stops being academic the moment a company points the same architecture at its own documents, tickets or customer data, which is where research agents become useful.

DeepSeek's privacy policy states that the company directly collects, processes and stores personal data in the People's Republic of China [10]. On 27 June 2025 the Berlin Commissioner for Data Protection and Freedom of Information reported the DeepSeek app to Apple and Google as illegal content under Article 16 of the Digital Services Act. In her office's assessment, the transfer of German users' data to China violates Article 46(1) GDPR: the EU has issued no adequacy decision for China, Chinese authorities have extensive access rights to data held by Chinese companies, and DeepSeek had not convincingly shown that the data are protected to an equivalent standard [11]. That notice concerned the consumer app, not the API. An API call still sends the prompt to the same operator.

We would not send client data, or anything that might contain personal data, to the DeepSeek API, and the re-run will not use it either. The workable alternatives are an open-weight model hosted in an EU region under a contract we control, or a provider that processes data in the EU. The three US services need their own transfer assessment before they see anything non-public. None of this is a statement about model quality. It is about where the text goes.

8. Reproducibility

Archived with this paper: the run screenshot published with the 2025 post (Figure 1). Code excerpts in Section 3.1 are transcribed from that screenshot; those in Sections 3.2 to 3.4 are quoted from the 2025 post, shortened where marked. The log in Section 3.5 is transcribed from the screenshot with module paths removed.

Not archived: the source code (a project folder named CURIOUS3 with agents, drives, memory and execution packages, visible in Figure 1), the full LLM interaction log of 10 July 2025, the generated projects, and any log of the multi-hour run described in the draft. Senva's public repositories do not contain this project.

Planned for the study: a public repository with the orchestrator, all five schedulers, prompts, the seed topic list, the frozen cluster model, per-call cost logs, all outputs and reviewer judgements.