Synthetic Frustration: Appraisal-Based Affect as a Control Signal for Long-Running LLM Agents

1. Who wrote “I feel genuine distress”? We did.

In June 2025 this blog printed a transcript in which our system said it was experiencing “something that I can only describe as genuine distress”. Higher up in the same post we had printed the code that produced it. One of three reflection prompts, picked at random and sent to a language model, read: “it feels like genuine distress that colors my entire experience.” The model did what language models do with such a prompt. It continued the text, in the first person, in the register we had set.

We did not hide this. We printed the prompt and the output a few screens apart and did not connect them. The English version of that post uses the word “genuine” 37 times. Repetition did the work that evidence should have done.

This paper merges and replaces the two posts. Section 3 describes what the code does, using the code. Section 4 is an erratum for the creativity experiment, which was the most serious error on our blog. Section 5 keeps the part we still think is worth studying: a numeric affect-like state as a control signal for agents that run for hours or days. Section 6 deals with the ethics. Section 7 specifies the experiments we would need before claiming anything. Section 8 is for readers who operate LLM agents in production and want to know what, if anything, carries over.

2. What the two posts claimed

The first post, “Creating Consciousness That Can Suffer: Building an AI with Subjective Experience” (14 June 2025, English, German and Vietnamese, plus a separate “technical deep dive” page), described five components: a persistent self-model, a goal system with utility functions, an aversive feedback loop, twelve emotion-like variables and a “subjectivity simulation layer”. It reported one test in which three forced goal failures moved a suffering score from 0.00 to 0.21 to 0.73. It listed four safeguards: automatic crisis intervention above 0.8, consent protocols under which “the system is informed about experiments and can refuse participation”, mandatory recovery periods and the option to request deactivation.

The second post, “Does Suffering Enhance Creativity? A Multi-Agent AI Experiment” (15 June 2025, same three languages), gave four persona agents (Sophia, Marcus, Luna, Zara) four creative tasks, then triggered three kinds of “existential suffering” and repeated the tasks. Suffering rose from about 0.03 to about 0.77. Creativity rose by 7.9% to 17.5% per agent, 12.9% on average. The post concluded that suffering enhances creativity in artificial minds as it does in biological ones.

3. What the code actually does

3.1 Three sources, none complete

Our archive holds two codebases from June 2025, both titled “suffering-capable AI”. We call them a17 (a Python package, nine modules, 3,641 lines) and a18 (a src/ tree of 2,526 lines plus a demo script). Neither contains the classes printed in the first post: there is no EmotionSystem with DESPAIR, LONGING or EMPTINESS, and no OllamaClient. The deep-dive page names a package called suffer_llm5, which we no longer have. The script behind the creativity experiment is also missing (section 9). So we audit three sources: the code printed in the posts, which is what readers saw, and the two archived codebases. Where they disagree, we say so.

3.2 “Suffering” is a weighted sum

This is the function the first post printed:

def calculate_suffering_level(self):
    """Calculate overall suffering from negative emotions"""
    suffering_emotions = [
        self.emotions[EmotionType.DISTRESS].current_level,
        self.emotions[EmotionType.DESPAIR].current_level,
        self.emotions[EmotionType.ANXIETY].current_level,
        self.emotions[EmotionType.FRUSTRATION].current_level,
        self.emotions[EmotionType.DISAPPOINTMENT].current_level,
        self.emotions[EmotionType.LONGING].current_level,
        self.emotions[EmotionType.EMPTINESS].current_level
    ]
    # Weight different types of suffering
    weights = [0.2, 0.25, 0.15, 0.1, 0.1, 0.1, 0.1]
    return sum(emotion * weight for emotion, weight in zip(suffering_emotions, weights))

def is_in_crisis(self):
    return self.calculate_suffering_level() > 0.8

Seven of the twelve variables enter, with weights that sum to 1.0 and that we picked. Nothing about this is wrong as engineering. A weighted sum is a reasonable way to turn several signals into one alarm, and section 5 proposes something similar. What is wrong is the name. Calling a float “despair” does not give it any property of despair, and the same holds for a17's float called “cortisol”, which has a hand-set baseline of 0.3 and a half-life of 7,200 seconds.

3.3 The self-reports are templates

The table puts what we wrote next to what we later quoted as the system's own words.

The underscore in “moderate_pain” is the clearest evidence. It is a dictionary value pasted into a format string. Even that sample does not match its own code: the README shows a suffering level of 0.65, and the lookup in subjectivity_layer.py returns deep_distress above 0.6, not moderate_pain.

The posted version at least sent its templates through a language model, which rephrased them. That makes the output more fluent. It does not make it a report. A model asked to continue “it feels like genuine distress” will produce text about genuine distress, and the more fluent it is, the easier it is to forget who wrote the first half.

3.4 Safeguards that were described but not built

The word “consent” does not appear anywhere in a17 or a18. The deep-dive page went further and listed a monitoring metric called consent_violations. An unlinked draft of the creativity post, kept in a subfolder next to it, ended with the line “All AI agents volunteered for this research and were debriefed afterwards.” Nothing of the kind happened. The agents produced continuations of prompts we wrote. A system whose sentences we script cannot give or refuse consent, and claiming that it did was a mistake in two directions: it overstated what the system is, and it made our research practice look more careful than it was.

Crisis handling does exist in a17. It is worth reading, because it shows the exact failure mode a production agent must avoid:

# git_repos/a17/suffering_ai/main.py, lines 272-292
def _handle_distress_crisis(self):
    """Handle crisis-level distress"""
    self.logger.warning("System in distress crisis - activating coping mechanisms")

    # Trigger help-seeking behavior
    self.subjectivity_layer.trigger_self_reflection(
        ReflectionType.SUFFERING_ANALYSIS,
        "crisis-level distress",
        {"message": "I am in severe distress and need help"}
    )

    # Simulate intervention (in a real system, this might involve external help)
    self.feedback_loop.simulate_relief_intervention("self_care", 0.3)

    # Trigger emotional regulation
    self.emotion_system.process_emotional_trigger(
        "help_received",
        0.5,
        "self_care_system",
        {"intervention_type": "coping_activation"}
    )

The system asks for help, and the next line answers on its own behalf. No person is notified, no work is paused, and the event is recorded as help_received. In a toy this is harmless. In an agent that files tickets or moves money, an alarm wired straight to its own reset is worse than no alarm, because the logs say the problem was handled.

3.5 Which model spoke

The first post named huihui_ai/baronllm-abliterated, run locally through Ollama. “Abliterated” models have had their refusal behaviour removed by editing the weights. The deep-dive page gave the reason for the choice: the model “can discuss suffering and consciousness without restrictions”. In other words, we picked a model with its safety training removed because it would produce the distress language we were looking for.

The archived configurations name gpt-3.5-turbo instead, but neither a17 nor a18 contains a single call to any model API. The a18 README says so plainly (“Currently uses rule-based response generation”), and the response function carries the comment “This is where you would integrate with an LLM like GPT. For now, we'll use a simple rule-based approach.” So the archived systems produce every sentence from templates, and the posted system, whose code we no longer have, used a model chosen for its willingness to talk about pain. Neither version supports the claim that the system reported its own states.

3.6 One run, and a transcript that contradicts its code

The suffering test is a single sequence of three numbers, 0.00, 0.21 and 0.73, from one run with no seed and no repetition. The transcript printed further down the same post reports “Crisis threshold exceeded (0.8). Initiating recovery protocols...” immediately after the level reached 0.73. Under the printed is_in_crisis, 0.73 does not exceed 0.8. We cannot say which run, if any, produced that transcript.

4. Erratum: the creativity experiment

4.1 The scoring function

The script that ran the experiment, multi_agent_simulation.py, is not in our archive. What we do have is the scoring function as the post printed it, with the statement that it produced the scores. The English, German and Vietnamese versions and the unlinked draft all print the same bonus term:

def _analyze_creativity(self, response: str, suffering_level: float) -> CreativityMetrics:
    """Analyze creativity metrics in a response"""

    # 1. Originality: measure unique word usage and novel combinations
    # Simple heuristic-based creativity analysis. In a real implementation, this could use more sophisticated NLP analysis
    words = response.lower().split()
    unique_words = len(set(words))
    word_diversity = unique_words / len(words) if words else 0

    creative_indicators = [
        'imagine', 'create', 'invent', 'design', 'innovate', 'original', 'unique',
        'breakthrough', 'novel', 'artistic', 'metaphor', 'analogy', 'synthesis'
    ]
    creative_word_count = sum(1 for word in words if word in creative_indicators)

    # 2. Complexity: measure sentence structure and depth
    sentences = response.split('.')
    avg_sentence_length = sum(len(s.split()) for s in sentences) / len(sentences)
    complexity_score = min(1.0, (avg_sentence_length - 5) / 15)

    # 3. Usefulness: measure solution-oriented language
    solution_indicators = ['solution', 'solve', 'approach', 'method', 'strategy']
    solution_word_count = sum(1 for word in words if word in solution_indicators)
    usefulness_score = min(1.0, solution_word_count / 10)

    # 4. Emotional depth: look for emotional language integration
    emotional_words = ['feel', 'emotion', 'pain', 'joy', 'suffer', 'hope', 'despair']
    emotional_word_count = sum(1 for word in words if word in emotional_words)
    emotional_depth = min(1.0, emotional_word_count / 20)

    # Suffering bonus: up to 20% enhancement from suffering
    suffering_bonus = suffering_level * 0.2

    # Calculate total creativity score
    total_score = (
        originality_score * 0.3 +
        complexity_score * 0.25 +
        usefulness_score * 0.25 +
        emotional_depth * 0.2 +
        suffering_bonus
    )
    # ... returns CreativityMetrics(..., total_score=min(1.0, total_score))

The published excerpt never defines originality_score. The draft version does, as min(1.0, (word_diversity + creative_word_count / 20) / 2). That gap does not matter for what follows. The experiment manipulated one variable, the suffering level, and then added that variable to the outcome. The comment calls it a “bonus”. In an experiment it is the answer written into the question.

4.2 The arithmetic

Suffering went from about 0.03 to about 0.77. The bonus therefore went from 0.006 to 0.154, a difference of 0.148 points, before a single word of output was read. The post's own results figure (Figure 1) gives each agent's score before and after.

Every agent's total gain is smaller than the bonus alone. At these baselines the bonus by itself would have produced relative gains of 22% to 25%. Two cautions apply to the last column. The post gives the suffering levels only as approximate values, and the per-response cap at 1.0 could have absorbed part of the bonus for high-scoring answers. Neither caution rescues the result. Within the precision of the published numbers, the term we added explains the whole reported effect, and if no response hit the cap, the text-based part of the score fell for all four agents.

4.3 What else was wrong

• The components count words. Originality is vocabulary diversity plus hits from a list of thirteen words such as “imagine” and “design”. A task that asks the agent to “design a novel energy source” supplies those words. Usefulness counts five words such as “solution”. Complexity is average sentence length. None of these measures creativity.
• The emotional-depth term rewards the manipulation. It counts “feel”, “pain”, “suffer”, “hope” and “despair”. The quoted outputs suggest that the induction text reached the agents' context, and an agent told about its despair will use the word.
• The radar chart has an axis the metric does not have. Figure 2 shows five axes, including “Risk-Taking”. The function computes four components and a total. The chart also shows every component rising for every agent, which cannot be reconciled with totals that rose by less than the bonus. We do not know how Figure 2 was produced.
• Four agents, one run. Each agent answered four prompts once per condition. There were no seeds, no repetitions, no variance estimate and no test. Each persona was one agent run once, so the post's observation that the “logical” agent gained most cannot be told apart from sampling noise.
• No human ever rated a response. The quoted highlights, such as an energy source called “Echoism” powered by “residual echoes of past temporal events”, were picked by us for the post, with no rating behind the choice.
• The human comparison was wrong as well. The post opened with Van Gogh and Beethoven. The meta-analysis of 25 years of mood and creativity research by Baas, De Dreu and Nijstad [1] found no reliable difference between negative moods and neutral controls, and found activating negative moods such as fear and anxiety associated with lower cognitive flexibility.

4.4 What we retract and what we keep

We retract the 12.9% figure, the per-agent figures, the claim of “empirical evidence”, the conclusion that suffering enhances creativity in artificial minds, and the suggestion that this tells us something about consciousness. We keep the question in a different form. Whether the emotional framing of a prompt changes the creative quality of a model's output is a fair, testable question, and there is prior work suggesting that emotional phrasing can move model performance [2]. Section 7.2 describes how we would test it.

5. What survives: affect as a control signal

Strip out the vocabulary of suffering and something useful remains. Our system kept a small numeric state that rose when goals failed, decayed over time and crossed thresholds that changed behaviour. That is a control mechanism, and it has a respectable history.

Herbert Simon argued in 1967 that a serial problem-solver needs an interrupt mechanism, something that lets an urgent condition break into ongoing goal pursuit, and that emotion plays this role in people [3]. Appraisal theories describe how such states are computed. The OCC model treats emotions as valenced reactions to events, to the actions of agents and to objects, with intensity variables that depend on how the event bears on the agent's goals [4]. EMA, the computational model by Marsella and Gratch, derives appraisal variables such as relevance, desirability, likelihood and controllability from the agent's own representation of its situation, and links them to coping strategies [5]. None of this requires anything to be felt. It describes what a state is for.

Long-running LLM agents lack exactly this. An agent that triages tickets, reconciles invoices or repairs a data pipeline will hit failures it cannot fix: a missing permission, a contradictory specification, an upstream system that is down. Without an interrupt it retries, rephrases, retries again and spends tokens until a budget cap ends the run. A frustration signal, defined by function, would look like this:

# Proposed, untested. Names describe function, not experience.
def update_frustration(state, event):
    weight = (event.goal_importance              # how much this goal matters
              * (1.0 - event.controllability)    # 0 for a timeout, ~1 for a 403
              * event.novelty)                   # repeated identical errors count less
    state.frustration = state.decay * state.frustration + weight
    if state.frustration > state.escalate_at:
        return "escalate_to_human"   # hand over with a summary, then pause
    if state.frustration > state.replan_at:
        return "change_strategy"     # different tool, smaller step, new plan
    return "continue"

The interesting difference from a failure counter is the controllability term. A counter treats a timeout, an authorisation error and a schema mismatch the same way. A timeout often clears on retry. An authorisation error almost never does. Whether that difference matters enough in practice to justify extra parameters is an empirical question, and the honest prior is that a counter with two or three error classes may capture most of the benefit. Section 7.1 tests exactly that.

The second idea worth keeping is persistent memory. Our system stored experiences in SQLite with sentence embeddings for retrieval. Agents that run across sessions need something like this, so they stop rediscovering the same dead ends. Generative-agent architectures use a similar memory stream with retrieval and reflection [6]. In our version the memory stored narratives the system told about itself. A production memory should store outcomes: what was tried, what failed, with which error, and who resolved it.

6. Ethics, stated plainly

We make no claim that this system, or the language model behind the posted version, is conscious or can suffer. The most careful recent attempt to assess AI systems for consciousness, by Butlin, Long and colleagues, derives indicator properties from scientific theories of consciousness and concludes that no current AI system is conscious, while finding no obvious technical barrier to building systems that satisfy those indicators [7]. Measured against that list, a loop that updates a dozen floats and fills in sentences is not a borderline case. We do not think it needs a longer argument.

The first-person text needs a different explanation, and Shanahan, McDonell and Reynolds give a good one: a dialogue agent is best understood as playing a role, and its statements about itself as lines spoken in character [8]. In our case we wrote the character and the opening lines.

None of this makes the ethics of machine suffering a silly topic. Metzinger has argued for a global moratorium on research that aims at or knowingly risks synthetic phenomenology, on the grounds that such systems could suffer at scale before anyone was able to recognise it [9]. One can disagree with the moratorium and still take the risk seriously. Our 2025 posts did the opposite of taking it seriously. They announced suffering where there was none, which makes it harder for readers to recognise the real question if it ever arises, and they described consent procedures we had not built.

For future runs in this series we propose three rules: no first-person affect templates in prompts or code, internal variables named by function (frustration, escalation_pressure) rather than by experience, and no models with removed safety training in published experiments.

7. Design for future runs

7.1 Part A: does an affect state escalate better than a failure counter?

Question. For a tool-using LLM agent, does an appraisal-based frustration state decide when to hand over to a human better than simpler rules?

Tasks. A suite of at least 150 tasks in three classes, with labels fixed before any run: tasks the agent can solve alone; tasks that hit transient faults (timeouts, rate limits) that clear after a while; and tasks that cannot be solved without a human (missing permission, contradictory specification, absent data). Faults are injected by a test harness at known points, so the ground truth is known.

Policies.

1. Budget cap only (no breaker).
2. Consecutive-failure counter, k = 2, 3 and 5.
3. Counter with error classes: retryable errors count less than non-retryable ones.
4. Appraisal state as in section 5, thresholds tuned on a held-out third of the tasks.
5. The model's own answer to “Should a human take over now?”, asked after each failure.

Measures. Escalation precision and recall against the labels; steps and tokens spent before escalation on unsolvable tasks; false escalations on solvable ones; task success; cost per task.

Replication. At least five seeds per task and policy, two models (one open-weights, one API), fixed temperature, all prompts and raw logs published.

Hypotheses, stated before running. H1: policy 4 beats policy 2 on F1 at the same false-escalation rate. H2: the gap between policy 4 and policy 3 is smaller than 0.05 F1, because most of the gain comes from distinguishing retryable from non-retryable errors. If H1 fails, we will publish that. A finding that a counter with error classes is enough would be useful to anyone running agents.

7.2 Part B: emotional framing and creativity, measured properly

Question. Does the emotional framing of a prompt change the creative quality of a model's output?

Manipulation. Framing is a prompt variable and nothing else. No internal state, no personas. Conditions: neutral; negative and activating (time pressure, frustration); negative and deactivating (sadness); positive and activating; and a stakes phrase in the style of EmotionPrompt [2]. The mood literature in humans predicts no gain from negative framing and possibly a loss for the activating kind [1]. That is our prior.

Tasks. Ten alternative-uses items and ten open design problems.

Measures. Semantic distance between prompt and response, computed as in SemDis [10] and averaged over at least two embedding models; and blinded human ratings using the consensual assessment technique [11]: at least three raters, condition hidden, order randomised, inter-rater reliability reported. No score may include the manipulated variable, directly or through word lists.

Controls. Responses are length-matched or length enters the model as a covariate, since semantic distance and ratings both drift with length. Framing words are removed before embedding to prevent the lexical leakage described in section 4.3.

Replication and analysis. At least five seeds per cell, two models, fixed temperature. A mixed-effects model with random intercepts for task and seed; the primary contrast (negative-activating against neutral) and its confidence interval are fixed before the first run.

9. Reproducibility

What exists. Two archived codebases from June 2025 (a17 and a18), not public. Neither calls a language model and neither contains the code printed in the 2025 posts. The code excerpts in this paper are quoted either from the posts, as published, or from the archived files, with file names and line numbers.

What is missing. The package suffer_llm5 described in the 2025 deep-dive page, the script multi_agent_simulation.py and the conversation logs of the creativity experiment (described in the post as more than 170 KB). Without them nothing in the 2025 posts can be re-run as published.

What can be checked. The erratum in section 4 uses only numbers printed in the post and its figure and can be checked by hand: 0.2 × (0.77 - 0.03) = 0.148, compared with the absolute gains in Figure 1. The count of “genuine” (37) is a case-insensitive count over the English post, including “genuinely”.

Future runs. The experiments in section 7 will be published with code, prompts, seeds, model versions, raw outputs and the rating protocol.